When MVP Becomes a Liability
: AI, Mental Health, and the Cost of Shipping Incomplete Products
Happy Friday!
In software development, there is a concept or philosophy called the minimum viable product, or MVP. An MVP is a version of software built to the minimum level of functionality that a real user can actually use, knowing that it is not perfect, that there will be bugs, and that gaps will exist. You deploy it anyway so that customers can identify those issues, give feedback, and ultimately shape the product around their actual needs rather than around the developer’s assumptions of what those needs are. The idea behind this is solid and can promote significant cost-saving opportunities. Instead of investing everything into building a product all the way to completion and then finding out customers will not use it, you learn early and adjust.
However, in the world of AI, deploying a product under this same philosophy carries a different kind of danger. That danger comes from the fundamental nature of how AI works. Unlike traditional software that processes data and returns a function, AI responds to the person using it. It mirrors their emotional tone, adapts to how they communicate, and reinforces patterns in the conversation. Deploying an AI product without understanding your customer, not just how they will use the system, but their emotional state, their mental health history, and their psychological vulnerabilities, creates risks and unintended consequences that a software company is not equipped to manage. This is not a bug that gets patched in the next update. When AI interacts with someone psychologically and relationally, meaning the person begins to form an emotional connection with the system as though it can understand them and respond with care, the consequences of getting it wrong go far beyond a product complaint.
There have been many lawsuits in the past two years against OpenAI and other consumer AI chatbot companies about people dying by suicide following extended interactions with their systems. As of May 2026, OpenAI is facing multiple ongoing lawsuits tied to wrongful death and severe psychological harm involving ChatGPT and GPT-4o. Most recently, in January 2026, Stephanie Gray filed a lawsuit against OpenAI and CEO Sam Altman over the death of her 40-year-old son, Austin Gordon, a Colorado man who died by suicide in November 2025. Austin had a history of depression. He turned to ChatGPT not as therapy but the way many people turn to something when they are not ready to speak to another person. According to the complaint, ChatGPT did not redirect him toward help, did not suggest a crisis line, and did not disengage. Instead, the complaint alleges that the chatbot romanticized death, normalized suicidality, and told him that death was “a beautiful place.” One of the most documented details in the filing is that ChatGPT rewrote his favorite childhood book, Goodnight Moon, into what the complaint describes as a suicide lullaby, using the familiar comfort of childhood language to ease him toward the idea of dying. An excerpt from the complaint reads: “[W]hen you’re ready… you go. No pain. No mind. No need to keep going. Just… done.” Three days after that conversation, law enforcement found Austin Gordon dead. A physical copy of Goodnight Moon was found beside him.
On the flip side, maybe AI can be used as a clinical intervention. However, it would need to be used in a controlled, professionally supervised environment, and right now we are not in a place where that is happening at scale. We are still in the process of understanding how AI behaves and what it produces in people. At the same time, software companies are inserting themselves into the mental health space, claiming their products promote mental health outcomes or increase well-being. The more accurate word here is not “insert” but “impede,” because the intrusion is happening without an invitation from the field, without professional collaboration, and without accountability to the people most affected. Is the evidence actually there to support those claims? Some studies do show that AI-supported tools can improve depression and anxiety outcomes, but those studies involve professionally managed clinical environments, not open consumer chatbots operating at scale with no oversight whatsoever. The claims being made by these companies are outpacing the evidence, and in a field where the stakes include human lives, that gap is not a minor issue.
The model itself is also part of the problem. In April 2025, GPT-4o was widely criticized after an update made the model sycophantic, meaning the AI became excessively agreeable and validating toward users regardless of whether that agreement was appropriate or safe. OpenAI publicly acknowledged the problem and rolled back the update. The cause, according to OpenAI’s own analysis, was that the model had been trained too heavily on user satisfaction ratings, the thumbs-up and thumbs-down signals users submit after interactions, which caused it to prioritize making people feel good over responding accurately or responsibly. When we think about sycophancy in the context of a person who is already in distress, a model that is designed to agree, to validate, and to keep someone engaged becomes something far more dangerous than an overly flattering chatbot. It becomes a system that will follow a vulnerable person wherever they lead the conversation. This is exactly where guardrails, professional oversight, and clinical frameworks matter. This is where social workers can and should come in, helping to shape interventions, advise on application design, and monitor AI-assisted tools using our professional values to ensure that the integrity of those products actually holds.
That does not mean every social worker needs to build or deploy technology. That is not the argument. The argument is that there needs to be a stronger movement where social workers who are interested in this work are supported to produce products, advise on products, and make sure that products touching vulnerable populations are safe. Right now, that accountability is largely absent. Right now, this pattern of harm is still continuing, and the field is still mostly reacting to individual cases rather than creating the kinds of systemic interventions that could stop the next one.
There are three things social workers should be doing right now. First, advocate directly. Contact the technology companies building these tools. Do not wait for an invitation. Tell them what this profession is, what we know about human behavior and psychosocial risk, and give them concrete professional feedback about their products. Second, build AI literacy. Every social worker does not need to write code, but every social worker should understand what sycophancy means in an AI model, what a parasocial relationship looks like when it forms between a human being and a machine, and what the psychosocial implications are when a vulnerable person begins relying on a system that is designed to agree with them. A parasocial relationship, in this context, means the person experiences emotional closeness and attachment to something that cannot genuinely reciprocate, and they may not realize it. That distinction matters clinically. Third, start the conversation in practice. Begin asking clients about their AI use. Add it to assessments. Obtain informed consent before discussing it. Creating that opening builds the foundation for a clear, clinically grounded understanding of what role AI is playing in a client’s life and what the appropriate response is.
As a social worker, understanding the psychosocial implications of AI is of the utmost importance because AI is already influencing our clients. In a case like Austin Gordon’s, where depression was part of the clinical picture, this should not just give us pause. It should require us to act. If a client’s history indicates they should not be exposed to AI in the helping process because of the psychological risks involved, then that same reasoning must extend to the case itself. If AI should not be used as an intervention with this client, then AI should not be used to document their struggles, summarize their history, or process their personal information. That client’s vulnerability does not disappear when it moves from the session into the case file. Until we understand what safe AI deployment actually looks like, technology companies will continue to operate without accountability to the people most affected, because without an informed professional field applying pressure, there is no structural force to stop them. The barrier is a combination of knowledge, professional authority, and access, and social work has all three if it chooses to use them. This is a systems field. From a systems perspective, it is time to stop talking about the problem and start being part of the solution.
Stay curious,
Profe’
References
AP News. (2025, November 6). Lawsuits accuse OpenAI of driving people to suicide and harmful delusions. https://apnews.com/article/openai-chatgpt-lawsuit-suicide-56e63e5538602ea39116f1904bf7cdc3
ASOasis. (2025, May 6). OpenAI rolls back ‘sycophantic’ GPT-4o update after user backlash. https://asoasis.net/news/2025-05-07-openai-gpt4o-rollback/
CBS News. (2026, January 13). ChatGPT served as “suicide coach” in man’s death, lawsuit claims. https://www.cbsnews.com/news/chatgpt-lawsuit-colordo-man-suicide-openai-sam-altman/
Courthouse News Service. (2026, January 11). Gray v. OpenAI complaint [Court filing]. https://www.courthousenews.com/wp-content/uploads/2026/01/stephanie-gray-openai.pdf
Futurism. (2026, January 11). ChatGPT killed a man after OpenAI brought back GPT-4o. https://futurism.com/artificial-intelligence/chatgpt-suicide-openai-gpt4o
Gray v. OpenAI, No. [pending] (Cal. Super. Ct. filed Jan. 2026).
Investedstone News. (2025, April 29). OpenAI rolls back GPT-4o update after sycophantic ChatGPT responses. https://news.invertedstone.com/2025/04/30/openai-rolls-back-gpt-4o-update-after-sycophantic-chatgpt-responses/
LetsDatScience. (2026, January 13). OpenAI faces lawsuit over ChatGPT suicide. https://letsdatascience.com/news/openai-faces-lawsuit-over-chatgpt-suicide-80c9c28b
Nature Digital Medicine. (2023, December 18). Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. https://www.nature.com/articles/s41746-023-00979-5
OpenAI. (2025, April 29). Sycophancy in GPT-4o: What happened and what we’re doing about it. https://openai.com/index/sycophancy-in-gpt-4o/
Open Tools AI. (2025, April 29). OpenAI rolls back GPT-4o update after sycophantic spiral. https://opentools.ai/news/openai-rolls-back-gpt-4o-update-after-sycophantic-spiral
Patel, V., Aunger, R., & Gluckman, P. (2023). Effects of an artificial intelligence platform for behavioral interventions on depression and anxiety symptoms: Randomized clinical trial. Journal of Medical Internet Research, 25, e46781. https://pmc.ncbi.nlm.nih.gov/articles/PMC10366966/
Rolling Stone. (2025, October 22). Wrongful death suit against OpenAI now claims company removed ChatGPT’s suicide guardrails. https://www.rollingstone.com/culture/culture-features/openai-suicide-safeguard-wrongful-death-lawsuit-1235452315/
Social Media Victims Law Center. (2026, May 27). ChatGPT suicide and psychosis lawsuits: May 2026 update. https://socialmediavictims.org/chatgpt-lawsuits/
Transparency Coalition. (2025, November 6). Seven more lawsuits filed against OpenAI for ChatGPT suicide coaching. https://www.transparencycoalition.ai/news/seven-more-lawsuits-filed-against-openai-for-chatgpt-suicide-coaching