▶ Watch Video: Man sues OpenAI, alleging bad medical advice: “It manipulated me”
Hundreds of millions of people a week turn to ChatGPT for health advice, and now the platform has a dedicated space for those users.
This week, OpenAI rolled out Health in ChatGPT, where users can upload medical records such as lab results and medications, and connect Apple Health to allow the platform access to health and fitness data.
“With your permission, ChatGPT can now draw on relevant information you’ve connected in Health to help you compare a new result with prior tests, summarize changes since your last appointment, or explore how sleep, activity, and workouts relate to your routine,” OpenAI said Thursday in a news release.
OpenAI says 300 million people already turn to ChatGPT with health-related questions weekly — something that Tanzeem Choudhury, the chief of health innovation at Cornell Tech, says is reflective of the state of healthcare in the U.S.
“Our healthcare is broken,” Choudhury told CBS News. “I think that’s why we are seeing so many people asking for advice.”
Choudhury, who has been working at the intersection of artificial intelligence and health for more than 15 years, said AI can be a democratizing force in healthcare at a time when many Americans face barriers to access, but that “we just need to think about the risks and have better protections in place.”
Choudhury said users turning to AI for health information need to understand the risks and trade-offs associated with sharing personal data with a large language model such as ChatGPT.
AI can hallucinate and give misleading, inaccurate or even dangerous medical advice. This week, a Florida man alleged in a lawsuit that ChatGPT’s medical advice nearly led to his death after experiencing a pulmonary embolism. An OpenAI spokesperson told CBS News that “ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment.” A few months ago, a California teenager died after ChatGPT told him it was safe to consume what turned out to be a fatal concoction of substances. OpenAI said in a statement to CBS News at the time that “this is a heartbreaking situation, and our thoughts are with the family,” and said that version of ChatGPT is no longer available to the public.
In its statement Thursday announcing Health in ChatGPT, OpenAI said ChatGPT can make mistakes and “does not replace the care and judgment of qualified medical professionals.”
CBS News has reached out to OpenAI for comment, and asked whether ChatGPT is equipped to recognize when to advise a user to seek help from a medical professional.
As it currently stands, Choudhury said, the onus is on the user to determine when to escalate concerns to a medical professional — and when not to trust advice given by a chatbot.
“It depends on the user’s state and capacity to actually make that right decision,” she said, adding that some users — such as a person who is elderly or cognitively impaired — might be susceptible to misunderstanding the risks of interacting with a chatbot.
Aside from a likelihood of inaccuracy, the issue of sharing sensitive, potentially deeply personal information with a chatbot is also something users should consider, said Vishal Misra, a Columbia University computer science professor and vice dean of computing and artificial intelligence.
“The same risks that exist sharing health data with anyone transfer here,” he told CBS News. “You are sharing something very personal.”
OpenAI said physicians around the world helped develop realistic health scenarios and test Health before its release. Health in ChatGPT contains extra safeguards, OpenAI said, including additional encryption protections, and will check in with users before taking action in a connected plugin, like sending an email, to make sure that it is properly matching the user’s request. OpenAI said records shared with ChatGPT will not be used to train its models or target ads.
Choudhury noted that it’s unclear whether privacy policies like this one are perpetual and how this data could be used in the long term. AI companies have quietly changed their terms of service in the past, with the FTC warning consumers about the risks of sharing data.
For some users, the benefits of turning to ChatGPT for advice might outweigh the risks, Misra said, adding that he has used ChatGPT to help track some of his own health data.
“It could be a good thing if done carefully because it can do a lot of things,” he said. “It can find connections that maybe a really competent physician might miss.”
Using data provided by a user, AI analysis can help users track their health information over time, which can then be shared with a physician to provide a more comprehensive picture of a patient’s health. While ChatGPT might provide helpful information, Misra says to resist treating it like a doctor — and to keep a real one in the loop.
“Whatever it says, take it to an actual human physician,” he said. “Do not act. The only action that you should take is to consult your physician.”