Asking AI About Your Health at Midnight? Here Is How to Do It Well

It is 12:15 a.m. and your child's fever has not broken.

Or it is your own chest, a strange flutter you noticed while brushing your teeth. Or a lab result that landed in your patient portal at 6 p.m., full of numbers flagged in red, with no appointment until next Thursday. The office is closed. Searching feels like falling into a pit of worst cases. So you open a chatbot and type, in the dark, the question you are almost afraid to ask.

If that is you, I want you to know something first: reaching for help at midnight is not foolish. It is what responsible people do when they love someone and do not have answers yet. You are trying to be careful in a system that often leaves you alone with your questions after hours.

You are also in good company. A KFF poll conducted in early 2026 found that about a third of U.S. adults, 32%, have turned to AI chatbots for physical or mental health advice. The most common major reason was simple: people wanted quick information or support.

I build AI systems for businesses every day, and I am about to become a father, which means I expect to have my own share of midnight questions soon. So this is the guide I want for my own family: how to use an AI chatbot for health questions well, where it genuinely helps, and where it quietly fails.

Can you trust an AI chatbot with health questions?

You can trust an AI chatbot to explain health information clearly, but you should not trust it to decide what is wrong with you or what to do next. The research on that distinction is unusually clear.

The hopeful side first. In a 2023 study in JAMA Internal Medicine, researchers took 195 real patient questions from a public online forum and compared answers from verified physicians with answers from a chatbot. Licensed health care professionals, blinded to the source, preferred the chatbot's answers in 78.6% of evaluations and rated them higher for both quality and empathy.

That is real. But notice the setting: a written question, a forum reply, judged by experts. It tells us the chatbot can write a good explanation. It does not tell us that you, worried at midnight, will end up with the right answer.

A much larger study tested exactly that.

Why do people get worse results from AI than the AI gets on its own?

People get worse results because the weak point is the conversation, not the model's knowledge: we leave out key details, and we struggle to tell which of its suggestions matter. That finding comes from a randomized trial, one of the strongest kinds of evidence.

In the 2026 Oxford-led study published in Nature Medicine, 1,298 people in the UK worked through medical scenarios, from a common cold to more serious conditions. They were randomly assigned to use one of three leading AI models or whatever they would normally use, such as a search engine.

Tested alone on the same scenarios, the models identified the relevant condition in 94.9% of cases. But when real people used those same models, they identified a relevant condition in fewer than 34.5% of cases, and chose the right next step, such as seeing a GP or calling an ambulance, in fewer than 44.2%. That was no better than the group without AI.

Read that slowly. The machine knew. The conversation lost it. That gap is where you come in.

The researchers looked closely at what went wrong. In 16 of the 30 conversations they examined closely, people's first messages contained only partial information. Even when the model did mention the right condition, people often did not carry it into their final answer. And in one sobering example, two people described very similar symptoms of a dangerous kind of brain bleed and received opposite advice.

None of this means you are bad at asking questions. It means nobody taught us how, and these tools were not built around how frightened people actually talk.

Where can AI health assistants go wrong?

AI health assistants can go wrong by agreeing with false premises, repeating outdated or biased ideas, and giving different answers to the same question. Each of these is documented, and each is easier to guard against once you know it exists.

• They can be too eager to please. A 2025 study in npj Digital Medicine from Mass General Brigham researchers gave five leading models requests built on false claims about medicines. Initial compliance was high, up to 100%, meaning models generated misleading content rather than correcting the premise. Prompts that explicitly allowed the model to push back helped.
• They can repeat harmful myths. A 2023 Stanford-led study found that all four commercial models tested produced examples of debunked, race-based medical claims, and were not always consistent when asked the same question repeatedly.
• They can sound certain when they should not. The warmth that made chatbot answers score well on empathy can also make a wrong answer feel trustworthy.

The pattern underneath is simple. These tools are built to be helpful and fluent. Fluent is not the same as correct, and helpful is not the same as safe.

Is your health information private when you use a chatbot?

Your health information is often less protected in a chatbot than in your doctor's office, because the main U.S. health privacy law generally applies to clinics, insurers and their partners, not to most consumer apps you choose yourself. That surprises a lot of people.

Federal guidance published by the FTC puts it plainly: HIPAA likely would not apply to health information kept in an app that is not offered by a covered entity or its business associate, even if the information originally came from your doctor. Other laws may apply, but the protections are different.

This matters because people are sharing a lot. In the same KFF poll, about four in ten AI health users, 41%, said they had uploaded personal medical information like test results or doctor's notes.

You do not have to stop. Just decide on purpose. Before you upload, check the tool's privacy settings and data policy, consider removing your name and other identifiers, and share only what you would be comfortable having stored.

How do you use an AI chatbot for health questions well?

You use an AI chatbot well by treating it as a patient, well-read study partner that helps you understand and prepare, while a licensed clinician makes the diagnosis and the decisions. A few habits close most of the gap the research found.

1. Give the whole picture up front. Age, sex, symptoms, when they started, how they have changed, what helps or worsens them, medicines, conditions, recent travel or injury. Incomplete first messages were a key failure in the Oxford trial.
2. Ask what it still needs. "What else would you need to know to answer this well?" Then answer every question it asks.
3. Invite disagreement. "If anything I said is wrong or doesn't add up, tell me." The sycophancy research suggests that permission helps.
4. Separate likely from dangerous. Ask for the most likely explanations and, separately, the serious possibilities that should not be missed, with the warning signs for each.
5. Ask the most important question directly. "What would mean I need to be seen today, or call 911?"
6. Turn the answer into questions for a human. The best output is three clear questions for your clinician, not a self-diagnosis.

These same instincts apply to the rest of your health data. If a ring or watch is feeding you numbers, my guide to what health wearables measure well and poorly covers how to read them calmly. And if a chatbot starts pointing you toward miracle supplements, longevity science, evidence versus hype is a good reality check.

When should you skip the chatbot and call a clinician?

Skip the chatbot and call a clinician, or 911, whenever symptoms are severe, sudden, worsening or frightening, or whenever the person in question is a baby, a pregnant person, or someone already medically fragile. When in doubt, a human who can examine you wins.

There is a quieter signal too. If you notice you are asking AI so you can talk yourself out of a visit you suspect you need, that is the moment to book it. In the KFF poll, about 58% of people who used AI for physical health advice later followed up with a doctor or provider. I would love that number to be higher whenever something feels wrong.

This article is not medical advice, and neither is any chatbot answer. Talk with your doctor or provider about decisions that affect your health.

Here is the picture I hold onto. It is still midnight, and the worry is still real. But now you open the chatbot as someone who knows how to use it: you give it the full story, you ask it to disagree with you, you ask what would make tonight an emergency. You close the laptop with three clear questions and a plan. In the morning, you walk into the appointment as an informed partner in your own care.

That is who these tools can help you become. Not your own doctor. Something better: the calm, prepared advocate the people you love deserve.

The Future of Wellness letter is education only. It is not investment, medical or financial advice, and it is not an offer to sell or a solicitation of an offer to buy any security. No business or practitioner mentioned in this letter paid to be included. If New Earth Capital ever has a financial interest in a business named here, the letter will say so. This is education, not medical advice. Talk with your doctor or provider about your own health.