Reading the Evidence

Scientists love long, complicated words, and journalists love to make things sound bigger than they are. So here is a quick guide to demystifying a scientific paper.

Everything in the Lab points at a source you can open. This page is about what to do once you have opened it, because a citation is only worth something if you can tell whether it holds.

Start with the abstract, and read it backwards

The abstract is the summary at the top of almost every paper, and it is almost always free even when the full paper is not. Sometimes it has its own subheadings; usually it has four rough parts:

1. what they asked (the hypothesis)
2. what they did (the method)
3. what they found (the results)
4. what they think it means (the conclusion)

The conclusion tells you what the authors think. The results tell you what they actually found, without the hot take — and everyone wants their research to mean something, so the conclusion often sounds more interesting than the results do. If the numbers in the results do not obviously add up to the claim in the conclusion, trust the numbers.

Three things to locate every time: how many people (or animals) were in the study, what they were compared against, and whether the difference was measured between two groups or within one group over time. A study with no comparison group is describing, not testing.

The words that are quietly doing the work

Research writing is rarely absolute. To make an absolute statement you need the evidence to back it, and at the edge of research we often do not have it yet. A few phrases worth listening for:

• “Associated with” is not “causes.” It means two things moved together — what scientists mean when they say correlation is not causation. It cannot tell you which caused which, or whether a third thing caused both. People who drink chamomile tea may sleep better; they may also lead calmer lives that leave room for tea.
• “May”, “suggests”, “is linked to” are all softer than they sound in a headline. They usually mark a finding that is real but early, or small, or not yet repeated.
• “In mice”, “in vitro”, “in a cell model” tells you the effect has not been shown in a human. Most things that work in a dish never make it to a person, and the dose is often far beyond anything a body would meet.
• “Statistically significant” is not “large.” It means the effect is unlikely to be pure chance, not that it matters. With enough people, a difference too small to feel becomes significant.

Who was actually in the study

A finding is only about the people it was measured in. Sleep research leans heavily on small samples of young, healthy adults — often students, and often men, because they are the people near a university lab. That does not make the work wrong, but it does mean a result in twelve fit twenty-year-olds might say little about you. When you see a striking claim, the first question to ask is not “is it true?” but “in whom, and how many?”

The percentage that fooled you: relative versus absolute risk

This is the single most common way a real finding is used to mislead, especially when a newspaper picks up a scientist’s hard work. Once you see it, you cannot unsee it.

Say a condition affects one person in every hundred thousand — that is 0.001% of people. A study finds that some exposure doubles the risk. As a headline, “doubles” sounds alarming. But the absolute risk has only gone from one in a hundred thousand to two in a hundred thousand. Your chance was, and remains, vanishingly small.

A percentage change means nothing until you know what it is a percentage of. “Twice the risk” of something that almost never happens is still almost never.

So whenever you meet a relative figure — “50% higher”, “three times more likely” — go looking for the absolute number. A good paper reports both, because scientists should show their data. A press release or news article usually reports only the relative one, because it is the number that travels (and sells). The same trick runs in reverse for benefits: a supplement that improves your odds by 20% has done much less than it sounds if the odds were tiny to begin with.

Not all evidence weighs the same

When two studies disagree, the kind of study usually settles it. A rough ladder, strongest first:

• Systematic reviews and meta-analyses pool many studies and weigh them together. The best single place to look for a general claim — provided the studies they pooled were any good.
• Randomised controlled trials assign people at random to the thing or to a control. The strongest way to test whether one thing causes another.
• Clinical guidelines from bodies like the American Academy of Sleep Medicine or NICE are expert consensus on what someone should actually do.
• Cohort and observational studies watch what happens without intervening. Good for associations, never proof of cause — often the only way to study lifestyle data.
• Single small trials are useful, but one result is a starting point, not a settled fact. Treat it as proof of concept.
• Animal and cell studies are evidence that something could work, not that it does in people.

One thing the ladder does not show: how old a study is matters for fast-moving fields, and barely at all for settled physiology. A 2004 paper on how the vagus nerve responds to a slow exhale is not out of date; a 2015 paper on a brand-new supplement might already be.

PubMed is a library, not a seal of approval

This one catches almost everyone. PubMed, PMC and a university’s .edu or .ac.uk address are places a paper can be stored, not marks of quality. Anything indexed on PubMed passed a basic filter, not a judgement that it is correct or important. Plenty of weak work is indexed there, and there are plenty of predatory journals that will publish almost anything for a fee. What actually decides how much to trust a paper is the study design and the journal’s standing, not the database you found it in. “It’s on PubMed” and “it’s on a government website” are not arguments.

Finding the paper: Google Scholar, briefly

Google Scholar searches the research literature rather than the open web, and it is free. A few habits make it far more useful:

• Sort by date for anything fast-moving, using the left-hand “since 2023” filter — but not for basic physiology, where the foundational paper is often decades old and still correct.
• “Cited by” under each result is a rough measure of how much other researchers leaned on it. A paper cited a thousand times is not automatically right, but it has been looked at.
• Look for the review. Adding the word review or meta-analysis to your search often finds someone who has already pooled the evidence for you.
• Chase the free full text. The link on the right is often a free PDF; if not, the abstract alone usually answers your question — and the authors may have posted a free copy on ResearchGate.

Spotting a press release in a lab coat

Most science you meet has been through at least one retelling, and each retelling changes it. The tells are consistent: a single dramatic number with no comparison; the word “breakthrough”; a claim about people that rests entirely on a study in animals or a petri dish; no link to the actual paper; a conclusion noticeably bolder than the method could support. When a story will not name the journal or link the study, that is usually because the study is thinner than the story. Go and find the paper. If you cannot, treat the claim as a rumour with good production values.

How we choose what to cite here

The same rules point back at us. When we write in the Lab, we try to cite the primary study rather than a news report of it. We prefer systematic reviews and randomised trials for anything we state plainly, and we say “small” or “early” or “mostly in mice” out loud when that is the honest description. We link every reference to a DOI or a permanent record so you can open it yourself. Where the evidence is genuinely thin, we say “nobody knows yet” rather than dress a guess as a finding.

Two things we will not cite: press releases and company literature dressed up as research, and our own products as evidence for a claim. If a sentence in the Lab could be read as marketing, it should not be carrying a citation at all.

That is the whole method. None of it needs a science degree — just the habit of asking, every time: in whom, how many, compared with what, and how big?