How do medical studies work and which ones can you trust?
In another post today we consider the relationship between drinking alcohol and cancer. This post provides some background information on different types of medical studies: how they are conducted, and how reliable the results are.
It is often not possible to identify the exact reason why an individual patient got a specific cancer or another illness in the first place. Doctors therefore have to examine large numbers of people, and try to find common factors in the patients. A famous example is the work linking smoking and lung cancer. Many possible hypotheses were proposed for the causes of lung cancer, but study after study found that by far the most likely reason was smoking. Smokers had significantly higher rates of cancer than non-smokers, whereas no other factor could be shown to have the same impact.
These studies are in effect looking backwards: interviewing people, and trying to find something in their past that differs between patients and non-patients. So you may find that eating more of a particular food is related to higher or lower rates of illness. But this line of reasoning is risky, since it may be that an unknown third factor is involved. For example, people who eat more kale may also be more health-conscious in other ways that you can’t measure. So just eating more kale might have no impact at all unless you change some other behaviours as well.
A more convincing type of study is to follow people for a long period of time; these are called cohort studies. This will usually involve interviews, medical tests, and other observations over many years. And slowly, as some people get ill, and others stay healthy, the researchers may be able to start understanding what the differences between these people are. This is still somewhat speculative, but is usually the best evidence we can have about public health matters. (In an ideal study, people would be randomly fed different foods, made to do different levels of exercise, and so on, over their whole lifetimes, but this is clearly not feasible, and would also not be ethical.)
There are also ways of combining the results of many individual studies together. This is called meta-analysis, and it gives more reliable results than any of the single studies alone, as they incorporate more evidence, and the impact of any small random variation (which is unavoidable in any research) will be much reduced.
It is therefore a good idea to ask a few questions when you read about medical matters in the news:
- Do the results come from a single study? If yes, take with a pinch of salt. The next experiment may well find the opposite to be the case.
- Do the results confirm what has already been discovered earlier? Take any surprising new findings with another pinch of salt.
- How many people were included in the study? More is better.
- Are the results based on several studies conducted over time, in different places, and on lots of different people? If yes, read on. But remember that medicine makes progress all the time, and new findings will come up!