Science · Inquiry & data skills
Identifying Bias in Research
Good science tries hard to be fair. Your job as a reader is to ask whether a study really was, and to spot the places where it might have tilted toward an answer.
Imagine two headlines about the same drink. One says "Energy drink boosts memory!" The other says "Energy drink fails to improve memory." They cannot both be the full story. Somewhere between the experiment and the headline, something may have nudged the result, and that nudge is what we call bias.
On the CAEC Science test you are not asked to memorize facts about energy drinks, biology, or chemistry. You are asked to evaluate investigations, to read a scenario and judge how trustworthy it is. Spotting bias is one of the most useful skills you can bring to that. Let's build it together.
What "bias" means in research
Bias is anything that pushes a study's result away from the truth in a particular direction. It is not the same as a random mistake. A random error scatters results both ways and tends to cancel out. Bias leans consistently one way, toward the answer someone expected, hoped for, or stood to gain from.
The skill is not to shout "biased!" at every study. It is to ask a calm, specific question: is there a reason this result might be tilted, and in which direction? Here are the four sources of bias you will meet most often.
Four common sources of bias
- Researcher preconceptions. The people running the study already expect a certain answer, so they (often unconsciously) measure, interpret, or report in ways that favour it.
- Funding or interest-group influence. Whoever pays for the study, or has something to win or lose from the result, may shape what gets asked, measured, or published.
- Selective reporting. The study shares the results that support its point and quietly leaves out the ones that do not.
- Unrepresentative samples. The group studied does not reflect the wider population the conclusion is applied to, so the result may not generalise.
Worked example: who paid for the study?
Read the scenario, then we will name the bias and say which way it likely tilts.
A company that sells a herbal sleep supplement funds a study on its own product. The researchers report that 80% of participants "felt they slept better." The company puts this on every bottle. The study did not compare the supplement to a sugar pill, and the people testing it knew they were taking the real product.
Walk through the four questions:
- Who paid? The company selling the supplement, it gains from a positive result. That is a clear interest-group influence.
- Who ran it? Researchers hired by that company, who may expect (and want) a good outcome.
- What was left out? There was no sugar-pill (placebo) comparison, so we cannot tell the supplement apart from the simple expectation of feeling better.
- Who was studied? We are not told, another gap worth noticing.
Reacting to bias: the careful way vs. the sloppy way
Once you spot a possible bias, how you respond matters. Compare two ways of reading the same supplement study.
"The company paid for it, so the supplement definitely does nothing and the whole study is a lie."
This overcorrects. Funding is a reason for caution, not proof the result is false. The reader has swapped one bias for the opposite one.
"The funding source and missing control group give a reason to expect a tilt toward a positive result, so I'd want an independent, placebo-controlled study before trusting the claim."
This names the bias, says which way it likely leans, and asks for stronger evidence, without pretending to know the final answer.
Identifying bias is about weighing evidence, not throwing it out. The strongest answers describe the bias, its likely direction, and what would settle the question.
Worked example: an unrepresentative sample
Now a different scenario, this time with a small data table to read.
A team wants to know whether a new fitness app helps the "average adult" lose weight. They recruit volunteers by posting only in online forums for competitive marathon runners. After 12 weeks they report the average weight change.
| Group recruited | Number of people | Already exercised daily? |
|---|---|---|
| Marathon runners | 200 | Yes |
| Office workers | 0 | N/A |
| Older or less-active adults | 0 | N/A |
Apply the questions. The one that jumps out is who was studied?
- Every participant is already a competitive marathon runner. The conclusion is meant to apply to the "average adult," but the sample contains none.
- People this fit may respond to a fitness app very differently from someone who currently does little exercise. The result cannot be stretched to the whole population.
Worked example: selective reporting
Selective reporting is sneaky because the numbers shown can be perfectly accurate, the bias is in what is hidden. Suppose a company ran the same test six times and only advertises the run that looked best.
Five of the six trials show a tiny improvement of 1–4%. Trial 6 shows 19%. The advertisement quotes only Trial 6. Each number is real, but choosing the single best result and hiding the rest is selective reporting.
A four-question checklist for any study
- Who ran it, and what did they expect? Watch for researcher preconceptions shaping how results are measured or interpreted.
- Who paid, and who benefits? Funding or interest-group influence is a reason for caution, not automatic dismissal.
- What was left out? Missing comparison groups, hidden trials, or absent "negative" results point to selective reporting.
- Who was studied? Check that the sample actually represents the group the conclusion is applied to.
- Always name the direction. A strong answer says which way the bias likely tilts the result, then asks what evidence would settle it.
Your turn: practice scenarios
For each one, name the main source of bias and say which way it probably tilts the result. Try it before you reveal the answers.
- A soft-drink maker funds and publishes a study concluding that sugary drinks have "no meaningful link" to weight gain. The study's lead scientist works for the company.
- A researcher convinced that a new teaching method works tests it on her own students, grades the results herself knowing which students used the method, and reports a big improvement.
- A survey about a city's satisfaction with public transit is handed out only to people already riding the bus on a weekday morning. The report says "riders are highly satisfied."
Tap to reveal the answers
- 1. Funding / interest-group influence (the maker pays and its own scientist leads it). Likely tilt: toward a reassuring "no link" result. Better evidence: an independent study with no financial stake.
- 2. Researcher preconceptions (she expects success and grades the results herself, knowing who used the method). Likely tilt: toward a positive result. Better evidence: a blind setup where the grader does not know which students used the method.
- 3. Unrepresentative sample (only current riders are asked, so people who quit the bus or never ride are missing). Likely tilt: toward high satisfaction. Better evidence: survey a representative slice of all residents, riders and non-riders alike.
Why this matters for the CAEC
The CAEC Science test is a skills and inquiry test, not a memory test, 35 questions in 90 minutes, with a calculator permitted. Most marks reward evaluating investigations: reading a scenario and judging how trustworthy it is. Spotting bias and naming its likely direction is exactly the kind of reasoning those questions reward, no matter whether the scenario is about biology, chemistry, physics, or earth science.
Want more practice like this? Explore the rest of our Science lessons, pick up the CAEC Ready Workbook for more worked scenarios, or start with a free sample to test yourself.
Disclaimer
This article is a general study lesson. CAEC Ready is an independent study resource and is not affiliated with or endorsed by any government, ministry of education, or official CAEC testing provider.