A 90% pass rate among people who took the final test is not the same as a 90% pass rate among everyone who enrolled.
An online course advertises a 90% pass rate. Before treating that as your chance of passing, check who was counted. If 100 people enrolled, 20 took the final test, and 18 passed, the advertised figure describes the test takers. Only 18% of the original group are known to have passed that test.
The other 80 people should not automatically be counted as failures. Some may have learned what they needed and left; others may have found the course confusing or impossible to finish. Without their results or reasons for leaving, we do not know. That missing information is part of judging the course, not a detail to remove from the calculation.
Research studies face the same problem when participants stop responding or miss a final assessment. A result based on the remaining people may be misleading if staying in the study is related to how well the treatment or programme worked for them. The invented course example below shows why both the numbers and the reasons for missing data matter.
When reading a study, find how many people entered each group. Then find how many had results recorded at the end. If the counts differ, ask why. People move or get busy. They may lose interest or find the task too hard. The reason they leave may be part of what you need to learn. The reason matters. In our made-up course, some may already know the topic. They may see no need for the test. Or perhaps the lessons were so confusing that most quit. The same final count can fit very different stories.
To distinguish these explanations, the course would need information about why people left and what they learned before leaving. The final test scores alone cannot settle that question.
Imagine a study of two exercise plans. One plan is hard to follow. People who dislike it leave and stop answering the final survey. The people who stay may be the ones who found it easiest. A report based only on those people can make the plan look too good. It leaves out those who did not stay. The Cochrane trial guide treats missing outcomes as a possible source of bias. Ask why results are missing. Is the reason linked to how people did? The percent missing is only part of the issue.
Ten missing results can matter a lot in one study and less in another. There is no single safe cutoff that fixes every case.
In the course example, 18 of the 20 test takers passed. Suppose none of the other 80 could have passed. Then the rate for all starters would be 18 out of 100. Suppose all 80 could have passed. Then the rate would be 98 out of 100. We know two test takers failed, so it cannot be 100 under that simple setup. Those extremes are not equally likely guesses. They show how much the missing outcomes could matter. Better information would narrow the range.
A real study can use more careful ways to study these gaps. Ask what else could be true of the missing people. Would that change the result?
A dropout joined the study, but we have no later result for them. A person who never joined is missing from the sample in a different way. Suppose a survey asks about buses and trains. It is shared only through an app. People who do not use the app may never see it. A large number of replies does not make that gap vanish. That is a sampling problem: who had a real chance to be included? OpenStax's sampling lesson explains why the way people are selected matters. You can get a million replies from the wrong group. You may still miss the people your claim is about.
Imagine asking only people in a swimming club whether they enjoy swimming. Ten answers may be too few to describe the club well. Ten thousand members would tell you much more about that club. They still would not be a fair picture of everyone in town. The problem is not only the count. It is how people got picked. More data may reduce some noise due to chance. It may still leave a whole group out. That is why large study should not end the discussion.
Check which population the sample represents and which people had no realistic opportunity to participate. A large number of responses from one group does not automatically justify a claim about a broader population.
A useful course report would state the original enrolment, the number assessed, and the number who passed. It would also describe known reasons for non-completion and identify the outcomes that remain unknown. That sounds less exciting than 90% success. It is also much more useful to someone deciding whether to join. A good report can give more than one view. Just say which group each number describes. Results among finishers answer one question. Results among all starters answer another. Do not let a writer switch between those groups in the middle of a claim.
Look for the number who started, the number whose results were measured, and the explanation for any difference. Then examine how the researchers handled the missing outcomes and whether other reasonable assumptions would change the conclusion. A final percentage is more informative when that path is visible. For the course, an honest description gives all three counts: 100 enrolled, 20 took the final test, and 18 passed. It can also report a 90% pass rate among test takers, provided it names that group. Readers can then decide what the result does and does not tell them about the experience of signing up.
Cochrane / 2024 / Website
Handbook 6.5. Used for assignment, missing outcomes, and measurement/reporting checks. All blog course and word-study numbers are invented worked examples. Checked 2026-10-09.
OpenStax / 2023 / Website
Open textbook reference. Worked blog examples are original and labeled as examples. Checked 2026-10-09.
