Lesson 5.3 · 25 min
Sampling and Bias
Planning a survey: primary or secondary data, quantitative or qualitative, sample size, and how bias creeps in.
In this lesson you will learn to…
- Describe the four stages of a statistical investigation
- Tell primary data from secondary data, and quantitative from qualitative
- Weigh up the advantages and disadvantages of large and small samples
- Spot sources of bias in a sample or a questionnaire and suggest improvements
Why this matters
Almost every news story that begins “a survey found …” is based on a sample. Whether you can trust it depends on who was asked, how many were asked and what exactly they were asked. Being able to judge that is one of the most useful skills in the whole statistics course.
Choosing how much data
| Small sample | Large sample | |
|---|---|---|
| Advantage | Quick and cheap to collect | More reliable, closer to the truth |
| Disadvantage | Less reliable; one odd result has a big effect | Slow and expensive |
You also need to decide the degree of accuracy — for example, rounding amounts spent to the nearest dollar, or times to the nearest minute.
How bias gets in
Intentional bias — choosing data to give the picture you want. An advert that quotes only its happiest customers is biased on purpose.
Unintentional bias — an accident of how you sampled. Asking the first 50 people into a cinema about their favourite film seems fair, but if 38 of them are men, the results lean towards men’s opinions.
Writing good questions
| Problem | Example | Better |
|---|---|---|
| Leading question | “Don’t you agree that …?” | “What do you think about …?” |
| Vague options | “How much exercise do you do?” | Tick boxes: 0–1 h / 1–3 h / 3–5 h / 5+ h per week |
| Personal or awkward | “How old are you?” | Age ranges: under 18 / 18–30 / 31–50 / over 50 |
| Asks for data you don’t need | Name, address … | Only ask what answers your question |
Run a pilot survey. Try your questionnaire on a few people first. Answers like ”?”, “half” or a dash show you which questions need rewriting.
Key rules and formulas
The data cycle
1 Ask a question → 2 collect data → 3 analyse the data → 4 use the results to answer the question. The results often lead to a follow-up question, and the cycle starts again.Primary and secondary data
Primary data you collect yourself — reliable, but slow. Secondary data someone else collected (a website, a teacher, a report) — quick, but you do not know how reliable it is.Quantitative and qualitative
Quantitative data is numerical (height, time, number of books). Qualitative data is descriptive (eye colour, type of book, "very good").Bias
A sample is biased if it does not fairly represent the whole population — for example, asking only people outside a swimming pool whether they like swimming.
Worked examples
Spotting a biased sample
Try it yourself first, then reveal the solution one step at a time.
Improving a questionnaire
Try it yourself first, then reveal the solution one step at a time.
Practice: sampling and bias
9 questions
Question 1 of 9Put in orderFoundation Put the four stages of a statistical investigation in order.Put these in the right order. Drag them, or use the arrow buttons.
- Use the results to answer the question
- Ask a question
- Collect data
- Analyse the data
Question 2 of 9Match the pairs Frankie wants to know whether teenagers eat a healthy diet. Match each way of collecting data to its type.Pick an answer, then choose the row it belongs to. You can also drag it.
- Asking ten people in the dinner queue what they are buying
- Asking the canteen staff for last month's sales list
- Looking up canteen food sales figures on the internet
- Using his own data sheet to record what his friends eat each lunchtime
Question 3 of 9Choose all that apply Select every variable that is quantitative.Select every correct answer — there may be more than one.
Question 4 of 9Multiple choice Andreas asks 20 people outside the swimming pool whether they like swimming. Why is his sample biased?Question 5 of 9Choose all that apply Beth surveys ten of her friends about healthy eating. Her questions include "Do you agree that eating too many sweets is bad for you? Yes/No". Select every reason her results might be biased.Select every correct answer — there may be more than one.
Question 6 of 9True or false A larger sample is usually more reliable, but it takes longer and costs more to collect.Question 7 of 9Multiple choice Jamie wants to know what condition people's phones are in after a year. Which question collects qualitative data?Question 8 of 9Multiple choice In a pilot survey, one person wrote "half" for "How much exercise do you do?". What is the best improvement?Question 9 of 9Fill the gaps Complete the sentences.Data you collect yourself is called data. Data collected by someone else is data. Trying a questionnaire on a few people first is called a survey.