Statistical Surveys: Bias in a Sample

Say NO to Bias

In statistics and research, bias is a four-letter word.

(I mean, obviously it's four letters no matter what, but in statistics, it's something that you should stay as faaaaaar away from as possible).

One of the most common types of bias that pops up in studies is selection bias. In these cases, the samples in our study are NOT representative of the overall population because of the way we selected them.

Most of the time, bias is the result of poor study design.

Undercoverage

One common type of selection bias is under-representation (also called undercoverage). In these cases, certain groups in the target population are not adequately represented.

Imagine that I want to do a study of neighborhood opinions on healthcare options for children. I decide to go door-to-door during the day to ask residents survey questions.

But think about it...because of the way I designed it, my study won't include anyone who isn't at home during the day. This means that working parents won't be able to participate because they won't be home when I come knocking. The opinions of a significant portion of the target population won't be represented in my study.

This is undercoverage. The graphic below demonstrates this concept visually.

Non-Response Bias

Another type of selection bias is called non-response bias. Like undercoverage, the result of non-response bias is that the subjects in our study are NOT representative of the target population.

Non-response bias is exactly what it sounds like. Subjects that have been selected for a study are sometimes unable or unwilling to participate. This is non-response.

Definition of Biased Sampling

A biased sample is a sampling method in which the method systematically favors some outcomes over others. In some cases, bias is intentional, such as if the researcher wants people to give a certain response (i.e., they want more people to say they like dogs over cats, so they ask for responses at a dog park, or dog show).

You do NOT want any bias for a noteworthy sampling. As long as you are sampling, you want it to be as random as you can make it for the most accurate/best results.

Types of Bias

Undercoverage: This happens when some members of the population are not adequately represented in the study. For example, if you are asking engineers in the engineering school how they feel about ice skating, then women will be underrepresented since women only make up 36% of the engineering school population.

Nonresponse Bias: These types of samples are ALWAYS biased. Sometimes, for one reason or another, individuals chosen for a sampling are unwilling or unable to respond. This is frequently seen in politics, when an individual favors one candidate, and will not answer a survey about their feelings about the opposing candidate.

Voluntary Response Samples: These types of samples are ALWAYS biased. This is because voluntary responses tend to oversample people who have strong opinions of the topic at hand, and undersample the people who are uninterested or not opinionated. This is because those who have strong feelings on a topic want to have their voices heard in order to change events/keep them the same.

Convenience Sample: Sometimes, a researcher cannot travel to a place that will give them a truly random sample, which will give them a non-biased sampling. Therefore, they will take samples from a group of people easy to contact, or from a location that is convenient to travel to.

Lack of Blinding: This is typically used for medical treatments. For example, let's say there are two medical treatments that are supposed to reverse balding, but there are different side-effects. Blinding is ensuring that no-one who may influence the final results by knowing which treatment they are receiving. Therefore, lack of blinding means that there are patients who do know which treatment they are receiving, therefore skewing the results, and creating a bias.

(Video) A Real World Example of Selection Bias

by Dr. Michael Housman

Dr. Michael Housman has a great, quick video on when there can be bias in a sample and explains it using one of my favorite examples.

$\large{\text{Summary}}$

He starts off be explaining that it is important to use correctly measure and using good methods in order to get good, accurate data. He uses a historical example from World War II to explain his point, where the British Royal Air Force missed a critical detail in their study of their fighter planes.

In this case there was selection bias as they were only looking at planes that were damaged and able to return while ignoring the planes that were hit in critical points and crashed. This left out important information and almost cost the RAF a lot.

Examples of Sample Bias

To have bias in a sample means to not have a neutral viewpoint in regard to the sample. Bias is an inclination toward one way of thinking which may be right or wrong, without adequately considering other viewpoints.

Examples of Bias:

• An employer who is biased towards women may reject to hire a woman in favor of a male applicant.
• A person who is biased towards a certain religion may assume people of that religion to be less than others.
• A person who is biased toward a political belief might express it through inappropriate name calling, or simply by refusing to believe their those holding this political belief could be right about anything.

From these few examples it is clear that bias is destructive and not representative of the people being biased towards.

Examples of a Bias in a Sample:

• Claire wants to see if the school lunches are big enough portions according to the rest of the students in her high school. She surveys the members of the swim team only.

Explanation:

In order to get unbiased results, Claire needs to ask a representative sample of her high school population. Asking only members of the swim team may skew the results of her research because the swim team may burn more calories on average than their high school peers, and hence require larger lunches than the average peer at their high school.

• Bob, a university student wants to know whether the campus is providing enough mental-health support for freshmen students. He decides to ask employees of Psychological Service Center on campus to participate in the study.

Explanation:

In order to avoid biased results, Bob needs to ask a representative sample of his freshman class and NOT just people who work for the Psychological Service Center. If a representative sample is not used in Bob's research people who work for the Psychological Service Center may skew the results as they are aware of what resources are available to freshman, but not all freshman may be aware of them.