The robustness of methodology and study design are the key elements in determining the quality of research. These are at the centre of any research proposal to aid the reproducibility of research findings. One of the issues that you should be aware of when designing your research is the possible introduction of research bias.

Bias exists in all research. It can occur at any stage of the research process, including study design, data collection, data analysis and publication. If not considered carefully, bias can impact the validity and reliability of the study findings.

As an applicant for a research grant, you should be able to understand the possible sources of bias that may affect your research, and clearly set out and justify the measures of avoiding the bias.

Some of the sources of bias that you should be aware of when designing your research include (but are not limited to) the following:¹

• Selection bias can be introduced when the characteristics of individuals or groups in a study differ systematically from the population of interest.
• Non-respondent bias may occur when non-responders from a sample differ in a meaningful way from responders.
• A recall bias may result from a systematic error due to differences in accuracy or completeness of recall to the memory of past events or experiences. This is especially important in studies that involve interviewing or self-reporting questionnaires.
• Allocation bias arises from a systematic difference in how participants are assigned to treatment groups and comparison groups in a clinical trial.
• Attrition bias can be introduced into the study’s result if there are systematic differences between people who leave the study and those who continue.
• Volunteer bias can occur when the subjects who volunteer to participate in a research project are different in some ways from the general population. This results in the data collected being non-representative of the population.
• Confounding is the distortion of the association between the independent and dependent variables because a third variable is independently associated with both. It’s a bias-related issue that you should be aware of when designing the research.

Bias can occur in both qualitative and quantitative study designs. In a qualitative study that involves an interview, for example, a representative sample of individuals should be selected carefully to avoid any systematic bias.  The sampling technique that you suggest for choosing participants for the study can be one of the sources of bias. Every sampling technique has its drawback, but some are better than others in reducing bias. To choose a sample that does not have a built-in bias, you may need to divide the population into groups, depending on how you think various characteristics will affect the result. For instance, purposive sampling is widely used in qualitative studies (within mixed methods approaches), but due to the lack of random selection, this is prone to selection bias and error. If this is the ‘best’ sampling option for your study, then bear the limitations of the sampling method in mind when interpreting the results of your study.

The issues of bias can also apply to the patient and public involvement (PPI) activities. For example, how do you select your PPI participants so that they are diverse and representative of the population (minimise selection bias and volunteer bias)? How neutral is the information you provide to the PPI participants (that you were not tempted to give limited or selected information to get the response you want)? Are you reporting only the positive outcomes for your PPI activities (positive-reporting bias)²?

The first step in avoiding or reducing research bias is recognizing its existence. Reviewers expect applicants to provide adequate information on the approaches to be followed to minimise the effects of bias or an explanation of why this would not be appropriate. Some of the methods used for reducing bias (depending on the study type) include randomisation, blinding, allocation concealment, stratification, adjusting for confounders, matching, sampling, etc. These methods, whenever planned to be used in the study, should be described well. If randomisation is used, for instance, it is advisable to clarify how it will be carried out. It’s equally important to explain why randomisation is not appropriate if it will not be used in a clinical trial.

Bias is not something that can be accounted for with statistics. Statisticians or data scientists can take some measures to look for bias at the level of data analysis, but they can never eliminate it. Increasing sample size can help in improving precision, but that doesn’t help if bias has already been introduced at the design stage. Get in touch with your local RDS and we can help your project design.

References

1. Catalogue of bias https://catalogofbias.org/biases/
2. Chew‐Graham (2016). Positive reporting? Is there a bias in reporting of patient and public involvement and engagement? Health Expectations, 19(3): 499–500. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5055241/