 Show
Reliability is a concept that refers to producing consistent results time after time. If you commission a qualitative research or evaluation project, how can you be sure it is reliable? Don’t you need statistics to make research reliable? No. Because, although the term ‘reliability’ is usually applied as a concept for testing or evaluating quantitative research, the idea is also used in all kinds of research. If a qualitative research project is reliable, it will help you understand a situation clearly that would otherwise be confusing. Qualitative research is about discussion, about delving into topics in depth, getting beneath the surface. It’s not a precise science. However, clients would not commission qualitative studies if there was no sense that the results would be reliable, that they could make confident
decisions based on the results. So, how can qualitative research be conducted with reliability? The project is credibleOne of the key criteria is that of internal validity, in which they seek to ensure that their study measures or tests what is actually intended. A key question to ask yourself is ‘How congruent are the findings with reality?’ A credible project is one that adopts established research methods, such as random sampling, a range of different research methods, iterative questioning and frequent client debriefing. The research is transferableThis is the extent to which the findings of one study can be applied to other situations. Since the findings of a qualitative project are specific to a small number of particular individuals and environments, it is impossible to demonstrate that the findings are applicable to other situations and populations. However, it is important to consider the findings of the qualitative study within the broader context of other people and settings, and whether similar projects and methods conducted in different environments would be of value. The process is dependableFor qualitative research to be dependable, the processes within the study should be reported in detail, thereby enabling a future researcher or evaluator to repeat the work, if not necessarily gain the same result. Thus, the research design may be viewed as a ‘prototype model.’ The findings can be confirmedThe extent to which qualitative findings can be confirmed is challenging, as it is difficult ensuring objectivity to qualitative research. But steps must be taken to help ensure as far as possible that the work’s findings are the result of experiences and ideas of the informants, and that evaluation findings are arrived at by considering solid evidence. It is important for qualitative researchers and evaluators to ensure their work is methodologically sound and that clients can rely on the findings. If the credibility, transferability, dependability and confirmability aspects of the research are taken into account, qualitative research should be as reliable as a dependable, loyal pooch. *The content of this blog was originally sourced from Lincoln and Guba’s Evaluative Criteria. Lincoln, YS. and Guba, EG (1985), Naturalistic Inquiry, Newbury Park, CA. Sage Publications. Published on July 3, 2019 by Fiona Middleton. Revised on October 10, 2022. Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a
method, technique or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure. It’s important to consider reliability and validity when you are creating your research design, planning your
methods, and writing up your results, especially in quantitative research. Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable. Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same
methods under the same circumstances, the measurement is considered reliable. You measure the temperature of a liquid sample several times under identical conditions. The thermometer displays the same temperature every time, so the results are reliable. A doctor uses a symptom questionnaire to diagnose a patient with a long-term medical condition. Several different doctors use the same questionnaire with the same patient but give different diagnoses. This indicates that the
questionnaire has low reliability as a measure of the condition. Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world. High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid. If the thermometer shows different temperatures each time, even though you have carefully controlled conditions to ensure the sample’s temperature stays the same, the thermometer is probably malfunctioning, and therefore its measurements are not valid. If a symptom questionnaire results in a reliable diagnosis when answered at different times and with different doctors, this indicates that it has high validity as a measurement of the medical condition. However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation. The thermometer that you used to test the sample gives reliable results. However, the thermometer has not been calibrated properly, so the result is 2 degrees lower than the true value. Therefore, the measurement is not valid. A group of participants take a test designed to measure working memory. The results are reliable, but participants’ scores correlate strongly with their level of reading comprehension. This indicates that the method might have low validity: the test may be measuring participants’ reading comprehension instead of their working memory. Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect your data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid. How are reliability and validity assessed?Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types. Types of reliabilityDifferent types of reliability can be estimated through various statistical methods. Types of reliability
Types of validityThe validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods. Types of validity
To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment) and external validity (the generalizability of the results). How to ensure validity and reliability in your researchThe reliability and validity of your results depends on creating a strong research design, choosing appropriate methods and samples, and conducting the research carefully and consistently. Ensuring validityIf you use scores or ratings to measure variations in something (such as psychological traits, levels of ability or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data.
Ensure that your method and measurement technique are high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge. For example, to collect data on a personality trait, you could use a standardized questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or findings of previous studies, and the questions should be carefully and precisely worded.
To produce valid generalizable results, clearly define the population you are researching (e.g. people from a specific age range, geographical location, or profession). Ensure that you have enough participants and that they are representative of the population. Ensuring reliabilityReliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable and reproducible.
Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved. For example, if you are conducting interviews or observations, clearly define how specific behaviours or responses will be counted, and make sure questions are phrased the same way each time.
When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results. For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions. Where to write about reliability and validity in a thesisIt’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper. Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy. Reliability and validity in a thesis
Cite this Scribbr articleIf you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.
Is this article helpful?You have already voted. Thanks :-) Your vote is saved :-) Processing your vote... What is internal validity in qualitative research?Internal validity refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors or variables. External validity refers to the extent to which results from a study can be applied (generalized) to other situations, groups or events.
How can we ensure the internal validity of a qualitative research?Validity in qualitative research can also be checked by a technique known as respondent validation. This technique involves testing initial results with participants to see if they still ring true.
What is internal validity and reliability?Reliability vs. Validity in Research | Difference, Types and Examples. What is internal and external validity in research?Internal validity examines whether the study design, conduct, and analysis answer the research questions without bias. External validity examines whether the study findings can be generalized to other contexts.
|
