validating qualitative research
- See if your theory explains the data
- Send it back to your participants, see if they agree with it.
Lenses from Charmaz (2006)
“Do the categories cover a wide range of empirical observations? Are there strong logical links between the gathered data and your argument and analysis?”
“Are your categories fresh? Do they offer new insights?”
“Do the categories portray the fullness of the studied experience?”
“Does your analysis offer interpretations that people can use in their everyday worlds?”
fit
- Do findings resonate with the study’s audience and it’s participants?
applicability
- Are the findings useful?
concepts
- Are the findings structured around concepts that a participant can understand?
contextualization
- Are the findings well situated in the context
logic
- Is there a logical flow of ideas
depth
- are there details that make it interesting?
variation
- are there examples within the findings that don’t fit the mold, and how does the analysis deal with those? Too much variation? The theory isn’t capturing the data. Too little? The theory is overly general
creativity
- Are the findings saying something new?
sensitivity
- how much were the findings based on preconceived notions, and how much did the researcher actually build off of the data?
evidence of memos (audit trail)
- is there a well fleshed out audit trail?
And just for the hell of it, a copy paste of the additional criteria:
Criterion #1. How was the original sample selected? How did later sampling occur? Criterion #2. What major categories emerged? Criterion #3. What were some of the events, incidents, and/or actions (indicators) that pointed to some of these major categories? Criterion #4. On the basis of what categories did theoretical sampling proceed? That is, how did theoretical formulations guide the data collection? After the theoretical sampling was done, how representative did the categories prove to be of the data? Criterion #5. What were some of the statements of relationships made during the analysis and on what grounds were they formulated and validated? Criterion #6. Were there instances when statements of relationships did not explain what was happening in the data (negative cases)? How were these discrepancies accounted for? Were statements of relationships modified? Criterion #7. How and why was the core category (if applicable) selected? On what grounds is the final analytic decisions made? Criterion #8. Are the concepts systematically related? To have theory, there must be systematic development of concepts and linkages of those concepts to form a theoretical explanation about some phenomenon. The key word is that theory explains. It doesn’t just describe. You have to answer the questions of “why” or “how come” and “when” this happens, then “is this likely” to happen? Linkages may be presented as a list of hypotheses or propositions. Or, they may be woven throughout the text in more subtle forms. It may be helpful to the reader if the researcher makes this clear to the reader. Criterion #9. Is variation built into the theory? Variation is important because it signifies that a concept has been examined under a series of different conditions and developed across a range of dimensions. Some qualitative studies report only a single phenomenon and establish only a few conditions under which it appears; also they specify only a few actions/interactions that characterize it, and a limited number or range of consequences. By contrast, when using this methodology there should be considerable variation built into the theory. In a published paper, the range of variations touched upon may be more limited, but the author should at least suggest that other writings include their specification. Criterion #10. Are the conditions and consequences built into the study and explained? Any explanation of variation should include the conditions under which it can be found and some of the consequences of action/interaction/emotional responses. Conditions should not be listed merely as background information in a separate chapter, but woven into the actual analysis with explanations of how they impact the events and actions in the data. These include, but are not limited to, economic factors, organizational policies, rules and regulations, social movements, trends, culture, societal values, language, and professional values and standards. Criterion #11. Has process been taken into account? Identifying process in research is important because it enables theory users to explain action under changing conditions. The conceptual scheme used to explain process is less important than attempts to bring it into the analysis. Criterion #12. Do the theoretical findings seem significant, and to what extent? It is entirely possible to complete a theory generating study, or any research investigation, yet not produce findings that are significant. If a researcher simply goes through the motions of doing research without drawing upon creativity or developing insight into what the data are reflecting, then this researcher risks the possibility of arriving at findings that are less than significant. By this, I mean that the research fails to deliver new information or to offer new insights or explanations. Remember there is an interplay between the researcher and the data, and no method can ensure that the interplay will be creative. This depends on four characteristics of the researcher: analytic ability, theoretical sensitivity, ability to think about data in different ways, and sufficient writing ability to convey the findings. Of course, a creative interplay also depends upon the quality of data collected or utilized. An unimaginative analysis may, in a technical sense, be adequately grounded in the data, but be limited for theoretical purposes. This is because the researcher either doesn’t draw on the fuller resources of data or fails to push data collection far enough. Criterion #13. Do the findings become part of the discussions and ideas exchanged among relevant social and professional groups? Findings are time and place specific, however, major concepts often have continued usefulness. Take concepts such as stigma, division of labor, uncertainty, stress, and negotiations. These concepts have proven their usefulness throughout the years, though the specific findings associated with them may have been modified and changed with time
