Other official information and services : www.belgium.be

6.2. General quality criteria

Author(s)

Most of the quality criteria are applicable to all research, both quantitative and qualitative. For example in 2008, Cohen and Crabtree (Cohen, 2008) reviewed and synthesized published criteria for good qualitative research. They identified the following general evaluative criteria: 1) ethical research, 2) importance of the research, 3) clarity and coherence of the research report, 4) use of appropriate and rigorous methods, 5) importance of reflexivity or attending to researcher bias, 6) importance of establishing validity or credibility, 7) Importance of verification or reliability. Researcher bias, validity, and reliability are most heavily influenced by quantitative approaches. Table 6 bridges quantitative and qualitative research by illustrating the parallels between criteria for conventional quantitative inquiries and qualitative research.

Table 6 – Lincoln and Guba’s translation of terms

Quantitative research

Qualitative research

Methods to ensure quality

Internal validity

Credibility:

Are the findings credible?

Member checks[a]; prolonged engagement in the field; data triangulation

External validity

Transferability:

Are the findings applicable in other contexts?

Thick description[b] of setting and/or participants

Reliability

Dependability:

Are the findings consistent and could they be repeated?

Audit – researcher’s documentation of data, methods and decisions; researcher triangulation

Objectivity

Confirmability:

To which extend are the findings shaped by the respondents and not researcher bias, motivation or interests?

Audit and reflexivity – e.g. awareness of position as a researcher and its influence on the data and findings

Source: Adapted from Finley,2006

In what follows we pay attention to some keywords appearing in Table 6.

Reflexivity

“Reflexivity is an awareness of the self in the situation of action and of the role of the self in constructing that situation.” (Bloor and Wood, 2006, p. 145)

Because in qualitative research, the researcher could not be ‘blinded’, he/she has to take into account subjectivity in an explicit way. To demonstrate this reflexive awareness during the research process, the following ‘good practices’ can be used (Green, 2009, p. 195):

  • Methodological openness: report steps taken in data production and analysis, the decisions made, and the alternatives not pursued.
  • Theoretical openess: theoretical starting points and assumptions should be adressed.
  • Awareness of the social setting of the research itself: be aware of the interactivity between the researcher and the researched.
  • Awareness of the wider social context, including historical and policy contexts and social values.

Triangulation

“Qualitative research is inherently multimethod in focus (Flick, 2002, p.226-227). However, the use of multiple methods, or triangulation, reflects an attempt to secure an in-depth understanding of the phenomenon in question. Objective reality can never be captured. We know a thing only through its representations. Triangulation is not a tool or a strategy of validation, but an alternative to validation (Flick, 2002, p. 227). The combination of multiple methodological practices, empirical materials, perspectives, and observers in a single study is best understood, then, as a strategy that adds rigor, breadth, complexity, richness, and depth to any inquiry(See Flick, 2002, p. 229)” (Denzin and Lincoln, 2008, p. 7).

Triangulation is the use of several scientific methods, both qualitative and quantitative, to answer the same research question(Bloor, 2006) . Often triangulation is understood as producing the same results by means of several methods, sources or analysts. However, different methods or types of inquiry are sensitive to different nuances, so that they may lead to somewhat different results. In fact, triangulation is more about finding inconsistencies to gain deeper insight into the relationship between the inquiry approach and the subject under study. Thus, finding inconsistencies do not weaken the credibility of the results, but rather strengthen it (Patton, 1999).

Five kinds of triangulation can contribute to the quality and consistency of qualitative data analysis:

  1. Methods triangulation: Information obtained through several methods is compared. These methods can be qualitative, or quantitative or both. Often qualitative and quantitative data can be fruitfully combined as they mostly elucidate complementary aspects of the same phenomenon(Patton, 1999) .
  2. Triangulation of sources: Information derived at different times and by different means is compared, e.g. comparing observational data with interview data, but also comparing what people say in public with what they say in private (Patton, 1999) .
  3. Analyst triangulation: Several observers, interviewers, researchers or analysts are used. By this way the potential bias that comes from a single person doing all the data collection and/or data analysis is reduced. In addition to several researchers or data analysts, analytical triangulation may also be to have those who were studied review the findings(Patton, 1999) .
  4. Theory/perspective triangulation:  It involves the use of different theoretical perspectives to look at the same data. Also, for example, data can be examined from the perspective of various stakeholder positions (Patton, 1999) .
  5. Member validation: It is a popular kind of triangulation that consists of “checking the accuracy of early findings with research respondents” (Bloor and Wood, 2006, p. 170).

These kinds of triangulation protect the researcher against the accusation that findings are an artifact of a single method, or source or investigator’s biases (Patton, 1999).

Transferability

Earlier in this report we argued that qualitative research is context sensitive and it is not aimed at making generalizations to the wider population. This may appear to contradict with the notion of transferability which is just about the extent to which findings of one study can be applied to other situations (external validity) (Merriam, 1998).

Transferability refers to the responsibility of the researcher to provide sufficient contextual information about the fieldwork to enable the reader to determine how far he can be confident in transferring the findings to other situations(Firestone, 1993). However, the situation might be complicated by the possibility that factors considered by the researcher to be unimportant, and consequently unaddressed in the research report, may be critical in the eyes of a reader(Firestone, 1993).