|
As these applications in Section three demonstrate, triangulation is clearly a core issue in any approach to methodological combination. The contributions in this volume also show, however, that triangulation is not the only way in which qualitative and quantitative methods can be combined. Besides triangulation, two other approaches to method combination can be distinguished: sequencing and what we will term "hybrids" (cf. SCHREIER in press). [31]
In the case of sequencing, qualitative and quantitative methods are employed within one and the same study, although in different phases of the research process. The most common example would be a qualitative phase of data collection which is followed by a quantitative phase of data analysis, as in the case of interviews which are coded and for which coding frequencies are determined; alternatively, data analysis might involve the construction of types by means of cluster analysis, the reduction of categories to a smaller number of dimensions by means of multiple correspondance analysis, etc. (for additional examples of sequencing cf. MAYRING in this volume). Sequencing in this sense can be employed within otherwise "quantitative" studies which aim at hypothesis testing (cf., however, WITT in this volume who warns against indiscriminately using qualitative data as part of a linear-quantitative research strategy). To the extent that qualitative research wishes to go beyond individual cases and to say something about the sample at large, maybe even the population, sequencing can also be part of a qualitative research strategy, taking place whenever a generalisation of qualitative findings occurs on an aggregate level. Looked at from this perspective, sequencing may even be said to constitute an inherent characteristic of many typically "qualitative" approaches, such as grounded theory, objective hermeneutics, comparative casuistics, and so on. [32]
By "hybrids" we mean approaches which do in themselves constitute a combination of qualitative and quantitative elements. These elements may be so closely "packed" as to be practically indistinguishablesystematic content analysis which combines the (qualitative) coding of texts with the (quantitative) calculation of coefficients of interrater agreement would be a case in point (RUSTEMEYER 1992; GROEBEN & RUSTEMEYER 1994). More often, hybrid approaches comprise a number of phases, some of which are qualitative, others quantitative; all, however, are equally necessary for achieving the objective of the approach. There are some examples in this volume, such as logographic analysis (SCHMITT, MEES & LAUCKEN), numerically aided phenomenology (KUIKEN & MIALL), or the qualitative experiment (KLEINING & WITT); others, such as the research program subjective theories (GROEBEN & SCHEELE 2000) or KUCKARTZ' approach toward case-oriented quantification (e.g., KUCKARTZ 1995) are not covered here. To the extent that these latter approaches combine qualitative and quantitative research phases, these "hybrids" are very similar to the strategy involved in sequencing. Hybrids and sequencing differ, however, in the sense that hybrids require precisely one and only one specific combination of qualitative and quantitative phases, whereas in sequencing any kind of combination is possible. [33]
There do, of course, exist other issues concerning the relation between qualitative and quantitative methods which might have been raised by the contributions to the volumesuch as strategies for the analysis of qualitative data on an aggregate level or questions concerning the methodological standards for evaluating the results of qualitative research (cf. the discussion in FQS: REICHERTZ 2000; BREUER 2000). That this was not the case is probably to some extent due to the original orientation of the volume which did not really invite such strictly methodological papers. Instead, as we said above, it is triangulation which drew the greatest amount of discussion. [34]
The usual emphasis in triangulation is on combining methods, e.g., survey questionnaires with non-standardised interviews, although examples are also common of studies where triangulation is claimed on the basis of using a number of data sources (self, informants, other commentators), a number of accounts of events, or a number of different researchers (see FIELDING & FIELDING 1986). The broad idea in the conventional approach to triangulation is that if diverse kinds of data support the same conclusion, confidence in the conclusions is increased. It is implicit here that this is only to the extent that different methods or different kinds of data have different types of error. Further implied is that these sources of error can be anticipated in advance and that their effects and magnitude can be traced when analysis is carried out. It is in this sense that LEVINS' (1966, p.423) declaration that "our truth is the intersection of independent lies" is so apt. [35]
The classical approach represented by CAMPBELL's work, and still widely encountered in psychology, is one seeking convergence or confirmation of results across different methods. In effect, this amounts to conducting two studies with the hope of arriving at the same conclusions, thus demonstrating that the conclusions are not artifacts of method and, in particular, associated with sources of invalidity characteristic of a given method. A key example is DENZIN (1970), whose original conceptualization of triangulation is explicitly related to the work of WEBB, CAMPBELL, SCHWARTZ and SECHREST (1966) on "unobtrusive measures". However, the term "triangulation" has acquired so many meanings and usages that it is now safer to use the terms "convergence" or "confirmation" when seeking cross-validation between methods. [36]
In fact this classic goal of seeking convergence across methods has always been relatively rare and is increasingly so as a motive for combining quantitative and qualitative methods. This is so particularly in social science research and even more so in applied social research. One reason for this is the obstacle one encounters when results fail to converge. But the rarity of classical triangulation as a reason for combining methods is also a response to the amount of effort that it takes to pursue the goal of producing convergent findings. As MORGAN (1998) notes, researchers in applied fields often cannot afford to put so much effort into finding the same thing twice. On the other hand, applied problems such as the factors influencing health are so various and complex that applied researchers are readily driven to appreciate the different strengths that different methods offer. This makes for a more flexible approach to methodological combination than is found in classic triangulation, but nevertheless represents an important motivation for combining methods. [37]
It must be apparent from the different constructions of triangulation mentioned above that there are degrees of rigour and/or formality in the operationalisation of the broad idea of triangulation. We might, for example, regard the idea that validity will be enhanced simply by drawing on data collected by different researchers using the same method as a relatively weak form of triangulation, while an approach based on the combination of different methods might be regarded as somewhat more rigorous. Even so, we have already begged a significant questionwhat is to count as "valid"? As virtually all readers of this journal will be aware, validity (or the idea of a "conclusion" about which we can be "confident") is a highly contested idea. [38]
While epistemological debate continues, with the virtual certainty that it will never conclude, we can nevertheless safely proceed with our concept of triangulation provided in each case where it is claimed the researchers make clear what criteria of adequacy and/or validity they intend to apply. But this is really only an extension of the idea that, for triangulation to be credibly claimed and demonstrated, it is necessary to identify in advance the characteristic weaknesses or types of error associated with given methods so that, when data from the different methods is combined, the possibility can be discounted that the methods might be susceptible to the same kinds of validity-threat. Where they are susceptible to the same weaknesses, combining them will, of course, do no more than multiply error. [39]
Thus, a great deal depends on the logic by which researchers draw on and mesh together data from the different methods.
What is involved in triangulation is not the combination of different kinds of data per se, but rather an attempt to relate different sorts of data in such a way as to counteract various possible threats to the validity of (their) analysis (HAMMERSLEY & ATKINSON 1983, p.199). [40]
While the social science application of triangulation is widely regarded as having originated in psychology, there is an established argument to the effect that qualitative research, and especially ethnography, is particularly well-suited to triangulation. Many have followed DENZIN's (1970) argument that triangulation should not only involve multiple methods ("data triangulation") but multiple investigators ("investigator triangulation") and multiple methodological and theoretical frameworks ("theoretical and methodological triangulation"). Each of the main types has a set of sub-types in DENZIN's formulation. Data triangulation may include time triangulation, exploring temporal influences by longitudinal and cross-sectional designs; space triangulation, taking the form of comparative research; and person triangulation, variously at the individual level, the interactive level among groups, and the collective level. In investigator triangulation, more than one person examines the same situation. In theory triangulation, situations are examined from the perspective of different theories. Methodological triangulation has two variants, "within-method", where the same method is used on different occasions (without which, one might suggest, one could hardly refer to "method" at all), and "between-method", where different methods are applied to the same subject in explicit relation to each other. [41]
Ethnography nearly always involves collecting different kinds of data (fieldnotes, interview transcripts, documents) from different sources (members, the researcherse.g., through fieldwork diaries, and independent commentators on the setting, e.g., those from another discipline or journalists). BURGESS (1984, p.5) adds to this an important elaboration, that the distinctive thing about ethnography in the context of triangulation is that it involves developing "relationships between the researcher and those researched". Such relationships make available a range of techniques for checking interpretations which arise from the more intimate and sustained nature of this form of fieldwork. [42]
It may be thought that all of this is to disregard the powerful critique of social and behavioural science epistemology brought to bear by postmodernism in recent years. However, one need not subscribe to the notion of absolute standards, objectivity and "truth" to see that triangulation has an important place in the research process. As BREWER (2000, p.76) puts it, "even in this type of (postmodern) ethnography, practitioners recognise that all methods impose perspectives on reality by the type of data that they collect, and each tends to reveal something slightly different about the same 'symbolic' reality". This means that data triangulation is necessary even in the type of ethnography where the applicable criterion is not the achievement of the objective knowledge of the social world, "not as a form of validity ... but as an alternative to validation" (l.c.). [43]
Even for those not in accord with postmodern perspectives, and who are oriented to notions of validity and reliability, triangulation in itself is no guarantee of internal and external validity. For example, let us consider KELLE's (this volume) third empirical case, where a qualitative enquiry took place into the operation of the job placement scheme in former-socialist East Germany, which had been endorsed as effective by (official) statistical analyses. The qualitative study suggested-to some, revealed-that what was in fact happening was that the job placement system was being manipulated by potential employees, who were merely finding their own work using informal channels, then colluding with employers to report a "vacancy" to the job placement scheme, which was then quickly "filled" by the collusive employee, yielding an apparent success for the job placement system. Let us assume that there is no doubt at all of an internal methodological kind about the rigour with which both the statistical analysis of the job placement system and the qualitative study of employees apparently placed by it were conducted. Does this example represent a successful case of triangulation or does it actually mean that we always need qualitative methods, since the quantitative findings do not seem to have contributed anything? [44]
Our answer would be that it is a case justifying the value of triangulationbecause, without the quantitative data providing one version of social reality we would not know how to value or assess those reports from the qualitative study about the workers manipulating the system. In order to identify in our analytic work with the qualitative data that data about the manipulation of the system raised a point worthy of enquiry we had to have the quantitative data suggesting that the official system was operating rather well. Even so, doubt remains. We might, for example, worry that, due to the almost-intrinsically limited scope of qualitative work, our research had simply managed to uncover those few renegade workers who had manipulated the system. One way we could address thatwithin the confines of qualitative methodwould be to inspect the data for talk in which workers reported satisfaction with the state job placement system. Perhaps this balanced the accounts where manipulation was reported? But another way we could address such doubt (and these procedures have their mirror image in studies where the quantitative data repudiate the qualitative data) would be to extend our programme of research to a third step, where, in light of the findings of the qualitative work, we constitute a further quantitative enquiry, but this time instead of using official employment data, we carry out an independent survey which specifically asks questions about the respondents' experience of the job placement process, for example, precisely how they learned of the vacancy which they then filled (i.e., did they hear about it from a friend or see it posted on a job card in the state job placement bureau). In this approach, initial quantitative data gives an official version of reality, this is called into question by qualitative work, and we seek a resolution of the conflicting versions by highlighting the process suggested by the qualitative work and seeking to establish whether it is more widely applicable. [45]
Thus, we might take the more modest view that the real value of triangulation is not that it guarantees conclusions about which we can be confident but rather that it provokes in researchers a more critical, even sceptical, stance towards their data. All too often in qualitative research (and examples exist in quantitative work, too), researchers are drawn to facile conclusions, of the sort which frequently lead outsiders to complain that the main product of social and behavioural research is the confirmation of what everyone knew by commonsense in the first place. Further, when analyses are challenged, qualitative researchers are prone to claim "ethnographic authority" (HAMMERSLEY & ATKINSON 1983), that is, they defend their interpretation not by adherence to systematic, established, externally-validated analytic procedures but by the (usually unassailable) fact that "they were there". They did the fieldwork, they collected the data, therefore they have the "best sense" of what the data may mean. [46]
Such a criterion for warranting inferences is deeply unsatisfactory. Among its several defects is the way it contrasts with the grounds on which the warrant for inferences from quantitative data can be established. Here use is made of statistical procedures whose steps are standardised, so that adherence to each stage can be checked by critics, and whose criteria for drawing a particular conclusion are not only explicit but precisely define the conditions under which a given conclusion can be assumed to hold or to break down. Triangulation offers a means for qualitative researchers to be more discriminating and discerning about their data, to take on the stance so often characteristic of the quantitative researcher, for whom conclusions are always "on test", hold only under specified conditions, and whose relationship to the data is not uncritical "immersion" but measured detachment. [47]
We are not arguing that qualitative researchers need to transform their approach to resemble that of the statistician, but we do argue that when we look at triangulation its value lies more in its effects on "quality control" than in its guarantee of "validity". A further benefit is that this approach promotes more complex research designs and that these oblige researchers to be more clear about what it is they are setting out to study. There will always be value in the relatively diffusely-focussed exploratory study, but as qualitative research gains legitimacy (and there is little doubt that it has done so in recent years in western Europe and in North America; FIELDING & LEE 2000), it increasingly tackles more precisely-specified topics and becomes more prominent in applied spheres such as policy-related research in fields like health behaviour and crime, where relevant research audiences (including research subjects and researchers themselves) want to feel "confidence" in the "conclusions". Indeed, it seems perverse even in purely exploratory work for researchers to be indifferent to the accuracy of their analyses. One might even argue that it is incumbent on researchers exploring hitherto obscure corners of the social world to employ research designs which accurately depict what has previously been unknown and which has thus far proved resistant to study by more conventional means. [48]
In that triangulation is much about the comparison and integration of data from different methods it is worth reminding ourselves of SIEBER's (1979) seminal argument on what qualitative work can do for quantitative work and what quantitative work can do for qualitative work. Bearing in mind that SIEBER's approach is grounded in a firmly positivist perspective, and beginning with data collection issues, qualitative work can assist quantitative work in providing a theoretical framework, validating survey data, interpreting statistical relationships and deciphering puzzling responses, selecting survey items to construct indices, and offering case study illustrations. In some cases the theoretical structure itself is a product of field experience. For SIEBER, quantitative data can be used to identify individuals for qualitative study and to delineate representative and unrepresentative cases. Turning to data analysis, SIEBER maintains that quantitative data can correct the "holistic fallacy" that all aspects of a situation are congruent, and can demonstrate the generality of single observations. Field methods sometimes suffer "elite bias", an over-concentration on certain respondents due to their articulacy, strategic placing in terms of access, and because researchers like to share their high status. Quantitative data can deal with this fault by indicating the full range that should be sampled. Among the things that SIEBER suggests qualitative data can contribute to quantitative research are depth, an idea of the range of core concepts, and the ability to solve puzzles that the more superficial quantitative data cannot address. [49]
It is worth making explicit that accepting the case for interrelating data from different sources is to accept a relativistic epistemology, one that justifies the value of knowledge from many sources, rather than to elevate one source of knowledge (or more accurately, perhaps, to regard one knowledge source as less imperfect than the rest). Those taking an approach favourable to triangulation in conventional terms are more likely to work from a perception of the continuity of all data-gathering and data-analysing efforts (e.g., as several of our contributors hold, to perceive that all data analysis involves "interpretation"). They are more likely to regard all methods as both privileged and constrained: the qualities that allow one kind of information to be collected and understood close off other kinds of information. [50]
It is important, then, not to be led by an enthusiasm for multiplying sources of information into forgetting to monitor the biases to which each method is susceptible. The conventional logic of triangulation's multiple sources of information is that by using several we can diversify biases in order to transcend them. We use a variety of independent methods with predictable and different characteristic kinds of error so we can look for things which are invariant or identical in the data which have been produced using different knowledge sources. But it is not just the search for points of co-incidence or agreement. In this conventional approach to triangulation, we have further to identify the scope of the processes across which they are invariant, the conditions under which the invariance occurs. We also need to explain failures of invariance, why given limits or conditions apply. It follows that the differences between findings from different knowledge sources can be as analytically illuminating as their points of coherence (as in, for example, the third empirical study in KELLE's contribution to this volume). [51]
Two main sources of bias are apparent in qualitative work: the tendency to select field data to fit a preconception of the phenomenon and how it should be analysed, and a tendency to select field data for analysis which are conspicuous because they are exotic at the expense of less dramatic, but possibly more indicative, data. While the rigidity of positivist methods helps researchers resist these faults, such work is not free of such problems either. But what makes it easier for quantitative researchers to trace such faults is that the character of the data, and the necessity to state hypotheses, make the researcher's assumptions more explicit and available for inspection by third parties. However, systematic observation can have some of the advantages of the survey, as in HUMPHREYS' (1970) study of impersonal sex in public toilets. He completed "fact-sheet" descriptions for each observation, later augmenting these with conventional fieldnotes, and claimed that this strategy gave "objective validity" to his data. It would be more accurate to say that a quality control mechanism was built into the data by incorporating into the data physical descriptors that could be checked. The point is that the introduction of a systematic element to the field observation facilitated attention to replication and comparison in a similar way to that normally associated with survey work. [52]
The advantages of combining methods should not lead researchers to subordinate their awareness that different approaches are supported by different epistemologies and logical assumptions, which require their handling by different terminologies. Results from different methods founded on different methods may, then, be combined but for a different purpose than that associated with the established approach to triangulation. Theoretical triangulation does not necessarily reduce bias, nor does methodological triangulation necessarily increase validity. Competing theories are generally the product of different traditions, so when combined they may offer a fuller picture but not a more "objective" one. Likewise, different methods draw on different (and often competing) epistemologies and while combining them can add range and depth it does not necessarily add accuracy. In this approach, when we combine theories and methods we do so to add breadth or depth to our analysis, not to pursue an "objective" truth. [53]
Rejecting absolute versions of truth, and the feasibility of absolute objectivity, is not the same as rejecting the standard of truth or the attempt to be objective. In things social and behavioural, our knowledge is always partial and intrinsically incomplete. We accept the abstraction or conclusion-with-identifiable-and-defined limits as invitational, suggesting implicitly the "constant and unevadable necessity for interpretation and change of aspect" (NEEDHAM 1983, p.32). This is, ultimately, the warrant for the triangulation paradigm. [54]
1) Formalisation is also at the core of the contribution by Lutz-Michael Alisch which will be added to this volume at a later time. <back>
Bailey, Kenneth D. (1994). Typologies and taxonomies. An introduction to classification techniques. Thousand Oaks etc.: Sage.
Bhaskar, Ronald (1975). A realist theory of science. Leeds: Leeds Books.
Bourdieu, Pierre (1996). Toward a reflexive sociology: a workshop with Pierre Bourdieu. In Stephen P. Turner (Ed.), Social Theory and Sociology: the classics and beyond (pp. 213-228). Oxford: Blackwell.
Breuer, Franz (2000, September). Über das In-die-Knie-Gehen vor der Logik der Einwerbung ökonomischen Kapitals-wider bessere wissenssoziologische Einsicht. Eine Erregung. Zu Jo Reichertz: Zur Gültigkeit von qualitativer Sozialforschung [18 paragraphs]. Forum Qualitative Sozialforschung / Forum Qualitative Social Research [On line Journal], 1(3). Accessible via: http://www.qualitative-research.net/fqs-texte/3-00/3-00breuer-d.htm [Access: 14.12.2000].
Burgess, Robert (1984). In the field. London: Routledge.
Denzin, Norman (1970). The research act. Chicago: Aldine.
Diesing, Philip (1971). Patterns of discovery in the social sciences. Chicago: Aldine.
Fielding, Nigel & Fielding, Jane (1986). Linking data: the articulation of qualitative and quantitative methods in social research. Beverly Hills, London: Sage.
Fielding, Nigel & Lee, Ray (2000). Patterns and potentials in the adoption of qualitative software: the implications of user experiences and software training (Social science methodology in the new millennium: Proceedings of the Fifth International Conference on Logic and Methodology, CD ROM. Köln: Zentralarchiv fur Empirische Sozialforschung, 2000).
Geertz, Clifford (1973). The interpretation of cultures: selected essays. New York: Basic Books.
Gergen, Kenneth (1985). The social constructionist movement in modern psychology. American Psychologist, 40(3), 266-275.
Groeben, Norbert & Rustemeyer, Ruth (1994). On the integration of quantitative and qualitative methodological paradigms (based on the example of content analysis). In Ingwer Borg & Peter Mohler (Eds.), Trends and perspectives in empirical social research (pp.308-325). Berlin/New York: de Gruyter.
Groeben, Norbert & Scheele, Brigitte (2000). Dialogue-hermeneutic method and the "research program subjective theories" [18 paragraphs]. Forum Qualitative Sozialforschung / Forum Qualitative Social Research [On line Journal], 1(2). Accessible via: http://www.qualitative-research.net/fqs-texte/2-00/2-00groebenscheele-e.htm [Access: 30.01.2001].
Hammersley, Martin & Atkinson, Paul (1983). Ethnography: principles in practice. London: Tavistock.
Harré, Rom (1970). The principles of scientific thinking. London: Macmillan.
Humphreys, Laud (1970). Tearoom Trade. London: Duckworth.
Kluge, Susann (1999). Empirisch begründete Typenbildung. Zur Konstruktion von Typen und Typologien in der empirischen Sozialforschung. Opladen: Leske & Budrich.
Kuckartz, Udo (1995). Case-oriented quantification. In Udo Kelle (Ed.), Computer aided qualitative data analysis. Theory, method, and practice (pp.158-166). London etc.: Sage.
Levins, Robert (1966). The strategy of model building in population biology. American Scientist, 54, 420-440.
Morgan, David (1998). Practical strategies for combining qualitative and quantitative methods: applications to health research. Qualitative Health Research, 8(3), 362-76.
Needham, Rodney (1983). The tranquillity of axiom. Los Angeles: University of California Press.
Reichertz, Jo (2000). Zur Gültigkeit von qualitativer Sozialforschung [76 paragraphs]. Forum Qualitative Sozialforschung / Forum Qualitative Social Research [On line Journal], 1(2). Accessible via: http://www.qualitative-research.net/fqs-texte2-00/2-00reichertz-d.htm [Access: 14.12.2000].
Rustemeyer, Ruth (1992). Praktisch-methodische Schritte der Inhaltsanalyse. Münster: Aschendorff.
Schreier, Margrit (in press). Qualitative methods in studying text reception. In Dick Schram & Gerard Steen (Eds.), The psychology and sociology of literature. In honor of Elrud Ibsch (pp.35-56). Amsterdam: Benjamins.
Sieber, Sam (1979). The integration of fieldwork and survey methods. American Journal of Sociology, 78(6), 1335-1359.
Van Maanen, John (1979). Special issue on non-quantitative methods in management research. Administration Science Quarterly.
Webb, Elizabeth; Campbell, Donald; Schwartz, Robert & Sechrest, Lee (1966). Unobtrusive measures: non-reactive research in the social sciences. Chicago: Rand McNally.
Nigel FIELDING is Professor of Sociology and co-Director of the Institute of Social Research at the University of Surrey. He has taught field methods and criminology at Surrey since 1978. His research interests are in qualitative methods, new research technologies, and criminal justice. He was Editor of the Howard Journal of Criminal Justice from 1985 to 1998, and is co-editor of the series "New technologies for social research" (Sage). He has published twelve books, four of them on aspects of methodology, and is currently working on the second edition of Computer programs for qualitative data analysis (with E. WEITZMAN and R. LEE) and a four volume set on Interviewing, both for Sage.
Institute of Social Research
Department of Sociology
University of Surrey
Guildford GU2 7XH, England
E-mail: n.fielding@surrey.ac.uk
Margrit SCHREIER, Dr. phil., is research assistant at the Department of General and Cultural Psychology, University of Cologne. Her current research interests include: methodology of the social sciences, combination of qualitative and quantitative methods, research program subjective theories, media psychology and media studies, psycholinguistics, attribution of responsibility, and the empirical study of literature.
Universität zu Köln
Psychologisches Institut
Lehrstuhl f. Allgemeine Psychologie und Kulturpsychologie
Herbert-Lewin-Str. 2
D - 50931 Köln, Germany
E-mail: m.schreier@uni-koeln.de
Please cite this article as follows (and include paragraph numbers if necessary):
Fielding, Nigel & Schreier, Margrit (2001, February). Introduction: On the Compatibility between Qualitative and Quantitative Research Methods [54 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal], 2(1). Available at: http://www.qualitative-research.net/fqs-texte/1-01/1-01hrsg-e.htm [Date of Access: Month Day, Year].
|