"Meta Interpretation": A Method for the Interpretive Synthesis of Qualitative Research

Abstract: Recognition of the need for good research synthesis dates back almost half a century (c.f .WRIGHT MILLS, 1959), although it is far more recently that specific methods of synthesis have been developed. NOBLIT and HARE (1988) argue that such methods have emerged because of the failings, in the eyes of both positivists and interpretivists, of traditional literature reviews which, while giving an overview of the field, are often descriptive and are rarely able to make sense of what the collection of studies reviewed has to say. The purpose of this paper is to propose a method for meta-interpretation which focuses on the interpretive synthesis of qualitative research, thus maintaining an interpretive epistemology that is congruent with the majority of primary qualitative research. The paper reviews and evaluates eight research methods or approaches that include some form of synthesis (literature review, systematic review, meta-analysis, meta-ethnography, grounded theory, cross-case comparison, secondary analysis of primary data, and interpretive phenomenological analysis). The key features of each approach are drawn out, and their implications for the construction of the meta-interpretation approach are discussed. The paper then outlines a potential procedure for meta-interpretation before concluding with some comments on the functions of synthesis in general and meta-interpretation in particular.

Basic research synthesis, the bringing together for discussion of the results, findings and conclusions of previous studies, is part of almost every research project. Research starts with a survey of relevant literature in the field, and a discussion of this is then presented to provide a context for the primary investigations that follow. However, it is rare for this type of secondary review of research to be presented in anything other than a descriptive way, a simple tour around previous work that highlights key issues and perhaps identifies gaps in the literature that primary research might fill. WOOD (2000) dismisses these reviews as pseudo-synthesis which, while not valueless, "are really little better than annotated bibliographies" (p.416). It is not surprising, therefore, that in recent years in a range of diverse fields there has been some frustration with these types of descriptive reviews, and thus interest in the synthesis of previous work as a primary research activity has developed. [1]

Leading the way, at least in the synthesis of quantitative work, has been the field of psychology, which has made widespread use of the meta-analysis technique initially developed in a seminal paper in the field of education by GLASS (1976). The assumption of this meta-analysis technique is that: "The findings of multiple studies should be regarded as a complex data-set, no more comprehensible without statistical analysis than would hundreds of data-points in a single study" (GLASS, McGRAW & SMITH, 1981, pp.12). Consequently, previous studies are regarded as data for analysis in such statistical meta-procedures. The need for the secondary statistical analysis of such data was highlighted by HUNTER, SCHMIDT and JACKSON (1982) who noted that the need in many areas of psychology is "not for additional empirical data but some means of making sense of the vast amounts of data that have accumulated" (p.27). As such, statistical meta-analytical techniques are seen by psychologists as not only offering a more objective means of synthesising the findings of previous research, but also as a way of moving literature reviews closer to the standards of scientific enquiry and repeatability that are applied to individual studies (WOOD, 2000; WOLF, 1986). Of course, this is only helpful if the studies that are being synthesised are quantitative in nature. [2]

The meta-analysis technique has also been utilised in medical science (e.g. BUCHER, HENGSTLER, SCHINDLER & GUYATT 2000), with a particular focus on aggregating evidence from what is seen as the gold standard for quality of the randomised controlled trial. However, medical science has also made widespread use of the systematic review (OXMAN, 1994) procedure to accumulate the best evidence (COOK, MULROW & HAYNES, 1997) for clinical decisions. Systematic reviews develop protocols to identify relevant research in a particular area, and are seen by those who use them as being more systematic and less subjective than traditional literature searches. However, much of this work is once again quantitative in nature. Nevertheless, there have been attempts to modify the systematic review procedure to incorporate qualitative work, not only in medicine, but also, and in a more widespread way, in policy and management. [3]

In Britain central government has, since the 1980s, placed an increasing emphasis on ensuring that policy and practice are informed through "a more rigorous and challenging evidence base" (TRANFIELD & DENYER, 2002). There has been a general concern for the effectiveness of service delivery, and since 1997 an evidence-based movement has developed under the New Labour regime of the British Prime Minister, Tony Blair (DAVIES, NUTLEY & SMITH, 1999a). This has seen the utilisation of best evidence systematic reviews to inform policy in urban regeneration (DABINETT, LAWLESS, RHODES & TYLER, 2001), nursing (EVANS & PEARSON, 2001), housing (DAVIES, NUTLEY & SMITH, 1999b; MACLENNAN & MORE, 1999), social care (MACDONALD, 1999) and criminal justice (LAYCOCK, 2000). Furthermore, in 2001 the Economic and Social Research Council (ESRC) funded the establishment of an Evidence Network of multi-disciplinary centres dedicated to the improvement of the evidence base for policy and practice in the social sciences. [4]

On a broader plain, there is a general concern that too little use is made of existing research in the social sciences, a point made by C. WRIGHT MILLS almost half a century ago: "There are never enough bricks and there are too few good synthesisers who wish to search out the bricks and thus put the wall together. These worthy people are usually too busy working on their own data!" (WRIGHT MILLS, 1959, p.65) NIEMI (1986) noted that single studies can rarely provide satisfactory answers to research questions, and that the need is for approaches that can extract the underlying trends and principles developed from the accumulation and refinement of a large body of studies. However, SOLESBURY (2002, p.92) reinforces the point made by WRIGHT MILLS in highlighting the preference for primary investigation over greater research utilisation

"Most research effort is expended on new primary research and yet, on virtually any topic you can name, there is a vast body of past research that may have some continuing value but mostly remains ignored. Social Science is very bad at the cumulation and re-use of past research results." [5]

Yet, as outlined above, there has been an increasing interest in both research synthesis and "the fuller exploitation of existing data and research findings" (SOLESBURY, 2002, p.90). However, such interest has largely led to the development of quantitative methods of synthesis, underpinned by a positivist epistemology. [6]

Much primary qualitative research is underpinned by an interpretivist epistemology, whilst most of the attempts to synthesise qualitative research are largely derived from positivist approaches. This creates a tension which can often lead to the quantification of qualitative findings for the purposes of synthesis. For example, some work on the incorporation of qualitative data into systematic reviews has seen the extension of a Bayesian approach to meta-analysis, which includes prior probabilities derived from qualitative research in the synthesis of quantitative research (ROBERTS, JONES, ABRAMS, DIXON-WOODS & FITZPATRICK, 1998). There are two issues in such an approach: firstly, the reduction of qualitative findings to a statistical measure of probability will result in the loss of much of the richness of the qualitative research; secondly, the approach relegates qualitative research to a secondary or supporting role to quantitative data largely, it would seem, because quantitative data is easier to synthesise. [7]

Other approaches to qualitative synthesis have focussed on the re-analysis of qualitative data sets (e.g. HEATON, 1998; WEED, 2002). However, such approaches can be problematic because, unless the synthesis is being conducted by the original researcher, qualitative data sets are usually not available in full, and so a full secondary analysis is not possible. In the examples given above, HEATON conducted her secondary analysis of one dataset (although she has since conducted "amplified analyses"), whilst WEED was able to draw on a wider range of full datasets from work in his own and partner institutions by Masters, Doctoral and Post-Doctoral researchers to draw up a typology of participation in sports tourism. Generally, however, there are two barriers to synthesising qualitative research in this way. The first is that most syntheses draw on published research articles, and the qualitative data found in such articles is almost always, and by necessity, illustrative rather than comprehensive. As such, unlike the summary statistics presented in quantitative research, there is no overall data measure, nor are there full datasets readily and widely available for most qualitative research. The second barrier is related to the first in that such secondary re-analysis, as a result of the need for full qualitative data sets, is necessarily restricted to research carried out by the researcher, or by his or her colleagues or research collaborators, where there is access to the full qualitative data set. This restricts the synthesis both in terms of the number of studies that can be included, and in terms of the limited number of perspectives that will have been brought to bear on the research area. [8]

Issues surrounding the re-analysis of qualitative data also highlight a further tension linked to the epistemological underpinnings of qualitative synthesis. As mentioned earlier, most qualitative synthesis methods have applied positivist synthesis to interpretive research. The aim of this paper is to assess the potential to conduct a synthesis that is underpinned by the same interpretive epistemology as much qualitative research, therefore remaining true to the epistemology of the research being synthesised. However, a key problem here is that an interpretive approach, by its very nature, will mean that a comprehensive coverage of previous research in the area, as aimed for by systematic review, is not possible, and as a result, even a representative sample is not possible. The tension is between epistemology and sample size (WEED, 2004)—how can a synthesis of qualitative research be achieved that adheres to an interpretivist epistemology whilst also achieving a large enough sample size to make the synthesis a worthwhile exercise? [9]

The aim of this paper is to develop a method for the interpretive synthesis of qualitative research which, given the focus on interpretation, is termed meta-interpretation. As such, the various issues outlined above will be returned to in the next section of the paper, which reviews eight research methods or approaches that include some form of synthesis. These methods are not all methods of synthesis, but they do all include some form of synthesis of data as a core part of the approach. The key features of each approach are drawn out, and their implications for the construction of the meta-interpretation method are discussed. Following this review, a number of fundamental features of "meta interpretation" are outlined and, finally, an attempt is made at suggesting a structure for such an approach. [10]

Eight methods incorporating some form of synthesis are discussed in this section. The discussions are not intended to be comprehensive and, as such, some methods that incorporate synthesis are not included here—e.g. meta-evaluation (WOODSIDE & SAKAI, 2001) and analytic induction (MANNING, 1982). Some of the methods identified involve synthesis across studies by unconnected researchers, whilst others involve synthesis between studies by the same researcher or research team. The eight methods and their key characteristics are briefly summarised in Table 1.

The four methods at the top of Table 1 are aggregative, whilst the four methods at the bottom of the table utilise primary data sets that have usually been collected by the same researcher or research team (although this is not always the case with secondary analysis). Of the aggregative methods, the two that aim for comprehensive sampling (meta-analysis and systematic review) are underpinned by positivistic epistemologies. Meta-ethnography is informed by an interpretive epistemology, but uses only a very small number of studies (2-5). IPA and grounded theory are also interpretive, but are usually restricted to the data/theories collected/established by the original researcher. The features of each of these methods and the issues they raise for a meta-interpretation approach to the synthesis of qualitative research are now discussed in more detail. [12]

The top row of table one moves from clearly positivistic methods on the left, to more interpretive methods towards the right. However, it would seem sensible to commence these discussions with the method of research synthesis that most researchers will be aware of and most familiar with, the traditional literature review. [13]

Whilst it has been included in this review, the literature review differs from the other methods of synthesis in a number of ways. Firstly, it is included in almost every research study as a scoping exercise that sets the context for the research to be conducted. Indeed, as such, the outcome of the review of literature is often set before it is conducted because the researcher has usually decided what research he or she wants to carry out, and then uses the literature review as a way of establishing the need for that research. As such, literature reviews are often presented as arguments for a particular point of view, or as a justification for particular research. Consequently, they rarely present a balanced view of a field, nor do they identify the full range of salient issues. Furthermore, literature reviews, in a number of cases, can lack analysis or evaluation. As suggested in the introduction, such reviews are little more than "annotated bibliographies" (WOOD, 2000, p.416) presented along the lines of: "Johnson suggests that X is an important motivation for Y; however, Jones believes that Z is the primary motivator in these cases". Such reviews involve no evaluation, and merely present a tour of research that the researcher sees as relevant to the research he or she is about to conduct. [14]

A second way in which the literature review differs from the other methods of synthesis in Table 1 is that it is rarely conducted as a primary research activity in its own right. That said, there have been a number of ground breaking literature reviews that have been published in their own right, and that have been influential in shaping fields of study. Furthermore, such reviews have rarely shown the deficiencies of many traditional literature reviews discussed in the previous paragraph. In fact, some authors (e.g. JONES, 2004) have argued that properly conducted systematic narrative literature reviews, which focus on the narrative itself as the method of review, can be more appropriate for the synthesis of qualitative research than systematic review methods aped from quantitative research. However, such systematic narrative literature reviews are rare, and while the use of narrative may be congruent with the meta-interpretation approach, the role of the traditional literature review seems set to remain, for the most part, as a context-setting justifying prelude to primary research. [15]

For many researchers, particularly those in policy and health care/medicine, the systematic review procedure has been seen as the logical alternative to the traditional narrative literature review. Systematic reviews have been used to compile the "best evidence" (COOK et al. 1997; TRANFIELD & DENYER, 2002) for clinical decisions and for policy making. They are often contrasted to traditional literature reviews because they are seen as objective, replicable, systematic and comprehensive. KLASSEN, JAHAD and MOHER (1998) define the systematic review as follows: "A systematic review is a review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesised according to a pre-determined explicit method" (p.700; emphasis added). [16]

The key features of the systematic review are italicised in the above quotation, and it is perhaps useful to take each of them in turn in aiming to understand the nature of the method. The first feature of a systematic review is that it aims for comprehensive coverage of a particular area; however, the second feature is that such comprehensiveness is within boundaries as reviews also seek to identify relevant studies. Initially, this is usually done through a search of various electronic databases and hand-searches of journals. The returns from the first round of searches can be overwhelming (often over 10 000 articles) and consequently, exclusion criteria are established, often by an expert panel, to reduce the studies down to a more manageable number. Those conducting systematic reviews believe the existence of pre-determined exclusion criteria reduces the likelihood of bias from the investigator(s) during the selection process (EVANS & CHANG, 2000). However, their rigid application can mean that important studies are excluded from the review because they do not conform to the investigator's or the expert panel's understanding of what is important or relevant. Of course, the need for pre-determined exclusion criteria is derived from the underlying nature of systematic review as a positivistic enterprise that attaches great significance to the scientific and objective nature of the procedure. As the aim in this paper is to develop a method underpinned by an interpretivist epistemology, any such method is not constrained by the accolade of science. In fact, meta-interpretation would see the location of the investigator in the review process as an important and vital part of the synthesis process. [17]

The final feature of the systematic review is the appraisal of studies according to pre-determined criteria. Simply put, this is the judgement of the quality of the studies included in the review (BOAZ, ASHBY & YOUNG 2002). As with exclusion criteria, quality criteria are pre-determined, and often relate to the nature of the method used. In some systematic reviews in the field of medicine, only studies conforming to what is perceived to be the gold standard of the randomised controlled trial are included. Yet, while this is not always the case, there is always an agreed set of quality criteria which studies have to satisfy in order to be included in the review. While on the surface this may seem sensible, it raises two problems. Firstly, studies including important and robust findings may often utilise unorthodox yet perfectly valid methods. Such studies would be excluded from many systematic reviews. Secondly, some studies, whilst perhaps being methodologically flawed in part, or with overall findings that do not appear important or relevant, may still be able to offer important insights into phenomena, or may contain important findings in parts of their investigations which are not reflected in the results of such studies as a whole. Again, such studies are lost to the systematic review. In developing the meta-interpretation approach, the value of studies is not pre-determined, rather, as recommended by NOBLIT and HARE (1988, p.16) "the worth of studies is determined in the process of achieving a synthesis". Consequently, no studies are pre-excluded, but they may be excluded during and after the analysis on the basis of criteria that are established during the analysis. In systematic review, which aims for comprehensive coverage, this approach would be impossible. However, meta-interpretation (as discussed later) does not aim for comprehensive coverage and, as such, can take a more ideographic approach to the exclusion of studies. [18]

It may seem strange to include meta-analysis in a paper that seeks to develop meta-interpretation as a method for the interpretive synthesis of qualitative research, since meta-analysis is clearly a quantitative statistical procedure. However, there are a number of issues that arise from a discussion of meta-analysis that might contribute to the development of the meta-interpretation approach. [19]

It is useful to consider meta-analysis after a discussion of systematic review, because in a number of cases meta-analysis is the method of analysis that follows the systematic review procedure (EVANS & CHANG, 2000). In fact, in some areas of the medical sciences, the two are seen as synonymous. However, meta-analysis should only be used when studies are similar in terms of their population, address the same substantive issue, and use the same statistical procedure and manipulations (WOLF, 1986). Invariably, the number of studies included in a meta-analysis will be smaller than would be included in a systematic review of the same area because of the need for comparable statistics and populations. [20]

The perception of meta-analysis is that it is a method that estimates an overall effect-size of a range of studies from the individual effect sizes of each individual study, thus giving greater power to the overall statistic (WOOD, 2000). It does this by calculating a mean of means of means: in the original study, a mean is taken of the effects of a particular variable for all subjects in the study, then variables are averaged to provide an overall effect size (mean) for that study, and then the effect sizes of a number of studies are averaged in the meta-analysis procedure. However, meta-analysis also uses statistical procedures to correct for bias in studies which may arise from a range of sources, but most commonly from sampling error, measurement error and range restriction. Correcting for such statistical artefacts allows meta-analysts to highlight inconsistencies across studies that are due to deficiencies in the theoretical perspectives used, rather than arising from such statistical artefacts, and commenting on such inconsistencies is a lesser known, but equally important, function of meta-analysis (BIDDLE, MARKLAND, GILBOURNE, CHATZISARANTIS & SPARKES, 2001). [21]

In drawing lessons from meta-analysis for the meta-interpretation approach, it is useful to consider the idea of correcting for bias/error and highlighting inconsistencies between studies. Part of the function of meta-analysis is to correct for differences between studies that are the result of bias, and to highlight differences that are unexplainable by the theory or framework being used. In relation to meta-interpretation, the same principles can be applied. On one hand meta-interpretation should aim to highlight differences between studies that are the result of different data collection methods or even of different researchers. Such differences should not be corrected for, but acknowledged in the analysis, and in many cases celebrated, because differences in data collection methods and researcher approaches can be important in obtaining insights that have not featured in other studies. On the other hand, meta-interpretation should focus on highlighting differences in studies that are not accounted for in the immediate situation. In many cases this will require an examination of context, as it is "meaning in context" (MISHLER, 1979) that will allow an effective synthesis of qualitative studies to take place. In the language of statistics, context is the independent variable. It is this awareness of difference, and the need to understand the reasons for such difference, rather than correct for it, that forms a key part of the analysis stage of the meta-interpretation approach. [22]

Meta-ethnography shares the same meta prefix with meta-analysis, but that is where the similarity ends. Meta-ethnography as an approach comes closest to the meta-interpretation method for the interpretive synthesis of qualitative research being developed here. However, there are a number of areas in which it falls short, not least of which is its restriction to the synthesis of two to five studies which, in the original monograph describing the procedure (NOBLIT & HARE, 1988), were largely conducted by research teams working on the same broad project. Nevertheless, there are important considerations arising from the meta-ethnography procedure that have important implications for the development of a meta-interpretation approach. [23]

The meta-ethnography procedure recognises the central importance of meaning in context highlighted in the previous section. VAN MANNEN, MANNING and MILLER (1988, p.5) note that an important part of interpretive qualitative research is that it: "... presumes a social and theoretical context within which substantive findings emerge. The recovery of this context, and its comparison across studies are the aims of meta-ethnography". [24]

A key difference between meta-ethnography and many other approaches to synthesis is that meta-ethnography does not use the primary data collected through interviews and observations as the raw data for the synthesis. The subject of the synthesis in meta-ethnography is the interpretations of the data (DOYLE, 2003). This has two advantages for the meta-interpretation approach. Firstly, the interpretations from almost all qualitative studies are included in published works, unlike the full raw dataset from interviews, observations and fieldnotes. While this is clearly a pragmatic reason for focussing on interpretations in the synthesis of qualitative research, it is no different from the pragmatism of excluding studies on the basis of what are often arbitrary criteria in order to achieve manageable numbers of studies in systematic review. Furthermore, there is a second, more substantive reason for the focus on interpretations related to the idea of meaning in context. RANTALA and WELLSTROM (2001) have noted that the re-analysis of inherited secondary data can be problematic because "the researcher conducting the re-analysis might understand the data differently from its collector" (p.88). As a consequence, meaning in context is lost. However, if the interpretations of the original researcher are used, then the focus on meaning in context is retained. The interpretations can convey such meaning in context, whereas raw data cannot (see later discussions on secondary analysis of primary data for more on this). [25]

The core of meta-ethnography is reciprocal translation, by which is meant the interpretations of studies are "translated into one another" (NOBLIT & HARE, 1988, p.11). Essentially, an attempt is made to establish a common language of interpretation. NOBLIT and HARE (1988, p.36) see the tool for this as being metaphoric reduction: "It is only through metaphoric reductions that we can achieve both abstraction and complexity, and create translations that preserve the relations between concepts". However, metaphoric reduction is, by necessity, a reductivist method, and a strange approach to use when it is also claimed that there is a close focus on the recovery and comparison of context. This has been highlighted by RIST (1990) who questions whether meta-ethnography can "capture the richness and depth of understanding of natural settings that so characterises the strength of qualitative work" (p.336). Consequently, whilst many of the principles of meta-ethnography are congruous with those of meta-interpretation, the reductivist nature of the approach is a clear point of difference. [26]

As noted earlier, the four methods of synthesis discussed so far are aggregative, in that they do not use data that has been collected for the purpose of synthesis, and the synthesis is usually conducted by a researcher other than the investigator who originally conducted the research. Grounded theory, cross-case comparison, secondary analysis of primary data and interpretive phenomenological analysis all involve some form of synthesis of data, but such data is usually from the same research project and, with the exception of secondary analysis, has usually been collected for the purpose of synthesis or comparison. [27]

There are a range of aspects of the grounded theory approach that might be relevant to meta-interpretation. Firstly, a key part of the approach is the aim of moving from substantive theories, grounded in particular research contexts, to a more generic formal theory with a broader application (GLASER & STRAUSS, 1967). Clearly this involves making linkages between a range of substantive theories and synthesising their findings to arrive at a broader formal theory. In making such linkages, a number of the tools of the grounded theory approach may be used, not least theoretical sampling and constant comparison. [28]

In making constant comparisons, a mini-synthesis is taking place as progress is made towards a substantive (and, perhaps later, a formal) grounded theory. Furthermore, if enough data does not exist on which to base the analysis, then more is sampled according to the issues that have emerged so far. In the grounded theory method, this form of "theoretical sampling" is a core part of the approach (STRAUSS & CORBIN, 1990), and it may hold the answer to the problem of selecting studies for inclusion in meta-interpretation. Rather than aiming for a comprehensive, or even representative, sample of studies in a particular area, an effort is made to include those studies that are theoretically relevant based on the analysis to date. Of course, the obvious question then becomes: how are the initial set of studies selected. Here, once again, one of the cornerstones of grounded theory may provide the answer. [29]

Grounded theory is based on the constant interplay of theory and data, and a iterative approach to the theoretical sampling of data where additional data is sampled in a number of iterations of data collection, until the analysis is saturated, by which is meant no further additional insights are emerging (PIDGEON & HENWOOD, 1996). However, in order to commence the analysis, grounded theorists develop a basic knowledge of the nature of the area under investigation, and this is called "theoretical sensitivity" (GLASER, 1978). Theoretical sensitivity allows researchers to enter a research site with an awareness of the area but, importantly, without having developed any pre-conceived notions about what they might discover. Similarly, meta-interpretation relies on the synthesiser having a broad awareness of the field to be synthesised, and thus being able to use that knowledge to select an initial sample of around four studies for the first round of synthesis. This initial selection of studies should be made on the basis of "maximum variation sampling" (LINCOLN & GUBA, 1985, p.233) of those studies that provide the greatest "opportunity to learn" (STAKE, 2000). In line with the general approach of theoretical sampling, "the choice of cases is usually made on conceptual grounds, not on representative grounds" (MILES & HUBERMAN, 1994, p.29). [30]

Therefore, following the selection of an initial set of studies for meta-interpretation, an iterative process of analysis and theoretical sampling takes place, thus increasing the breadth and depth of the sample of studies being synthesised. However, "the aim of theoretical sampling is to refine ideas, not to increase the size of the original sample" (CHARMAZ, 2000, p.519), and so when theoretical sampling ceases to add any further insights to the analysis, theoretical saturation can be assumed to have been reached, and the final synthesis can take place. [31]

It is this iterative approach to theoretical sampling that is the major contribution of grounded theory to the meta-interpretation approach, as it resolves the tension between retaining an interpretive epistemology, and conducting a synthesis of a large enough sample to make the synthesis worthwhile. Its use will be developed further in the final section which describes in full a potential procedure for meta-interpretation. [32]

Methods of cross-case comparison come in a number of forms. Such methods are usually utilised in multi-site studies where researchers or research teams have investigated a number of related cases, and an analysis that identifies the common features of such cases is conducted (MILES & HUBERMAN, 1994). Cross-case comparison may be based on both qualitative and quantitative data, but the methods of comparison will almost always be conducted along quantitative lines. [33]

At its simplest, cross-case comparison involves drawing up a matrix of features that have been found to be present in the cases, and marking whether each feature is present or not present in each case. Consequently, the ways in which cases differ or are similar are identified. However, when the analysis is derived from detailed qualitative descriptions—"thick descriptions" (GEERTZ, 1973)—of the cases, then cross-case comparison can be a reductivist method that is based on the quantification of qualitative data. In fact, there have been methods of synthesis, such as Qualitative Comparative Analysis (e.g. RAGIN, 1989; RANTALA & WELLSTROM, 2001), which take this quantification even further and make use of Boolean algebra to assess the relationship between the particular features of individual cases. Such quantification of qualitative data would have no place in the meta-interpretation approach as it results in the loss of much of the richness of qualitative research. [34]

Some of the problems relating to the secondary analysis of primary data have already been discussed earlier in this paper. Chief among these are the problems surrounding gaining access to full qualitative datasets when much published qualitative research includes only illustrative data. However, the discussions of meta-ethnography indicated that one way to overcome this problem is to focus on the interpretations of the data as the primary subject for secondary analysis. Not only are such interpretations widely available, but they can also convey "meaning in context" (MISHLER, 1979) that can be difficult to draw out from raw data. As issues relating to the secondary analysis of primary data have already been discussed, the rest of this section will focus on the secondary analysis of interpretations, and specifically on issues of validity and reliability arising from such an analysis. [35]

An important point to note at the outset is that the re-interpretation of original research is not a valid way to proceed. Because original interpretations have been made in a research context that cannot be re-captured, it is not possible to re-interpret the original findings and retain a focus on meaning in context. The context in which the research is located will be inextricably tied in with the original interpretations. Consequently, the original interpretations must be trusted, or the study should be excluded from the analysis. [36]

Earlier discussions of the systematic review procedure highlighted the need for an ideographic development of criteria for the exclusion of studies from the analysis in meta-interpretation. Such an ideographic approach, where the place of studies in the synthesis is determined in the analysis, allows for the quality control of the synthesis procedure. Terms such as validity and reliability are often used in qualitative research, but in a way that bears little resemblance to their meaning in quantitative statistical research (SPARKES, 2002). Consequently, the broader term research quality is preferred here, and this is taken as referring to ensuring the quality and integrity of the meta-interpretation approach. There are two issues surrounding research quality in meta-interpretation. Firstly, ensuring the quality of the studies included in the synthesis. As noted above, the original interpretations cannot be re-conducted, therefore these interpretations must be trusted. If during the analysis the interpretations of the original researcher appear flawed, then the study concerned should be excluded from the synthesis. Similarly, if it becomes impossible during the analysis to adequately include one or more of the studies concerned in the synthesis, then such studies should be excluded on the basis of being beyond the boundaries of applicability of the synthesis (PAWSON, 2001). In each of these cases, such exclusions will have been made to secure the integrity of the synthesis and to ensure its quality. [37]

The second issue surrounding research quality is ensuring the integrity and quality of the synthesis itself. The key to this lies in the open and transparent nature of the procedure, and in leaving a clear "audit trail" (SMITH, 2003; YIN, 1989) of decisions and interpretations made during the course of the synthesis. Here another advantage of using interpretations as raw data emerges, as all the raw data (i.e. the original interpretations included in the published studies that are part of the synthesis) are readily available and open for inspection (DOYLE, 2003). As such, the potential for public scrutiny of meta-interpretation is far greater than that of the vast majority of qualitative research, and than that of many quantitative studies. This alone goes a very long way towards ensuring the integrity and trustworthiness of the meta-interpretation approach. [38]

Interpretive phenomenological analysis (IPA) is an approach developed by SMITH (1996) which focuses on interpreting the life experiences of interviewees and representing a view of the world from interviewees' perspectives. As the approach focuses on the detailed experience and understanding of research participants, it implies a commitment to fully analyse individual cases before attempting to analyse a group of interviews as a whole. It is this movement from the individual case to groups of cases that represents the synthesis element of this approach. SMITH (1996, p.264) outlines IPA as follows:

"The aim of IPA is to explore the participant's view of the world and to adopt, as far as is possible, an 'insiders perspective' (Conrad, 1987) of the phenomenon under study. At the same time, IPA also recognises that the research exercise is a dynamic process. While one attempts to get close to the participant's personal world, one cannot do this directly or completely. Access is both dependant on, and complicated by, the researcher's own conceptions which are required in order to make sense of that other personal world through a process of interpretative activity." [39]

IPA recognises that the researcher needs to be located in the research dialogue in order to get close to an insiders perspective but also that such a perspective can never fully be achieved as the researcher cannot fully or completely understand the world of the interviewee. Consequently there is a "double hermeneutic" (SMITH, JARMAN & OSBORN, 1997) of interpretive activity. Firstly, the research participant is interpreting his or her own life experience and discussing that with the researcher and, secondly, the researcher is interpreting the experience of the participant as told to him or her. The interpretations of the researcher, as noted in the previous section, are important as they carry the context of the interaction with the research participant with them. Consequently, the preferred quality control procedure, both for IPA and for meta-interpretation, is that of an audit trail rather than any form of external "member checking" (SMITH, 2003) as those checking the interpretations cannot have a full understanding or appreciation of the context in which the research interactions take place. As a result, the audit trail is not left to enable others to assess the interpretative decisions made, but to make transparent the procedure and to demonstrate the reasonableness of the analysis. [40]

The double hermeneutic of IPA, and indeed of most qualitative research, becomes a triple hermeneutic when studies are included in meta-interpretation. Here, the meta-interpretations of the synthesiser are added to those of the original researcher and the research participant. This interpretation of interpretations of interpretations is no different to the mean of means of means employed in meta-analysis discussed earlier. The triple averaging of meta-analysis has no less potential to lose the individual nuances of each individual data point, than the triple hermeneutic that takes place in meta-interpretation. It is a basic hazard of the synthesis process that the move from the specific to the generic will result in the loss of some individual differentiations. However, the method of synthesis should be constructed to allow for as much of the detail of the individual cases to be carried as far through the synthesis process as is possible. [41]

The discussions of the eight methods of synthesis in the preceding pages have attempted to draw out key features of a potential approach to the interpretive synthesis of qualitative data. Given the focus on interpreting interpretations, this approach has been termed meta-interpretation. The next section of the paper briefly summarises the main features of such an approach from the above discussions before detailing a potential procedure for meta-interpretation. [42]

In discussing a range of methods of synthesis, the previous section has identified a number of issues relating to the development of a meta-interpretation approach to the interpretive synthesis of qualitative research. From these discussions, it is possible to identify five fundamental features of a potential meta-interpretation procedure:

These features each form part of the potential procedure for meta-interpretation illustrated in Figure 1.

Figure 1: A potential procedure for meta interpretation. Click here to increase the picture. [44]

Perhaps the most useful way to explain the procedure for meta-interpretation illustrated in Figure 1 is to describe step-by-step a potential meta-interpretation procedure. The first step in conducting a meta-interpretation is to identify the area in which the synthesis is to take place. Given the inductive iterative nature of the procedure, this initial choice will, like the commencement of a grounded theory investigation, involve establishing a research area rather than a research question (STRAUSS & CORBIN, 1990). Once the research area has been established, an initial selection of around four or five contrasting studies, selected on the basis of "maximum variation sampling" (LINCOLN & GUBA, 1985, p.233), is established. This selection is informed by an awareness of, or theoretical sensitivity to, the research area (GLASER, 1978). The procedure then moves on to the initial analysis which, given the focus on "meaning in context" (MISHLER, 1979) involves a concurrent thematic and context analysis of the studies in question (ALTHEIDE, 1996). During this analysis it is possible that issues may emerge that warrant the exclusion of one or more of the studies, and that a range of issues for further investigation may emerge. [45]

Following the initial analysis, the first consideration is the need for the exclusion of any of the studies. As this is the first iteration of the analysis, there will be no exclusion criteria from previous iterations, so any exclusions will be on new bases for exclusion. Studies may be excluded for a range of reasons, but two potential examples may be that they are beyond the boundaries of the current synthesis (PAWSON, 2001) or that the quality of the research is flawed. In either case, such studies are excluded, specific reasons for exclusion are noted in detail, and the generic exclusion criteria is developed accordingly. [46]

Once exclusions have been dealt with, the range of conceptual issues arising from the initial analysis can be identified and, if theoretical saturation (PIDGEON & HENWOOD, 1996) has not been reached, which is unlikely on the first iteration, the literature is searched further for studies that address the issues that emerged from the analysis. Once such studies are identified, the dataset is developed further to include both the studies analysed previously and those added from the most recent search. At this point the second iteration of the meta-interpretation is begun as the studies undergo a further thematic and context analysis. [47]

As with the first iteration, following the analysis, the need for any exclusions is considered. Firstly, the exclusion criteria from the previous analysis is considered. If a study fails such exclusion criteria it is duly excluded from the analysis. However, following this exclusion there is a consideration of whether the exclusion criteria from the previous iteration remain relevant given the analysis in the current iteration. If it appears that they may not, then all previously excluded studies are reviewed for potential re-inclusion, and the dataset is re-developed and a further iteration is commenced with a new analysis. If previous exclusion criteria remain relevant, then any new bases for exclusion are considered and the reasons for studies' exclusion during this iteration are noted and the generic exclusion criteria developed further if necessary. [48]

Following the consideration of exclusions, once again the range of conceptual issues that emerged are considered and the need to further theoretically sample (STRAUSS & CORBIN, 1990) is assessed. The meta-interpretation continues through as many iterations as are necessary until theoretical saturation (PIDGEON & HENWOOD, 1996) is reached and no new insights are emerging from the analysis. At this point, the final findings of the meta-interpretation, and any emergent theoretical insights, can be developed and written up. However, alongside the meta-interpretation findings, a statement of applicability is written, which clearly identifies the boundaries of applicability (PAWSON, 2001) of the findings. Such boundaries are identified by a thorough review of the exclusion criteria and the studies excluded throughout the meta-interpretation. The statement of applicability, like the transparent audit trail (SMITH, 2003, YIN, 1990), plays an important role in ensuring the quality and integrity of the meta-interpretation. [49]

As the introduction to this paper noted, there has been a growing interest in research synthesis as a primary research activity. However, while there are a number of established synthesis methods, and still further research approaches that involve some form of synthesis, to date there has not been an available approach for the interpretive synthesis of qualitative research that allows the synthesis of more than around five studies. The discussions in this paper have attempted, through drawing out the nature of research synthesis from a review of other synthesis approaches and considering the possibilities for interpretive synthesis, to establish a method of meta-interpretation for the interpretive synthesis of qualitative studies. Whilst the potential procedure outlined in the previous section has not been fully road-tested, a meta-interpretation is currently underway relating to the motivations and behaviours of a range of sports spectators, building on previous work by WEED (2003). Indications to date are that the procedure is offering some potentially useful insights into this research area. However, it is hoped that the flow chart (Figure 1) and explanations in the previous section provide enough detail to allow other researchers to try out the meta-interpretation procedure if they are minded to do so. [50]

A final word is perhaps appropriate on the function of synthesis procedures in general, and meta-interpretation in particular. The introduction identified the growth of an "evidence-based" (TRANFIELD & DENYER, 2002) movement in both policy and medicine that strives to base clinical and policy decisions on the "best evidence" (COOK et al, 1997). Consequently, syntheses conducted in these areas are often carried out with the aim of recommending or establishing the most appropriate policy or clinical procedure. However, the aim of meta-interpretation is perhaps more generic, being simply to contribute to the improvement of the body of knowledge in a particular area. This may mean that the "what works in which situations and for whom?" questions (SOLESBURY, 2002) that may drive synthesis in evidenced-based policy and medicine are often less relevant for meta-interpretation than developing a broader understanding of the processes and dynamics of human behaviour and experience in a particular research area. [51]

However, notwithstanding the above discussion, there are two key points that should be made in closing about the functions of synthesis. The first relates to a call for further research. PAWSON (2002, p.39) believes that synthesisers should: "... resist the more-research-is-needed call, since it is the duty of the reviewer to make the best possible use of the material at his or her disposal". In fact, it could be argued that any synthesis with a main conclusion that more research is needed has been pointless. The underlying assumption of synthesis is that such procedures can add to the body of knowledge. This leads on to the second key point about the function of synthesis: that synthesis should be synergistic. This is highlighted by STRIKE and POSNER (1983, p.346):

"Synthesis is usually held to be activity or the product of activity where some set of parts is combined or integrated into a whole ... [However, synthesis] ... involves some degree of conceptual innovation, or employment of concepts not found in the characterisation of the parts". [52]

The value of a synthesis can perhaps be determined by the extent to which it is synergistic, the extent to which it produces insights that are more than the sum of the parts. It is hoped that the meta-interpretation approached developed here will provide a useful way of providing such synergistic insights in the interpretation of qualitative research. [53]

Biddle, Stuart J.H.; Markland, David; Gilbourne, David; Chatzisarantis, Nikos L.D. & Sparkes, Andrew C. (2001). Research methods in sport and exercise psychology: quantitative and qualitative issues. Journal of Sports Sciences, 19, 777-809.

Boaz, Annette; Ashby, Deborah & Young, Ken (2002). Systematic reviews: what have they got to offer evidence based policy and practice. ESRC UK Centre for evidence based policy and practice: working paper 2.

Bucher, Heiner C.; Hengstler, Peter; Schindler, Christian & Guyatt, Gordon H. (2000). Percutaneous transluminal coronary angioplasty versus medical treatment for non-acute coronary heart disease: meta-analysis of randomised control trials. British Medical Journal, 321, 73-77.

Charmaz, Kathy (2000). Grounded theory: objectivist and constructivist methods. In Norman K. Denzin & Yvonne S. Lincoln (Eds.), Handbook of qualitative research (2nd edition) (pp.509-535). London: Sage.

Conrad, Peter (1987). The experience of illness: recent and new directions. Research in the Sociology of Health Care, 6, 1-31.

Cook, Deborah J.; Mulrow, Cynthia D. & Haynes, R. Brian (1997). Systematic reviews: synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5), 376-380.

Dabinett, Gordon; Lawless, Peter; Rhodes, John & Tyler, Peter (2001). A review of the evidence base for regeneration policy and practice. London: Department of the Environment, Transport and the Regions.

Davies, Huw T.O.; Nutley, Sandra M. & Smith, Peter C. (1999a). Editorial: What works? The role of evidence in public sector policy and practice. Public Money and Management, 19(1), 3-4.

Davies, Huw T.O.; Nutley, Sandra M. & Smith, Peter C. (1999b). The rise and rise of evidence in health care. Public Money and Management, 18(1), 9-16.

Doyle, Lynne H. (2003). Synthesis through meta-ethnography: paradoxes, enhancements and possibilities. Qualitative Research, 3(3), 321-344.

Evans, David & Chang, Anne (2000). Changing practice: evidence based practice information sheets for health professionals. Adelaide: The Joanna Briggs Institute for Evidence Based Nursing and Midwifery.

Evans, David & Pearson, Alan (2001). Systematic reviews: gatekeepers of nursing knowledge. Journal of Clinical Nursing, 10(5) 593-599.

Glaser, Barney (1978). Theoretical sensitivity. Mill Valley, CA: Sociology Press.

Glaser, Barney & Strauss, Anselm (1967). The discovery of grounded theory. Chicago, Illinois: Aldine.

Glass, Gene V. (1976). Primary, secondary and meta-analysis of research. Educational Research, 5, 3-8.

Glass, Gene V.; McGraw, Barry & Smith, Mary L. (1981). Meta-analysis in social research. Newbury Park, CA: Sage.

Heaton, Janet (1998). Secondary analysis of qualitative data. Social Research Update, 22, 1-6.

Hunter, John E., Schmidt, Frank L. & Jackson, Gregg B. (1982). Meta-analysis: cumulating research findings across studies. Beverley Hills, CA: Sage.

Jones, Kip (2004). Mission drift in qualitative research, or moving toward a systematic review of qualitative studies, moving back to a more systematic narrative review. The Qualitative Report 9(1), 95-112. Retrieved 11/10/2004, from http://www.nova.edu/ssss/QR/QR9-1/jones.pdf.

Klassen, Terry P.; Jahad, Alejandro R. & Moher, David (1998). Guides for reading and interpreting systematic reviews. Archives of Pediatric & Adolescent Medicine, 152(7), 700-704.

Laycock, Gloria (2000). From central research to local practice: identifying and addressing repeat victimization. Public Money and Management, 20(4), 17-22.

Lincoln, Yvonne S. & Guba, Egon G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

MacDonald, Geraldine (1999). Evidence-based social care: wheels off the runway? Public Money and Management, 19(1), 25-32.

Maclennan, Duncan & More, Alison (1999). Evidence, what evidence? The foundations for housing policy. Public Money and Management, 19(1), 17-24.

Manning, Peter K. (1982). Analytic induction. In Robert B. Smith & Peter K. Manning (Eds.), Qualitative methods: Volume II of handbook of social science methods. Cambridge, MA: Ballinger.

Miles, Matthew B. & Huberman, A.Michael (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

Mishler, Eliot (1979). Meaning in context: Is there any other kind? Harvard Educational Review, 119(1), 1-19.

Niemi, Richard G. (1986). Series editor's introduction. In Fredric M. Wolf (Ed.), Meta-analysis—quantitative methods for research synthesis (pp.4-5). London: Sage.

Noblit, George W. & Hare, R. Dwight (1988). Meta-ethnography:synthesising qualitative studies. London: Sage.

Oxman, Andrew D. (1994). Systematic reviews—checklists for review articles. British Medical Journal, 309, 648-651.

Pawson, Ray (2001). Evidence-based policy: the promise of realist synthesis. Evaluation, 8(3), 340-358.

Pawson, Ray (2002). Does Megan's Law work? A theory-driven systematic review. ESRC UK Centre for evidence based policy and practice: working paper 8.

Pidgeon, Nick & Henwood, Karen (1996). Grounded theory: practical implementation. In John T.E. Richardson (Eds.), Handbook of qualitative research methods for psychology and the social sciences (pp.86-101). Leicester: BPS Books.

Ragin, Charles C. (1989). The logic of the comparative method and the algebra of logic. Journal of Quantitative Anthropology, 1, 373-398.

Rantala, Kati & Wellstrom, Eeva (2001). Qualitative comparative analysis and a hermeneutic approach to interview data. International Journal of Social Research Methodology, 4(2), 87-100.

Roberts, Karen A.; Jones, David R.; Abrams, Keith R.; Dixon-Woods, Mary & Fitzpatrick, Ray (1998). Meta-analysis of qualitative and quantitative evidence: an example based on studies of patient satisfaction. ASC Conference: thinking bigger, thinking smaller, learning about qualitative research, London, UK, April.

Smith, Jonathan A. (1996). Beyond the divide between cognition and discourse: using interpretive phenomenological analysis in health psychology. Psychology and Health, 11, 261-271.

Smith, Jonathan A.; Jarman, Maria & Osborn, Mike (1997). Doing interpretive phenomenological analysis. In Michael Murray & Kerry Chamberlain (Eds.), Qualitative health psychology: theories and methods (pp.218-240). London: Sage.

Smith, Jonathan A. (2003). Validity and qualitative psychology. In Jonathan A. Smith (Ed.), Qualitative psychology: a practical guide to research methods (pp.232-235). London: Sage.

Solesbury, William (2002). The ascendancy of evidence. Planning Theory and Practice, 3(1), 90-96.

Sparkes, Andrew C. (2002). Telling tales in sport and physical activity: a qualitative journey. Champaign, Illinois: Human Kinetics.

Stake, Robert E. (2000). Case studies. In Norman K. Denzin & Yvonne S. Lincoln (Eds.), Handbook of qualitative research (2nd edition) (pp.435-454). London: Sage.

Strauss, Anselm L. & Corbin, Juliet (1990). Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage.

Strike, Kenneth & Posner, George (1983). Types of synthesis and their criteria. In Spencer Ward & Linda J. Reed (Eds.), Knowledge structure and use (pp.343-362). Philadelphia: Temple University Press.

Tranfield, David & Denyer, David (2002). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Report to the ESRC (grant no. IMRC19). Cranfield Business School, UK.

Van Mannen, John; Manning, Paul K. & Miller, Marc L. (1988). Series editors' introduction. In George W. Noblit & R. Dwight Hare, Meta-ethnography: synthesising qualitative studies (pp.5-6). London: Sage.

Weed, Mike (2002). Sports tourism and identity: developing a sports tourism participation model. 10th Congress of the European Association of Sports Management, Jyvaskyla, Finland, September.

Weed, Mike (2003). Emotion, identity and sports spectator cultures: a comparative approach. 11th European Congress of Sports Psychology (FEPSAC), Copenhagen, Denmark, July.

Weed, Mike (2004). Epistemology versus sample size: towards a method for the synthesis of qualitative research in the sport sciences. 1st International Conference on Qualitative Research in Sport and Exercise, Liverpool, UK, May.

Wolf, Fredric M. (1986). Meta-analysis—quantitative methods for research synthesis. London: Sage.

Wood, Peter (2000). Meta-analysis. In Glynis M. Breakwell, Sean Hammond & Chris Fife-Schaw (Eds.), Research methods in psychology (2nd edition) (pp.414-425). London: Sage.

Woodside, Arch G. & Sakai, Marcia Y. (2001). Meta-evaluations of performance audits of government tourism marketing programs. Journal of Travel Research, 39, 369-379.

Wright Mills, Charles (1959). The sociological imagination. New York: Oxford University Press.

Yin, Robert K. (1989). Case study research, design and methods. California: Sage.

Dr Mike WEED is with the Institute of Sport and Leisure Policy in the School of Sport and Exercise Sciences at Loughborough University. He is interested in the relationship between sport and tourism, and in particular in the behaviour patterns and motivations of sports tourists and sports spectators. He is also interested in Youth Football Development and is directing a range of consultancy projects in this area funded by agencies such as the Football Association, 4 Children and Coca-Cola Enterprises. Dr WEED is interested in meso-level and innovative qualitative approaches to research and he convenes a Qualitative Research Group within the School of Sport and Exercise Sciences. He sits on the Editorial Board of The Qualitative Report and is co-ordinator of "The Research Synthesis Project", an international collaboration of researchers working on methods of research synthesis in sport and exercise.

Institute of Sport & Leisure Policy,
School of Sport and Exercise Sciences
Loughborough University
Loughborough, Leicestershire, LE11 3TU, UK

Weed, Mike (2005). "Meta Interpretation": A Method for the Interpretive Synthesis of Qualitative Research [53 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(1), Art. 37, http://nbn-resolving.de/urn:nbn:de:0114-fqs0501375.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License