Abstract: When discussing mixed methods research from a methodological point of view, it must be acknowledged that they have been developed from rather simple research designs to complex and multilayered frameworks in which qualitative content analysis (QCA) is seemingly a key method for analyzing non-numeric data. This can be attributed to the versatile procedures and functions for which QCA can be used.

Against this background, we describe and analyze the roles of QCA within different advanced mixed methods designs. First, we outline the implementation of QCA within a typical methodological framework, which consists of three hierarchical layers. Second, mixed methods research designs and the role of QCA in several design types are characterized. Third, three design types stemming from studies from educational sciences and sports science are presented. With these three elements, we demonstrate that QCA can be used in different roles ranging from dominance to subordination, thus implementing different rationales. Finally, we formulate some conclusions and suggestions for further research.

Key words: qualitative content analysis; mixed methods research; advanced mixed methods designs; quality criteria; rationales

4.1 The subordinate role: QCA in a quantitatively driven design—a study from RT_PE

4.2 The dominant role: QCA in a qualitatively driven design—a study from sport sociology

Since the early 1990s, mixed methods research (MMR) has become a prosperous branch in the social and behavioral sciences. This is true for both philosophical and methodological approaches, as well as for the development of research designs (HESSE-BIBER & JOHNSON, 2015; JOHNSON, 2017; TEDDLIE & TASHAKKORI, 2009). Focusing on the design perspective, MMR initially consisted of rather basic concepts in which single strands of qualitative and quantitative research approaches, respectively, were usually combined (CRESWELL, 2015a, 2015b; CRESWELL & PLANO CLARK, 2011; TASHAKKORI & TEDDLIE, 2003a). Today, a multitude of advanced and complex conceptions, combined with various techniques of data collection and analysis can be found (JOHNSON, 2018; ONWUEGBUZIE & HITCHCOCK, 2015; SCHOONENBOOM & JOHNSON, 2017); however, all provide tools for promoting the overall goal of avoiding the blind spots of a mono-method study, as well as expanding and strengthening the conclusions of a study. [1]

In terms of developing design structures, various questions have been intensively discussed within the MMR community (CRESWELL, 2015b; MORSE, 2015; SCHOONENBOOM & JOHNSON, 2017). As a consequence, the MMR community has compiled comprehensive lists of both quality issues for mixed methods designs (BRYMAN, 2006; JOHNSON & CHRISTENSEN, 2014; ONWUEGBUZIE & JOHNSON, 2006), as well as primary and secondary design dimensions (JOHNSON & KÖNIG, 2016; SCHOONENBOOM & JOHNSON, 2017). While specific quality criteria have been proven to be urgently required—as those from quantitative or qualitative research do not work for MMR—design dimensions are necessary in order to provide specific information on how to construct a mixed methods design. Hence, researchers discuss issues such as the theoretical drive of a study (MORSE & NIEHAUS, 2009), rationales for mixing (BRYMAN, 2006), as well as time structure and weighting of strands (CRESWELL & PLANO CLARK, 2011). [2]

Another feature of this debate is that more and more methods have been brought into these designs (ONWUEGBUZIE & HITCHCOCK, 2015), offering a great variety of analyses and the respective integration of them. However, it is remarkable that researchers have consistently and regularly applied coding or category building as a key technique for analyzing qualitative data within a variety of designs targeting diverse research aims. In the English-speaking world, coding is often associated with research styles such as grounded theory methodology (CHARMAZ, 2014; GLASER & STRAUSS, 1967; STRÜBING, 2014), interpretative analysis (FINK & GANTZ, 1996), or ethnographic content analysis (ALTHEIDE & SCHNEIDER, 2013), but the term "qualitative content analysis" (translation from the German "qualitative Inhaltsanalyse"; KRACAUER, 1952)¹⁾ is seldom found. As the other already mentioned research approaches are relatively well-known and are also widely applied within qualitative research in Germany (in which codings and categories are handled slightly differently), we suppose that the idea behind QCA is inherent in the English-speaking scientific community without explicitly being labeled as such. Focusing on the German conceptions, we are aware of the fact that there is not one single and holistic QCA, but a substantial amount of different positions and variations that have to be distinguished (SCHREIER, 2014; STAMANN, JANSSEN & SCHREIER, 2016). Against this background and after having analyzed papers in different national and international journals, we assume that QCA can fulfill different roles within MMR. [3]

On the basis of these preliminary thoughts, the role(s) of QCA is (are) approached and analyzed in MMR in four steps in this article: Initially, we discuss how QCA can be implemented into a framework conception of MMR which goes beyond the simple definition of combining elements of qualitative and quantitative research (Section 2). We then analyze MMR design structures in detail to identify different roles of QCA within them, when it is used within the research process and how the data obtained in this way is utilized (Section 3). Next, we show the application of QCA in different mixed methods research designs by following a more recent development of MMR Designs—the concept of crossover (HITCHCOCK & ONWUEGBUZIE, 2019; ONWUEGBUZIE & HITCHCOCK, 2015)—and thereby demonstrating that QCA can be characterized by multi-dimensional ("fluid") roles within an equal-status design (Section 4). Finally, and after a short summary, we address how the use of QCA benefits complex studies, and discuss the advantages and disadvantages of using it for different approaches (Section 5). [4]

When dealing with MMR and thinking about the meaning of the very term, we immediately find that a mixed methods design is characterized by the combination of at least one qualitative and one quantitative research component, or, as JOHNSON, ONWUEGBUZIE and TURNER (2007, p.123) put it: "Mixed methods research is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches [...] for the broad purposes of breadth and depth of understanding and corroboration." Similar descriptions can be found by CRESWELL (2009, 2015b), KUCKARTZ (2014), as well as JOHNSON and CHRISTENSEN (2014). Going one step beyond this simple characterization TEDDLIE and TASHAKKORI (2009) defined MMR as mono- or multi-strand designs and dissociated it from mono-method and multimethod studies in their well-known methods-strands matrix. However, we argue that researchers applying any of these approaches do not fully consider recent discussions. MMR is no longer an issue of methods only, but has been developed into a systematic framework. [5]

Focusing on the MMR framework introduced by GREENE (2015) and GREENE, CARACELLI and GRAHAM (1989), we maintain that MMR should first and foremost be differentiated into and examined on the levels of paradigm, methodology and method. The traditional juxtaposition of qualitative and quantitative elements is subsequently discussed in this text. GREENE's interactive combinations or mixing at three levels can be explained as follows: [6]

"Paradigm" refers to different philosophical traditions or worldviews and epistemological logics to social inquiry, in which MMR was conceived largely as one "solution" to the relatively intense "paradigm wars" of the 1970s and 1980s (GREENE, 2015, p.607). Consequently, this level of MMR has to be regarded as the most abstract one. There has been much debate about the role of paradigms in MMR; however, in the face of past calls for each researcher to operate within a single paradigm, it turns out that some researchers or practitioners find many positive features in more than one paradigm (JOHNSON, 2017). Perhaps the most popular paradigm in MMR is pragmatism, despite the presence of additional worldviews in MMR including transformativism (MERTENS, 2009), realism (MAXWELL & MITTAPALLI, 2010), the dialectical stance (GREENE & HALL, 2010), and dialectical pluralism (JOHNSON, 2017). In particular, dialectical pluralism is sometimes framed as a meta-paradigm because it is used to provide a philosophical process theory for creating new mixed paradigmatic syntheses on a study-by-study basis. The key point of dialectical pluralism is that paradigms can and often should be mixed by putting them into juxtaposition and dialogue, thus creating new dynamic syntheses (JOHNSON, 2018). [7]

"Methodology" refers to both research methods and research conceptions in quantitative research, e.g., experiments or non-experimental research, and theoretical frameworks in qualitative research, like case studies or grounded theory methodology. Methodologies include a system of inquiry which means they are built on either a qualitative (inductive) or a quantitative (deductive) logic. In recent years, several mixed methodologies have been developed, e.g., mixed methods-grounded theory (GUETTERMAN, BABCHUK, HOWELL SMITH & STEVENS, 2017), mixed methods-phenomenology (MAYOH & ONWUEGBUZIE, 2015) and mixed methods-experiments (DRABBLE & O'CATHAIN, 2015). [8]

"Method" refers to a more technical level, within which different ways of collecting, processing, analyzing, and presenting data are integrated. Examples of methods are tests, interviews, questionnaires, etc. Each of these major methods can be conceptualized as having a qualitative, a quantitative, and a mixed version. Mixing can be done within a single method (e.g., a mixed questionnaire or mixed observation) to produce qualitative and quantitative data, which is called intra-method mixing (JOHNSON & TURNER, 2003). More commonly, two or more different methods are mixed with each other (e.g., a qualitative focus group and a quantitative questionnaire) to produce qualitative and quantitative data; this is called inter-method mixing (ibid.). The use of both of these approaches can produce true MMR. [9]

In summary, we state that MMR is not only about methods, but is related to three levels (GREENE, 2015) which differ in their abstractedness. Thus, in investigating the role of QCA in MMR, one has to take into account the combination of methods at multiple levels of reality and analysis, as well as across multiple disciplines. [10]

QCA is first and foremost called a method (KUCKARTZ, 2014), which is applied in research practice by different techniques. These techniques consist of diverging strategies of category building and with them researchers aim at generating different sorts of categories. However, in a first stage of analyzing different techniques of QCA, we may generally state that there are two basic strategies of category building, namely a deductive and an inductive approach, as well as a combination of both, the deductive-inductive procedure (KRIPPENDORFF, 2013; MAYRING, 2015; NEUENDORF, 2017; SCHREIER, 2012). Going one step further, the deductive approach can be characterized by the fact that in using it, researchers operationalize theoretical concepts as categories, whereas an inductive approach ranges from the successive summary to—based on grounded theory methodology—open coding (STAMANN et al., 2016). At this point we are aware of the fact that these strategies have to be regarded as two opposites on a continuum which encompasses intermediate forms, e.g., the previously mentioned deductive-inductive combination. As different methods of constructing categories are of different significance for research practice (MAYRING & GLÄSER-ZIKUDA, 2008), we would like to follow the idea of a continuum because with it we have the chance to construct a further module for the MMR framework on the level of methods. [11]

With reference to GREENE's (2015) framework, QCA may not only be regarded as a method (of data analysis), but also as a methodology, because it is not only used in category building and even coding procedures (STAMANN et al., 2016), but is also applied throughout the entire research process ranging from deriving the research question and data analysis to the presentation of results (MAYRING, 2019; GLÄSER & LAUDEL, 2009). Extending this methodological argument, QCA can be applied in a variety of research designs, from rather inductive approaches to others which come close to deductive procedures that are somewhere in-between these two opposites. [12]

In conclusion, we assume that variations of QCA can be used for various MMR methodologies, fulfilling different functions. Based on this presumption we introduce the term "research design," which is defined as a bridge between methodology and method. This concept can be used to answer the question of how methods are deployed, implemented, and combined (HUNTER & BREWER, 2015). [13]

With reference to the issue at focus, a MMR study is basically conducted with an overall theoretical drive (MORSE & NIEHAUS, 2009) because with it the researcher is focused primarily on either "exploration-and-description or on testing-and-prediction" (SCHOONENBOOM & JOHNSON, 2017, p.112). In the first case, the theoretical drive is called "inductive" or "qualitative"; in the second case, it is called "deductive" or "quantitative." In addition to MORSE and NIEHAUS (2009), a study can include more than one or even multiple research questions, thus it might be comprised of several theoretical drives (SCHOONENBOOM, 2016), which allows an equal status MMR (GREENE, 2015; SCHOONENBOOM & JOHNSON, 2017). [14]

Meanwhile, SCHOONENBOOM and JOHNSON's approach (2017) has been broadly accepted because it has been argued to have several advantages ranging from philosophical positions to the question of method. With this conception we can differentiate between three main types of MMR, each of them having a unique epistemological position and different time structures, as well as different weightings (JOHNSON et al., 2007; MORSE & NIEHAUS, 2009). [15]

Quantitatively-driven or quantitative-dominated MMR (QUAN/qual) is a type of mixed research in which one relies on a quantitative, post-positivist view of the research process, while concurrently recognizing that the addition of qualitative data and approaches is likely to be beneficial, since one can broaden one's horizon of understanding. The qualitative part may precede the quantitative part, thus having the function of developing an instrument or a theoretical framework. It may also follow the quantitative part of the study, which allows researchers to explain or engross statistical results. Finally, both can be embedded concurrently, e.g., to improve intervention designs (SCHOONENBOOM & JOHNSON, 2017). In view of the issue at focus, we therefore assume that QCA is put into a subordinate role supporting the respective quantitative strand(s). This can be realized at different points of time and with different purposes. However, QCA as a method has to be regarded as vital for the particular study itself (CRESWELL, 2015b). [16]

Qualitatively-driven or qualitative-dominated MMR (QUAL/quan) is a type of mixed research in which one relies on a qualitative, constructivist, poststructuralist-critical view of the research process, while concurrently recognizing that the addition of quantitative data and approaches are likely to be beneficial for most research projects. In this approach the qualitative strand may also come before a statistical analysis, thus allowing the researcher to generate a kind of theoretical framework which will be structured or validated by applying the quantitative strand. Alternatively, the qualitative strand may begin after an explorative statistical analysis which is the basis for an interrogation or observation. It may also be implemented in an inductive-simultaneous design, where the core component is qualitative and the supplemental component is quantitative. In this approach QCA has a superior or dominant role, regardless of its point in time, as it may precede or follow a subordinate quantitative strand. Thus, we can conclude that in the quantitative part of the study the researcher must accomplish the task of preparing, supporting, or enhancing the inquiry with QCA. [17]

Equal status MMR (QUAL/QUAN) is the home for the person that self-identifies primarily as a MMR researcher. They are likely to believe that qualitative and quantitative data and approaches will add insights to most, if not all, research questions. Equal status MMR can be realized within a concurrent design (QUAL+QUAN), as well as in a sequential design which may either be organized as QUAL -> QUAN or as QUAN -> QUAL (MORSE, 1991). Within this set of designs, QCA is regarded as an equitable element to the respective quantitative method, no matter in which chronological position it is implemented (TASHAKKORI & TEDDLIE, 2003b). As a consequence, we can complete our set of terms and add the term "equivalent" to the hitherto discussed roles of QCA. [18]

Regarding interlinking mixed methods design structures with QCA as a methodology, we can summarize that, firstly, QCA can be used in three different roles within three macro design structures, and is implemented at various points during the research process. Secondly, and continuing this line of thought, we can recognize a similar structure between MMD structures and QCA as a method and the respective strategy of category building. QCA as a method, then, consists of either a deductive approach, which may more likely be moved towards quantitative dominant research designs, or an inductive approach with a rather high degree of overlap with qualitative-dominant designs, or something like a mixed position and the deductive-inductive approach. Additionally, this coincides with different paradigmatic positions described by GREENE (2015). These arguments are summarized and displayed in Figure 1.

However, so far we have not discussed a further dimension for designing MMR, namely, the rationales for using QCA within MMR. This is, however, vital to every MMR study as rationales provided by researchers specify the overall purpose in order to expand and strengthen the conclusions of a study. Therefore, rationales must be regarded as contributions to answering research questions in a more detailed and enhanced matter (JOHNSON & CHRISTENSEN, 2014; ONWUEGBUZIE & JOHNSON, 2006). [20]

A popular classification of purposes or rationales of MMR was first introduced by GREENE et al. (1989), based on an analysis of published mixed methods studies. Although further classifications have been made (e.g., BRYMAN, 2006), this approach is still relevant because later approaches regularly draw on it. Referring to SCHOONENBOOM and JOHNSON (2017, p.110), the following five purposes for mixing methods are distinguished:

"1. Triangulation seeks convergence, corroboration, correspondence of results from different methods;

2. Complementarity seeks elaboration, enhancement, illustration, clarification of the results from one method with the results from the other method;

3. Development seeks to use of the results from one method to help develop or inform the other method, where development is broadly construed to include sampling and implementation, as well as measurement decisions;

4. Initiation seeks the discovery of paradox and contradiction, new perspectives of frameworks, the recasting of questions or results from one method with questions or results from the other method;

5. Expansion seeks to extend the breadth and range of inquiry by using different methods for different inquiry components." [21]

Transferring these purposes to the issue of the role of QCA within MMR, it can initially be argued that all these rationales can be approached by including QCA in a design, which opens up the following threads:

We conclude that QCA and the various approaches it offers (MAYRING & BRUNNER, 2010) are used in different roles along the basic paradigms in MMR: dominant, subordinate or equal status. Additionally, and considered through the lenses of different rationales, QCA can then be in the role of different services: support for further insights, for discovering contradictions or for development of further research steps. The following examples will show how QCA works in different roles within various MMR designs. [23]

Usually, MMDs are categorized into basic and advanced designs. Whereas basic designs are the core designs underlying all mixed methods studies (CRESWELL, 2015b), advanced MMDs are characterized by the fact that something is added to the traditional and basic structures (ibid.). Usually, this goes along with greater complexity in either concurrent or successive approaches (ONWUEGBUZIE & HITCHCOCK, 2015). A prominent example is the idea of crossover, in which one form of data is analyzed utilizing techniques usually associated with the alternative paradigm (GREENE, 2007). Presenting this approach, we first show how QCA is used in a subordinate role which is realized by explaining variance (a statistical coefficient) in a multilevel longitudinal analysis of fitness training (KÖNIG, 2019). Secondly, we use QCA to develop a coding frame for the topic "volunteer work in German sports clubs" which is enhanced by a quantitative systematization of the categories using correspondence analysis (KÖNIG, 2018); this in turn corresponds with a dominant role of QCA in crossover studies. In addition, we show in a third approach that the role of QCA need not be fixed from the beginning, but can change throughout a study (KANSTEINER, 2018). [24]

4.1 The subordinate role: QCA in a quantitatively driven design—a study from RT_PE

Researchers in physical education (PE) increasingly deal with the question of whether the acquisition of skills and exercise result in effects, and if these effects are stable (KÖNIG, 2014). Automatically, this results in research questions with which researchers aim at both individual and group-specific effects. In addition, a closer look at the data structure shows that we are usually confronted with "nested" data; which means, for example, that several measurements belong to one person, several persons belong to one class or group, and several classes to one school (HEADLEY & PLANO CLARK, 2019). Consequently, we have to accept that these different data levels must not be analyzed independently. [25]

From an empirical point of view, such dependencies have to be taken into account to avoid statistical bias. This can be done using multilevel regression models with which researchers are able to model context effects. However, even the use of these multivariate approaches is stretched when quantitative incompatibilities are to be explained (ONWUEGBUZIE & HITCHCOCK, 2015). In a recent study, we addressed strength exercises in PE (KÖNIG, 2016, 2019) and could show that Level 3 predictors (different exercise methods) had a reciprocal effect on Level 2 factors (gender) when explaining change over time. This is displayed in Figure 2.

Figure 2: Effect of Level 3 predictors on Level 2 factors (KÖNIG, 2019, p.53) [26]

In summary, a multilevel model can be used to detect differences of controlled effects of program variables; however, it cannot be used to explain the differences found in a three-level model, computed with HLM 7.1 (RAUDENBUSH, BRYK, CHEONG, CONGDON & TOIT, 2011)²⁾, even if various predictors are added to the model to improve it. [27]

As a consequence, additional qualitative data has to be integrated into the multilevel regression model to understand the reciprocal effects of the Level 2 and 3 predictors. This step was realized using semi-structured interviews with different participants of the treatment, which were conducted around measurement point 2. Data analysis was accomplished with MAYRING's QCA (2015) conception of deductive development of categories. This means that we followed a deductive logic, thus implementing a more quantitative-oriented approach (MARVASTI, 2019; MAYRING, 2000). This approach can be justified by the idea of applying theoretically derived categories from the science of training. In other words: we followed a deductive logic in the qualitative strand, thus acting in accordance with the principles of quantitative-dominated MMR. [28]

In this analysis we showed that three main categories could be identified, which we labeled as "relevant goals," "intensity of program," and "attractiveness." In Table 1 we give an overview of the qualitative results, including some exemplary quotes from selected text passages. [29]

With these categories we can explain why two different exercise methods—implicit and explicit training—caused reciprocal effects. Obviously, boys and girls perceived the two methods differently as to their intensity and attractiveness. These perceptions lead to different motivations which in turn caused different effects. However, the qualitative results ought to be critically examined as these explanations are only suitable for our sample and should only be generalized with caution (SCHOONENBOOM, 2016).

Summarizing this quantitative dominated crossover design and the role of QCA within it, we can say that the qualitative strand helped to explain quantitative coefficients in a multilevel model; therefore, QCA plays a subordinate role in this design because it "only" assumes a minor function in the whole analysis. However, the use of quantitative research alone would not have been sufficient without this qualitative support, thus the inclusion of QCA is not redundant. Generalizing this point of view, it can be said that QCA can be used to help enhance advanced and multivariate approaches by explaining statistical coefficients with categories and subcategories. In conclusion, the inclusion of qualitative elements in the analysis can be helpful in answering the research question. [31]

4.2 The dominant role: QCA in a qualitatively driven design—a study from sport sociology

In Germany, sports clubs have become widespread and sophisticated institutions within which sport and recreational activities for nearly all subgroups of society are offered. Since the very beginning this has mainly been accomplished on the basis of honorary work. However, during the last two decades more and more people have drawn back from social engagement for the traditional tasks in the clubs. Hence, it is of great relevance for sport associations to learn more about their members' changing motives and attitudes towards social commitment. As a consequence, a study was conducted in which the motives of active volunteers of all ages and different sports were analyzed to find out more about this phenomenon. [32]

This was a qualitative-dominated crossover-study (ONWUEGBUZIE & HITCHCOCK, 2015) that was implemented with a mixed methods and mono-strand conversion design (TEDDLIE & TASHAKKORI, 2009). The study design is displayed in Figure 3.

Focusing on the process of data analysis in detail, we initially implemented the technique described by KUCKARTZ (2014). This means that we summarized the individual cases in a first step and then analyzed categories in an iterative process on a case-by-case basis (KUCKARTZ, 2019). As our questions were rather open, categories were built inductively, especially with reference to one of our focal interests, the "decline or lack of young people in volunteer engagement." Throughout this circular process the following six main categories have been built and defined as follows:

All main categories were differentiated into several subcategories. To add value to the qualitative data, we converted them into numbers ("quantizing") to find statistic patterns (SANDELOWSKI, VOILS & KNAFL, 2009; VOGL, 2017). Defined as "the process of assigning numerical values (nominal or ordinal) to data conceived as not numerical" (SANDELOWSKI et al., 2009, p.209), the process of transforming coded qualitative data into quantitative data (DRISCOLL, APPIAH-YEBOAH, SALIB & RUPERT, 2007; VAN VELZEN, 2018) or transforming qualitative data into numerical format (COLLINGRIDGE, 2013), quantitizing is used to analyze narratives to explore the existence of possible hypotheses or examine the salience of some discourses of interest and how they differentially occur in social actors' narratives (NZABONIMPA, 2018). As quantitizing is by no means transparent, uncontentious, or apolitical (LOVE, PRITCHARD, MAGUIRE, McCARTHY & PADDOCK, 2005) it must be made completely clear which categories are coded with which numbers. [35]

In the study in question, we decided to implement the absent vs. present-dichotomy for quantitizing qualitative data (SANDELOWSKI et al., 2009). Despite some critical points (NZABONIMPA, 2018), two reasons speak in favor of this method: First, we assumed that broaching a subject attributes individual importance to an issue; second, this dichotomy can easily be transformed into quantitative data as "present" means that this category was definitely mentioned by an interviewee and "absent" that it was not. Finally, using this way of counting was to be adequate for the research topic "lack of adolescents in honorary work" because we could correlate groups and categories, thus creating visual patterns. A selected result is displayed in Figure 4.

Analyzing and discussing this exemplary analysis of homogeneity we could clearly detect two visual patterns which enhance our qualitative results due to systemization. With Pattern 1, the fact that the group "education" (referring to our participants being still at school, at university, or in an in-service training) was in close connection to the presence of categories like attitude, personal conditions, and investment was exposed. This means that for these participants the categories were of high importance, as they represented obstacles for volunteer work. Participants of the group "employment," in contrast, tended to omit such statements in the interviews. [37]

Summarizing this study, we can say that the qualitative strand was more important as with it we were able to generate a kind of small-scale theory. The quantitative results were only used to give a kind of structure for the inductively-built categories, thus fulfilling a minor, however, indispensable role. This can be associated with a dominant role of QCA within a qualitative-dominated crossover design. [38]

After pointing out the possible differences in taking on one or the other part in MMDs, we can show in another example of our work that the decision for using a specific rationale and for the weighting of different strands is sometimes difficult to elaborate distinctly because more than one aspect could be addressed. Likewise, within one study the contribution of using QCA could shift. [39]

In the study we refer to, the extent to which the current school leadership recruitment processes in Germany fulfilled the requirements of equality in the sense of non-discrimination against the background of gender mainstreaming and diversity management was investigated (KANSTEINER & KLOSE, 2017). No research had been done on this issue so far and, consequently, an exploratory approach was chosen. Since the recruitment process slightly differs from federal state to federal state, but nevertheless the same main principles (several steps in the assessment, a selection commission decides collectively on officially expected competencies) are followed, semi-structured interviews were conducted in two German states. Interviewees were school leaders, members of the selection commission, the school authority, and the staff committees, as well as representatives of the community and teachers. All of them (n=28) were asked about their personal experiences and insights in the practices. The idea was to create a data-driven questionnaire (CRESWELL, 2015a). Data collection was carried out as a survey across all 16 German federal states. The qualitative strand facilitated new perspectives, for example how important the informal communication between applicants and members of the school authority is in order to get detailed information about the specific leading position. At the same time, everyone in the process was conscious that not all applicants obtain the same pieces of information, but no one considered this unequal treatment as discrimination. Also, we found that participants in the school leadership recruitment processes referred to stereotypes when consulting how well an applicant matched with the context of the school. Regarding this, the categories "gender" and "age" were primarily taken into consideration by the selection commission (KANSTEINER, 2018). [40]

The questionnaire following our qualitative analysis consisted of items which reflected those categories set as relevant. With it we aimed to verify if there was approval for the viewpoints of the interviewees throughout Germany. Because of the possibility of specific dynamics in the process of social construction of difference (ONNEN & RODE-BREYMANN, 2017) which may occur in the recruitment processes because of slight differences in the procedural rules of the various states, we also included additional items. They referred to those categories of difference that are also usually part of the diversity discourse, like cultural and religious background, sexual orientation or disability (CZERNECKA, 2013). Additionally, we included questions that did not evolve from the analysis of the qualitative strand, but were deduced from lines of the discourse that mattered to us, e. g., how much school leaders work and whether job sharing is possible. [41]

Thus, we did not only verify to what extent experts all over Germany confirmed the situation as it was analyzed, but also asked for further aspects like their involvement in the process of recruitment and their overall experiences of discrimination, their opinion on how to assemble the leadership teams (men, women, old, young, ...) at schools according to the specific background of the school, and the expectations which they saw school leaders confronted with. Finally, data collection was run in an extended version and we then aimed at detecting further contradictions or constructions of differences by certain groups in comparison. The analysis thus became somewhat exploratory (HESSE-BIBER & JOHNSON, 2015). [42]

What at first glance seemed to follow the rationale of "development" and the weighting "qual -> QUAN" turned out to be equally a shared importance in the sense of triangulation in equal status "QUAL -> QUAN," and additionally served the rationale of initiation and sought to discover paradox and contradiction. The results of the questionnaire (n=203) were used to improve the monitoring. However, with this step the researchers were also able to trawl through the data to look for further results, which may go beyond the issue of approval, but also served as exploration. A result we won by this shifting of the quantitative strand is the partly contradictory opinion that women were considered as equally capable of leadership as men, but at the same time they were regarded as more involved in raising children and having less time for their leadership position. Further exploration led us to the result that female participants more often regarded differing treatment of men and women in informal situations as discriminatory than men did. As a final example, we found that there is a high expectation to see school leaders as "multifunctional miracle workers"³⁾ (HUBER 2009, p.506), whereas members of the school authority agreed surprisingly less with that description (initiation). [43]

Since we finally looked at the quantitative survey and the results from it as likewise independent from the initial strand, it came out as a deviation from the original design and the rationale was expanded towards irritation. Due to this research experience we did not only detect QCA in different roles against the background of the well-structured system of MMR (see above), but we also acknowledged some fluid characteristics within MMR designs. For us "fluid" should not be identified with the idea of "emergent" (CRESWELL & PLANO CLARK, 2011) or of a fully integrated design (TEDDLIE & TASHAKKORI, 2009); more likely, it is related to slight changes or extensions in the instruments, in the sampling, or in the modes of analysis that occur along the way. [44]

We argue that the role of QCA in MMR can be systemized at three levels (Figure 1): Approaching the topic from an overview of the discourse, QCA as a method appears to be one of the main approaches in the qualitative strands of many MMR studies. This seems to be particularly true for studies conducted by researchers from Germany, which increasingly leaves a mark on the international MMR discourse. In contrast, in the English-speaking contributions it appears less specific which qualitative method is applied, since researchers hardly ever use the term QCA. Readers have to identify by the way coding and categorizing are applied whether an approach similar to QCA or another qualitative approach was implemented. [45]

Regarding the systematic framework and the taxonomies that researchers lean on in designing MMR, the use of QCA fulfills several roles depending on its function within the research project—superior, subordinated, or equal in position to quantitative elements—and the utilization of the obtained results: enlarging and corroborating insights, irritating results, or executing preliminary work. As we also demonstrate with our crossover designs, QCA is moved to the edge of the qualitative part of the continuum without switching into the other methodology. Thus, QCA can be fruitfully used to combine qualitative data with quantitative data due to the subsumptive logic on which it is built. [46]

According to the practical experiences of researchers using MMR, QCA can also be regarded as sometimes being fluid within one research project, having more than one of the aforementioned roles because research questions and research necessities need to be somewhat rearranged. Although major functions are set at the beginning of the research properly, they might be changed because of modifications researchers find reasonable in the course of the project. [47]

In conclusion, QCA can be regarded as a heterogeneous and multilayered element of MMR depending on the principles the researcher follows in designing a project. This applies to methodological issues as well as to design questions. It can also be recognized that with increasing interest in advanced designs, the application of QCA can contribute to create more differentiated views on social situations. The more complex designs are used and the more ways of transferring data to and from qualitative sources are applied, the more the use of QCA makes sense because it provides the widest range of applications. [48]

Following the scientific discussions that took place at the latest conferences of MMIRA⁴⁾ in Durham (2016) and Vienna (2018), we noticed that increasingly more research questions address complex social situations. To answer them, researchers often apply QCA. We also noticed an increasing interest in combining quantitative approaches with other methods of qualitative analysis like grounded theory methodology or case studies. We assume though, that QCA will remain a relevant qualitative method in MMR in the years to come because the requirements for its practical application seem to be easier to be realized and the process of developing results seems to be more systematic and retraceable. The closeness of QCA to the quantitative paradigm (BURZAN, 2016) can be linked to the effective application of it to advanced designs in MMR that are currently of great interest. However, QCA alone should not be interpreted as a MMR approach, but referred to as a "hybrid instrument" (p.30). [49]

If we were to express the role of QCA in a metaphor, we imagine a sibling⁵⁾ which can be the guiding elder, who is leading the younger or a fraternal twin. Overall, she can be as strong as, weaker or stronger than her sibling, and be a comrade to play with, sharing the same interest or being challenging for gaining new ideas and further playgrounds. In one conjoined game, though, she might change her role during play, regardless of her original role among the siblings. [50]

1) It is an interesting fact that this term is not or only marginally used in famous English textbooks (CRESWELL, 2009; JOHNSON & CHRISTENSEN, 2014; KRATHWOHL, 2004; NEUENDORF, 2017). <back>

2) The abbreviation stands for hierarchical linear and nonlinear modeling. <back>

3) "Multifunctional miracle worker" is our translation of the expression "multifunktionales Wunderwesen," which Stephan HUBER used to describe the excessive expectations of competencies and achievement that meet a school leader and that they try to live up to. <back>

4) The abbreviation stands for the Mixed Methods International Research Association, https://mmira.wildapricot.org/ [Date of Access: October 30, 2019]. <back>

5) SCHOONENBOOM and JOHNSON (2017, p.109) used this metaphor to explain the relationship of mixed methods and multimethod research. <back>

Altheide, David & Schneider, Christopher (2013). Qualitative media analysis (2nd ed.). Thousand Oaks, CA: Sage.

Burzan, Nicole (2016). Methodenplurale Forschung: Chancen und Probleme von Mixed Methods. Weinheim: Juventa.

Charmaz, Kathy (2014). Constructing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.

Collingridge, Dave (2013). A primer on quantitized data analysis permutation testing. Journal of Mixed Methods Research, 7(1), 81-97.

Creswell, John (2009). Research design. Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.

Creswell, John (2015a). A concise introduction to mixed methods research. Thousand Oaks, CA: Sage.

Creswell, John (2015b). Revisiting mixed method and advancing scientific practices. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.57-71). New York, NY: Oxford University Press.

Creswell, John & Plano Clark, Vicki (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Czernecka, Julita (2013). The idea of diversity management. In Martina Endepohls-Ulpe, Claudia Quaiser-Pohl & Vera Ruthsatz (Eds.), Diversity and diversity management in education. A European perspective (pp.35-46). Münster: Waxmann.

Drabble, Sarah & O'Cathain, Alicia (2015). Moving from randomized controlled trials to mixed methods intervention evaluations. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp.406-425). New York, NY: Oxford University Press.

Driscoll, David L.; Appiah-Yeboah, Afua; Salib, Philip & Rupert, Douglas J. (2007). Merging qualitative and quantitative data in mixed methods research: How to and why not. Ecological and Environmental Anthropology, 3(1), 19-28.

Fink, Edward & Gantz, Walter (1996). A content analysis of three mass communication research traditions: Social science, interpretative studies, and critical analysis. Journalism & Mass Communication Quarterly, 73(1), 114-134.

Gläser, Jochen & Laudel, Grit (2009). Experteninterviews und qualitative Inhaltsanalyse. Wiesbaden: VS Verlag für Sozialwissenschaften.

Glaser, Barney & Strauss, Anselm (1967). The discovery of grounded theory: Strategies for qualitative research. New York, NY: de Gruyter.

Greene, Jennifer (2007). Mixed methods in social inquiry. San Francisco, CA: Wiley.

Greene, Jennifer (2015). Preserving distinctions within the multimethod and mixed methods research merger. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.606-615). New York, NY: Oxford University Press.

Greene, Jennifer & Hall, Jori (2010). Dialectics and pragmatism: Being of consequence. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral science (pp.119-143). Thousand Oaks, CA: Sage.

Greene, Jennifer; Caracelli, Valerie & Graham, Wendy (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274.

Guetterman, Timothy; Babchuk, Wayne; Howell Smith, Michelle & Stevens, Jared (2017). Contemporary approaches to mixed methods-grounded theory research: A field-based analysis. Journal of Mixed Methods Research, 13(2), 179-195.

Headley, Marcia Gail & Plano Clark, Vicki L. (2019). Multilevel mixed methods research designs: Advancing a refined definition. Journal of Mixed Methods Research. Online first.

Hesse-Biber, Sharlene & Johnson, Robert Burke (Eds.) (2015). Oxford handbook of multimethod and mixed methods research inquiry. New York, NY: Oxford University Press.

Hitchcock, John H. & Onwuegbuzie, Anthony J. (2019). Developing mixed methods crossover analysis approaches. Journal of Mixed Methods Research. Online first.

Huber, Stephan (2009). Schulleitung. In Sigrid Blömeke, Torsten Bohl, Ludwig Haag, Gregor Lang-Wojtasik & Werner Sacher (Eds.), Handbuch Schule: Theorie – Organisation – Entwicklung (pp.502-511). Bad Heilbrunn: Klinkhardt.

Hunter, Albert & Brewer, John (2015). Designing multimethod research. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.185-205). New York, NY: Oxford University Press.

Johnson, Robert Burke (2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156-173.

Johnson, Robert Burke (2018). Moving forward with mixed methods research. Keynote Presentation, MMIRA International Conference, Vienna, Austria, August 23, 2018.

Johnson, Robert Burke & Christensen, Larry (2014). Educational research. Quantitative, qualitative, and mixed approaches (5th ed.). Thousand Oaks, CA: Sage.

Johnson, Robert Burke & König, Stefan (2016). "Mono or mixed?" Potentials and limitations of mixed methods research. In Gregor Lang-Wojtasik & Stefan König (Eds.), Bildungsforschung revisited (pp.21-42). Ulm: Klemm & Oelschläger.

Johnson, Robert Burke & Turner, Lisa (2003). Data collection strategies in mixed methods research. In Abbas Tashakkori & Charles Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.297-319). Newbury Park, CA: Sage.

Johnson, Robert Burke, Onwuegbuzie, Anthony & Turner, Lisa (2007). Toward a definition mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133.

Kansteiner, Katja (2018). Mixing data and concepts of antidiscrimination for research on the relevance of "gender" in the recruitment process of school leaders in Germany. In Universität Wien & MMIRA (Eds.), MMIRA International Conference 2018—Book of abstracts (pp.150-151). Vienna: University Press, https://mmira2018.univie.ac.at/fileadmin/user_upload/k_mira2018/20180821_MMIRA_2018_book_of_abstracts.docx [Date of Access: October 30, 2019].

König, Stefan (2014). "Killing two birds with one stone"—On the effectiveness of implicit training processes in physical education. International Journal of Physical Education, 51(3), 15-28.

König, Stefan (2016). A plea for mixed methods research in research on teaching in physical education. Sportwissenschaft, 46(3), 179-187.

König, Stefan (2019). Evaluating fitness training in physical education—A quantitative dominated crossover mixed methods multilevel study. International Journal of Multiple Research Approaches, 11(1), 45-60.

Kracauer, Siegfried (1952). The challenge of qualitative content analysis. Public Opinion Quarterly, 16, 631-642.

Krathwohl, David (2004). Methods of educational and social science research. An integrated approach. White Plains, NY: Longman.

Krippendorff, Klaus (2013). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.

Kuckartz, Udo (2014). Qualitative text analysis: A guide to methods, practice and using software. Thousand Oaks, CA: Sage.

Kuckartz, Udo (2019). Qualitative content analysis: From Kracauer's beginnings to today's challenges. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 20(3), Art. 12, http://dx.doi.org/10.17169/fqs-20.3.3370 [Date of Access: October 2, 2019].

Love, Kevin; Pritchard, Catharine; Maguire, Kevin; McCarthy, Ann & Paddock, Paul (2005). Qualitative and quantitative approaches to health impact assessment: An analysis of the political and philosophical milieu of the multimethod approach. Critical Public Health, 15(3), 275-289.

Marvasti, Amir B. (2019). Qualitative content analysis: A novice's perspective. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 20(3), Art. 32, http://dx.doi.org/10.17169/fqs-20.3.3387 [Date of Access: October 2, 2019].

Maxwell, Joseph & Mittapalli, Kavita (2010). Realism as a stance for mixed methods research: Linking theory to practice. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral science (pp.145-167). Thousand Oaks, CA: Sage.

Mayoh, Joanne & Onwuegbuzie, Anthony (2015). Toward a conceptualization of mixed methods phenomenological research. Journal of Mixed Methods Research, 9(1), 91-107.

Mayring, Philipp (2015). Qualitative Inhaltsanalyse (12th rev. ed.). Weinheim: Beltz.

Mayring, Philipp (2019). Qualitative Inhaltsanalyse – Abgrenzungen, Spielarten, Weiterentwicklungen. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 20(3), Art. 16, http://dx.doi.org/10.17169/fqs-20.3.334316 [Date of Access: October 2, 2019].

Mayring, Philipp & Brunner, Eva (2010). Qualitative Inhaltsanalyse. In Barbara Friebertshäuser, Antje Langer & Annedore Prengel (Eds.), Handbuch Qualitative Forschungsmethoden in der Erziehungswissenschaft (pp.323-334). Weinheim: Juventa.

Mayring, Philipp & Gläser-Zikuda, Michaela (Eds.) (2008). Die Praxis der qualitativen Inhaltsanalyse. Weinheim: Beltz.

Mertens, Donna (2009). Transformative research and evaluation. New York, NY: Guilford.

Morse, Janice (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40(2), 120-123.

Morse, Janice (2015). Issues in qualitatively-driven mixed-method designs. Walking through a mixed-method project. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.206-222). New York, NY: Oxford University Press.

Morse, Janice & Niehaus, Linda (2009). Mixed method design: Principles and procedures. Walnut Creek, CA: Left Coast Press.

Neuendorf, Kimberly (2017). The content analysis guidebook. Thousand Oaks, CA: Sage.

Nzabonimpa, Jean Providence (2018). Quantitizing and qualitizing (im-)possibilities in mixed methods research. Methodological Innovations, 2(5), 1-16.

Onnen, Corinna & Rode-Breymann, Susanne (2017). Zum Selbstverständnis der Gender Studies: Methoden – Methodologien – theoretische Diskussionen und empirische Übersetzungen (L'AGENda). Leverkusen: Barbara Budrich.

Onwuegbuzie, Anthony & Hitchcock, John (2015). Advanced mixed analysis approaches. In Sharlene Hesse-Biber & Robert Burke Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.275-295). New York, NY: Oxford University Press.

Onwuegbuzie, Anthony & Johnson, Robert Burke (2006). The validity issue in mixed research. Research in the Schools, 13(1), 48-63.

Raudenbush, Stephen; Bryk, Anthony; Cheong, Yuk; Congdon, Richard & Toit, Mathilda (2011). HLM 7. Hierarchical linear and nonlinear modelling. Lincolnwood, IL: Scientific Software International.

Sandelowski, Margarete; Voils, Corinne & Knafl, Georg (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208-222.

Schoonenboom, Judith (2016). The multilevel mixed intact group analysis: A mixed method to seek, detect, describe, and explain differences among intact groups. Journal of Mixed Methods Research, 10(2), 129-146.

Schoonenboom, Judith & Johnson, Robert Burke (2017). How to construct a mixed methods research design. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 69(2), 107-131.

Schreier, Margrit (2012). Qualitative content analysis in practice. Thousand Oaks, CA: Sage.

Schreier, Margit (2014). Varianten qualitativer Inhaltsanalyse: Ein Wegweiser im Dickicht der Begrifflichkeiten. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 15(1), Art. 18, http://dx.doi.org/10.17169/fqs-15.1.2043 [Date of Access: September 7, 2019].

Stamann, Christoph; Janssen, Markus & Schreier, Margrit (2016). Qualitative Inhaltsanalyse – Versuch einer Begriffsbestimmung und Systematisierung. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 17(3), Art. 16, http://dx.doi.org/10.17169/fqs-17.3.2581 [Date of Access: August 29, 2019].

Strübing, Jörg (2014). Grounded Theory: Zur sozialtheoretischen und epistemologischen Fundierung eines pragmatistischen Forschungsstils (3rd ed.). Wiesbaden: Springer VS.

Tashakkori, Abbas & Teddlie, Charles (Eds.) (2003a). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.

Tashakkori, Abbas & Teddlie, Charles (2003b). The past and future of mixed methods research: From data triangulation to mixed model designs. In Abbas Tashakkori & Charles Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp.671-701). Thousand Oaks, CA: Sage.

Teddlie, Charles & Tashakkori, Abbas (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.

Teddlie, Charles & Yu, Fen (2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1(1), 77-100.

Van Velzen, Joke (2018). Students' general knowledge of the learning process: A mixed methods study illustrating integrated data collection and data consolidation. Journal of Mixed Methods Research, 12(2), 182-203.

Vogl, Susanne (2017). Quantifizierung. Datentransformation von qualitativen Daten in quantitative Daten in Mixed-Methods-Studien. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 69(Suppl2), 287-312.

Dr. Katja KANSTEINER is a professor of educational science at the University of Education Weingarten, Germany. Her main research interests are leadership and professional development at schools, gender and diversity in teaching and leading, and education for democratic citizenship. She lectures in teacher education and further program for education, and is a member of the leading team of the international Master program "School Development."

Erziehungswissenschaft
Pädagogische Hochschule Weingarten
Kirchplatz 2, 88250 Weingarten, Germany

Dr. Stefan KÖNIG is a professor of sport science at the University of Education Weingarten, Germany. Since 2008 he has been the director of the research center for secondary education. His main research interests are instructional research focusing on exercise and motor learning, team sport, and mixed methods research. He is also a consultant for several sport associations.

Sportwissenschaft
Pädagogische Hochschule Weingarten
Kirchplatz 2, 88250 Weingarten, Germany

Kansteiner, Katja & König, Stefan (2020). The Role(s) of Qualitative Content Analysis in Mixed Methods Research Designs [50 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 21(1), Art. 11, http://dx.doi.org/10.17169/fqs-21.1.3412.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License

	Category	Implicit exercises	Intentional exercises
Girls	Attractiveness	"I was highly motivated because of the interesting games"	"I believe that such strength exercises are boring"
	Intensity	"I did not really realize that this was for improving fitness"	"My goodness, this was exciting"
	Relevant goals	"I really like sport disciplines like track and field"	"... these exercises will not improve my performance outside PE"
Boys	Attractiveness	"For me this was boring"	"... overexerting oneself is great fun"
	Intensity	"These were no real challenges for me"	"I simply like exhausting things in sport"
	Relevant goals	"Things like swimming are not relevant for me"	"... I expect muscle growth"