Mixed Methods Research on Learning and Instruction—Meeting the Challenges of Multiple Perspectives and Levels Within a Complex Field

Abstract: In this paper, we present and discuss mixed methods research in the context of research on learning and instruction. Education as a field of research can be viewed as highly complex. This complexity is reflected at various levels of the educational system, which are highly interrelated, and where multiple perspectives must be considered, as well as in the reciprocal and intertwined relationships between factors related to learning and instruction. Therefore, we first introduce one of the central theories on the quality of learning and instruction: the offer-and-use model. Second, we review the methodological foundations of research on learning and instruction. Two methodological approaches are discussed in more detail and their strengths and weaknesses are elaborated. Third, we present two studies from our research program and focus on their methodological implementation. Thus, we illustrate significant challenges and opportunities for implementing a mixed methods study in schools. Finally, we discuss the implications for school-based mixed methods research. We conclude the paper with general suggestions on how mixed methods approaches might be further advanced in applied school-based research.

Key words: learning; instruction; school; self-regulated learning; offer-and-use model; intervention; mixed methods

4. How Can the Quantitative-Qualitative Divide be Addressed in Mixed Methods Research—General Reflections and Two Examples

5. Conclusion and Future Perspectives for Mixed Methods Research on Learning and Instruction

Education, or more specifically, learning and instruction, can be regarded as a very broad field. Researchers have not yet found the holy grail of instruction. In other words, due to the complexity of educational processes, it is impossible to identify one single factor defining high-quality instruction, and this situation is likely to continue in the future. Nevertheless, educational researchers still aim to identify core factors and processes that contribute to desired educational outcomes, which are normatively set. At the same time norms steadily change over time with significant implications for both educational theory and the methodology used by researchers to develop and refine those theories. [1]

In research practice, educational researchers are aware that the proposed associations between micro-, meso-, and macro-level contextual factors (FEND, 1980) and student learning outcomes are never the same for all students, as students themselves bring different basics to the situation and make meaning differently out of the situation. Given this understanding, researchers need a research approach or paradigm with which they can account for the expected regularities between factors (e.g., teachers' high educational competence is likely to lead to better teacher-student relations). At the same time, by choosing a certain paradigm, researchers should not neglect the subjectivity of the respective situation (e.g., students interpret the same teaching behavior differently, consequently leading to different reactions). [2]

We believe that applying mixed methods approaches grounded in dialectical pluralism as a metaparadigm (JOHNSON, 2015) is a fruitful avenue for addressing these two interrelated facets of how classroom realities are constructed. More specifically, integrating qualitative and quantitative research perspectives in the methodology implemented is necessary and elementary for a more comprehensive analysis of learning and instruction. [3]

In this paper we specify how mixed methods can be utilized to examine complex problems in research on learning and instruction. In so doing, we begin by giving an overview of the complexity of the phenomenon learning and instruction (Section 2), followed by an overview of the common methodological foundations in instructional research in general (Section 3). Based on these theoretical and methodological reflections we then discuss the challenges that arise when applying mixed methods in instructional research and give two examples of mixed methods studies from our own research programs (Section 4). We conclude our arguments by developing implications for future research in learning and instruction with a mixed methods methodology (Section 5). [4]

Educational scientists have proposed several core theories to understand learning and instruction in schools. In German-speaking countries, the offer-and-use model, initially introduced by FEND (1980) and further developed by HELMKE (2015), has become popular. In the model, educational researchers have integrated various theories of learning and instruction to produce a kind of meta-model which they have found useful for illustrating the complexity of the phenomenon. As SEIDEL (2014) contended, we cannot fully test this model empirically within one study; however, using the model as a meta-framework enables researchers to classify and systematize studies in the field. [5]

According to the model (Figure 1), learning outcomes have multiple determinants and are influenced by various macro-, meso-, and micro-level factors and their interplay (ibid.). The differentiation between the macro-, meso-, and micro-levels dates back to FEND (1980) and his theory on schools. Embedded in a broad social system, schools are affected by macro-level factors (e.g., the educational system), meso-level factors (e.g., the school climate), and micro-level factors (e.g., the classroom climate). This distinction between macro-, meso-, and micro-levels is part of the offer-and-use model. Importantly, learning outcomes should not be narrowly defined as purely cognitive outcomes (i.e., student achievement) but should also include non-cognitive outcomes such as emotions, motivation, or self-regulation, mirroring a broad understanding of Bildung [education] as developing students' personalities (BIESTA & STENGEL, 2016; KLIEME, 2019). Two main factors in the model are considered important for student outcomes: what teachers offer students and how students use this offer. For example, students' achievements in a particular subject (e.g., math, geography, history) depend on the wider educational context (e.g., how the school system is structured), the school context (e.g., how teachers cooperate in the respective school), the classroom context (e.g., the amount of heterogeneity in the class), teachers' competencies (e.g., teachers' subject knowledge) and how teachers teach in class (e.g., cognitive activation, teacher support, and classroom management). The offer is determined by all these factors. In addition, students use this offer differently, as reflected in the realized learning activities depending on their individual characteristics (e.g., motivation, intelligence) and their wider learning environment (e.g., their socioeconomic status). Learning outcomes are thus a result of offer-and-use factors, their interactions, and the underlying processes.

Figure 1: Offer-and-use model according to SEIDEL (2014, p.858). Click here to download the PDF file. [6]

In describing the offer-and-use model, it becomes evident that learning and instruction are phenomena with multiple determinants. On the one hand, they can be described by complex (reciprocal) relations between factors on the individual and contextual level. On the other hand, they are highly individualized processes that strongly depend on the students and the meaning that they attribute to the situation, which consequently affects how they perceive and use the offer. Researchers in the field intensely discuss the context-specificity and situatedness of core concepts in learning and instruction (e.g., students' achievement, motivation, or self-regulated learning), resulting in methodological advancements on how to appropriately address those factors (GLÄSER-ZIKUDA & JÄRVELÄ, 2008; JÄRVELÄ & BANNERT, 2021; VOLET & VAURAS, 2013; WINNE, 2019). [7]

We have repeatedly used the term process in our description, thus suggesting that learning and instruction are fluid rather than static phenomena. When researchers adopt this perspective, they will face high demands when constructing theories and utilizing the appropriate methodology. Schools can be regarded as valuable societal institutions that are in a constant process of change. On the one hand, this is due to emerging social changes that have a substantial impact on education, such as digitization or globalization. Digitization processes, as one of many macro-level examples of change, have a strong influence on the organization of schools and instruction, for example, when integrating new media into the classroom. On the other hand, on the micro-level, educational researchers must constantly modify their beliefs about high-quality learning and instruction. In this context, researchers are increasingly emphasizing students' self-regulation in learning, which reflects the shift from a predominantly behaviorist to a more constructivist understanding of learning processes (see on this discussion: HMELO-SILVER, DUNCAN & CHINN, 2007; KIRSCHNER, SWELLER & CLARK, 2006). In response to this shift, changes in the classroom environment were initiated (e.g., an increase in open or low(er) guided instruction) accompanied by an increasing promotion of students' competencies in self-regulated learning (e.g., as reflected by the need for students to acquire meta-cognitive skills; BOEKAERTS, PINTRICH & ZEIDNER, 2000; SCHUNK & GREENE, 2018; ZIMMERMAN & SCHUNK, 2011). [8]

Changes on one level (macro, meso, micro) are likely to carry over to other levels because the three levels are inextricably and reciprocally linked. Consequently, educational theories and methodologies that can be used to account for the processual nature of learning and instruction in general and changing (societal) conditions in particular need to be constantly adapted and refined. As an example of a change at the macro-level, researchers are increasingly raising the question of how the digitization of instruction can and should be designed to support students' learning and personality development in a digitized learning environment and, more generally, in an increasingly digitized world (see, for example, OECD, 2021). This issue has implications for almost all components of the offer-and-use model (HELMKE, 2015). For example, if we seek to understand learning and teaching in digitized environments, we need to (theoretically) develop models of teachers' digital literacy (e.g., CAENA & REDECKER, 2019; MISHRA & KOEHLER, 2006), to define learners' digital competencies (FRAILLON, AINLEY, SCHULZ, FRIEDMAN & DUCKWORTH, 2020), and to identify and describe quality indicators for digitized learning environments (BLÖMEKE, 2000; LORENZ & ENDBERG, 2019; REDECKER, 2017). [9]

In sum, in developing the offer-and-use model of learning (e.g., HELMKE, 2015), educational researchers have presented a helpful meta-model to understand the complexity of student learning in context (macro, meso, micro), underscoring the dynamic and multilayered nature of the phenomenon under study. Due to the interdependency of the factors introduced in the offer-and-use model, educational researchers committed to research on learning and instruction face many methodological challenges in determining how best to empirically investigate these complex interrelations in context. In the following sections, we first describe the methodological foundations often encountered in research on learning and instruction and then consider how mixed methods approaches may be useful in overcoming potential challenges. [10]

To understand the methodological challenges of research on learning and instruction, it is necessary to identify common research paradigms in educational science. In doing so, not only is the scope of our analysis narrowed down, but also the basic precepts for applying quantitative and qualitative methods in school research become clear. According to KUHN (2012 [1962]), research paradigms are generally recognized scientific achievements that are useful for solving problems at a given time. Although KUHN attributed a pre-scientific character to social sciences, his work had a profound influence on their development. As a further consequence of KUHN's reflections, qualitative research became well established in the social sciences during the second half of the 20th century (HAMMERSLEY, 2008) by researchers strengthening constructivist perspectives and challenging the hegemony of the quantitative paradigm. Historically, the reception and application of mixed methods in school and classroom research has evolved in parallel with that in the social sciences in general (MEJEH & HAGENAUER, 2021). In this regard, note that the term paradigm is used in many ways that differ not only in a purely linguistic sense. For example, the terms methodological traditions (MOSS & HAERTEL, 2016) or world views (CRESWELL & CRESWELL, 2017; JOHNSON & CHRISTENSEN, 2019; ONWUEGBUZIE & LEECH, 2005) are more or less used synonymously. Regardless of the rhetorical differences (GUBA, 1990; MORGAN, 2007), according to the widely-used definition suggested by LINCOLN and GUBA (1985), a paradigm is based on four essential questions:

An exhaustive list of all paradigms important for educational science is not the subject of this paper, if only because approaches are used inconsistently and the terminology differs (HAMMERSLEY, 2012). Nevertheless, researchers have identified four basic paradigms for the social and educational sciences, which will be outlined briefly (CRESWELL & CRESWELL, 2017; MACKENZIE & KNIPE, 2006; TASHAKKORI & TEDDLIE, 2003): First, this is the postpositivist paradigm, characterized by critical realism in which an objective reality exists but cannot be fully grasped. The primary object of research is the observation and measurement of an objective reality, which, however, remains a "regulative ideal" (GUBA, 1990, p.23) that scientists can only attempt to attain by their research. Second, we refer to the constructivist paradigm, characterized by an interpretive (verstehen) perspective in which the behavior of individuals is examined in context. In contrast to the postpositivist paradigm, researchers following the constructivist paradigm are not concerned with testing theory, but rather with generating new theory. Thirdly, the transformative paradigm is characterized by a focus on changing the lives of marginalized individuals. At its core is the conviction that political change can be generated through scientific knowledge. A specific feature of this paradigm is a participatory approach, which means that the people being researched are included in planning and conducting the research. A key characteristic of the fourth paradigm, namely pragmatism, is a problem-oriented understanding of research, characterized by an action-oriented approach. In pragmatism, different methodological approaches can be combined throughout the research process (e.g., sampling, design, or analysis). For social sciences in general—and educational science in particular—the development of a common trend during the 20th century can be identified: Initially, a largely positivist model of science was predominant, and the influence of qualitative (and thus more interpretative) researchers and their work increased from the 1960s onwards (HAMMERSLEY, 2012). [12]

When we focus more specifically on research on learning and instruction, the valuable work of MOSS and HAERTEL must be highlighted. In their review, they compile and discuss the ten most important methodological traditions (2016, p.133):

Not all ten methodological traditions can be discussed in detail here. However, we briefly outline two approaches and their role in classroom research, one of which is traditionally assigned to the qualitative research spectrum and the other to the quantitative research spectrum. In research on learning and instruction, researchers working with quantitative methods primarily seek to draw intervention-based causal inferences from experimental and quasi-experimental designs. The focus of this approach lies on the variable-based operationalization of constructs, such as student achievement, well-being, or motivation. Researchers examining learning and instruction in this way use research designs characterized by the use of various statistical methods (e.g., regression analysis or hypothesis testing), the selection of random samples, falsifiable questions, or statistical generalization. Their goal is to report results factually and objectively (MOSS & HAERTEL, 2016). Experimental or quasi-experimental designs are an example of a quantitative approach. School researchers develop theoretical assumptions deductively with the aim of generalization. Theories are applied and tested in this perspective, yielding answers to questions such as To what extent does reading training increase the reading ability of lower secondary school students?¹⁾ or To what extent does instructional design influence the well-being of elementary school students? [14]

In ethnography, a qualitative research approach, researchers rather aim to interpret the actions of actors within their social context. The distinction between action and behavior is essential (ibid.). In the context of research on learning and instruction, social phenomena are the object of study. In the classroom, for example, researchers conduct field studies to collect and analyze interactions between various actors. Researchers accompany school classes over a longer period of time and they can continuously adapt their research questions, data collection, data analysis, and presentation of results based on an iterative process. In terms of methodology, school researchers primarily use participatory observation or interviews, although in recent years the use of video observations has proven particularly helpful for research on learning and instruction, especially because researchers using this data collection method have good opportunities for connecting to quantitative classroom research. In ethnographic research approaches, data analysis is guided by the principles of constant comparison and analytic induction (ERICKSON, 1986). While the former means making continuous comparisons between different units of analysis (e.g., students, teachers, or entire school classes), the latter refers to generalizing the content of the insights gained through repeated analysis of the data material (e.g., searching for certain patterns of interaction between school administrators in a group interview). Result reporting in ethnographic research on schools is characterized by descriptions, assertions, or conclusions that are based on patterns that have been discovered (MOSS & HAERTEL, 2016). This allows school researchers to study negotiation processes among peers, for example, by examining peer interactions on the playground or during open learning in more detail. Thus, researchers can reconstruct and analyze various value systems based on the differences between them. Ethnographic research on schools has a theory-developing function and is characterized by an indeterministic, inductive, and context-sensitive nature. [15]

However, the two examples outlined above also share the shortcomings of monomethod research approaches in the context of research on learning and instruction. With regard to experimental and quasi-experimental research designs, the question arises of how the challenges of internal and external validity are managed (CRESWELL, 2012; CRESWELL & CRESWELL, 2017). In longitudinal studies, for example, unintended time effects may arise if interventions are planned and implemented over multiple school years. Consequently, school researchers find themselves in a situation where they have to find comparison groups that have the same experiences outside of treatment and are the same age. However, this approach often proves difficult to implement, especially in school research. Participant selection may also be affected because students with the same characteristics are sought so that valid results can be achieved in experimental and quasi-experimental studies. The randomized assignment of students to an intervention or control group is usually difficult to realize (e.g., waiting control groups). Furthermore, sample mortality is an additional challenge. This is a major issue, because during a school year, teachers may change their classes, students may move to a different location, or school structures may change (e.g., formation of new classes). Regarding external validity, school researchers often face the challenge of not being able to generalize their findings because the setting selected is too specific, which is one of the difficulties associated with sample generation in research on learning and instruction (GOPALAN, ROSINGER & AHN, 2020). The generation of large samples could help, but they are not the norm, except in large national or international education panels. In large panel studies, however, researchers are confronted with the problem that they often do not examine interventions, so that statements can be made about correlations but not about causal influences (HSIAO, 2007). [16]

Similar to quantitative research approaches, certain weaknesses can also be ascribed to qualitative research approaches in research on learning and instruction. Ethnographers, for example, are confronted with the problem of going native (e.g., LeCOMPTE & PREISSLE GOETZ, 1982): If researchers accompany a school class for a longer period, they may run the risk of identifying too strongly with the observed class events and losing the necessary distance to reflect on what they have observed. In addition to the challenge that researchers must possess excellent observational skills, a data collection problem for qualitative school researchers is highlighted in this example as well. Biases in collecting data (e.g., researchers are seen as disruptive in the school or teachers respond in interviews based on social desirability) can be caused by researchers' direct interaction within the school setting. Furthermore, qualitative research is usually very time- and resource-intensive, which is why only a small number of cases can be considered. It is true that qualitative social researchers—and thus also qualitative school researchers—do not aim for statistical but theoretical generalizations (DIRIWÄCHTER, VALSINER & SAUCK, 2005). The theoretical generalization of research results refers to the (further) development of theories. Qualitative findings are generalized on a solid theoretical foundation, which means that the quality of the selection of cases to be studied is decisive, not their quantity (FLICK, 2017). Nevertheless—or precisely because of this—qualitative researchers are likely to be confronted with the question of whether their results do not ultimately merely depict a fragmentary picture of the subject area. If, for example, the actions and behavior of students at recess are investigated to reconstruct a prevailing system of norms for interacting at a school, the question always arises whether the interaction between learners in break situations at other schools is similar, the same, or different. This limitation stems from the assumption of the non-determinacy of human action (KELLE, 2015) in qualitative social research in general and ethnographic field research in particular. [17]

4. How Can the Quantitative-Qualitative Divide be Addressed in Mixed Methods Research—General Reflections and Two Examples

In the context of research on learning and instruction, disparities in defining the objects and goals of instruction necessitate the use of different (i.e., qualitative or quantitative) methods (HELSPER & KLIEME, 2013). For example, if we follow the offer-and-use model outlined above, we are primarily concerned with determining the quality of instruction. When researchers label instruction good or effective (BERLINER, 2005; KLIEME, 2019; REUSSER, 2018; SEIDEL & SHAVELSON, 2007), they thereby already suggest fundamentally different interpretations of what is considered important in the analysis of instructional processes. Norm setting (in the sense of good or effective) in classroom research can be grouped into two different categories: observed normativity on the one hand and observational normativity on the other. In the former, researchers aim to determine what classroom actors themselves understand by good teaching, while the latter involves referring to research on learning and instruction, and thus categorizing what happens in the classroom as "effective" or "less effective" (PRAETORIUS, MARTENS & BRINKMANN, 2021, p.5)²⁾. In this sense, qualitative school researchers tend to investigate normative conceptions and practices of action in terms of observed normativity (e.g., How does a teacher behave in a heterogeneous classroom setting? and How does the teacher perceive the classroom climate?), while quantitative school researchers pose questions of observational normativity and tend to answer issues related to selected aspects of instruction (e.g., To what extent does the cognitive activation of students influence their performance?). [18]

So, the analysis of learning and instruction, as described above, is based on methodological traditions with specific characteristics and orientations through which research approaches are shaped. Furthermore, due to this distinction, it becomes possible for researchers to reflect on observed and observational normativity (PRAETORIUS et al., 2021). Following the idea of observational normativity, a stated goal in quantitative research on learning and instruction, with its statistical procedures, can be "investigating interactions between students' individual dispositions and learning activities, teachers' professional competencies and current actions, students' social contextual conditions, and teachers' learning activities, social contextual conditions, and interactional processes" (HELSPER & KLIEME, 2013, p.285). In qualitative school research, in contrast, scholars are only rudimentarily interested in these questions since, from this perspective, pedagogical action in its entirety is always characterized by structural uncertainty. Thus, instruction can be understood as a learning space in which students learn, but often not in the sense of what teachers want them to learn (HELSPER & KLIEME, 2013). Following the idea of observed normativity, qualitative researchers are mostly interested in how schools and classrooms are constituted as social realities or how educational orders are shaped as power structures (BÖDER & RABENSTEIN, 2021). Thus, it becomes obvious where qualitative and quantitative research on learning and instruction overlap and where they can fruitfully complement each other. [19]

In mixed methods research, researchers can systemically combine qualitative and quantitative approaches in school and classroom research (GLÄSER-ZIKUDA, SEIDEL, ROHLFS, GRÖSCHNER & ZIEGELBAUER, 2012; HAGENAUER & GLÄSER-ZIKUDA, 2019; MEJEH & HAGENAUER, 2021). Schools as institutions—and teaching as their essential constituent element—can be seen per se as a space in which pedagogical interactions are always embedded between the two extremes of situativity and structuredness (social action) described by KELLE (2017). Actors in schools follow their own theory of action, which presupposes a certain form of participation in the scientific process. At this point, different methodological traditions for research on learning and instruction can be combined, because schools are complex organizations characterized by distinct features (FEND, 2008) and thus can be approached with different levels of analysis that can be connected in the context of mixed methods research. For example, when teachers are involved in the development of a new instructional intervention (action research, e.g., ALTRICHTER, POSCH & SPANN, 2018) or whenever school culture needs to be further developed (design-based research, e.g., PREDIGER, GRAVEMEIJER & CONFREY, 2015). [20]

In the following, we will describe two examples of mixed methods studies in more detail. By doing so, we will discuss the challenges and potentials of mixed methods studies in educational research based on concrete examples. [21]

Promoting students' self-regulated learning (SRL) is one of the central topics of research on learning and instruction, and the magnitude of its relevance has become clear again during the COVID-19 pandemic (keyword: distance learning). Different models of SRL now exist (e.g., BOEKAERTS, 1992; WINNE & HADWIN, 1998; ZIMMERMAN, 2000), all of which have been empirically tested repeatedly. By referring to these models, educational researchers believe they are able to account for different aspects of learning. More specifically, not only cognitive processes, but also students' motivation and emotions need to be considered to make learning effective. By using the offer-and-use model, researchers have a strong basis for analyzing learning and instructional processes in schools. On the one hand, from a theoretical point of view, the reciprocal relationships between individuals and context can be theorized at various levels (supply, use, outcome). On the other hand, from a methodological point of view, different analytical possibilities emerge, including the combination of qualitative and quantitative methods. Thus, positioning SRL models within the framework of the offer-use-model can be seen as connecting two theories "of varying degrees of abstractness" (KELLE, 2015, p.596). While exploring micro-processes by studying SRL in the classroom, researchers are able to frame SRL processes in a larger context (in this case, the students' learning processes, which are also determined by school or family structures, for example) by employing the offer-and-use model as a macro-theory. Two studies from our research program that can be positioned within the offer-and-use model are presented in detail below, with a focus on their methodological implementation. Both studies can be classified as intervention studies and assigned to the broad research field of SRL. Intervention studies are vital to education research, as educational researchers ultimately aim to enhance the quality of learning and instruction based on strong empirical evidence. [22]

We first outline the Emotional and Cognitive Learning (ECOLE) study (GLÄSER-ZIKUDA, FUß, LAUKENMANN, METZ & RANDLER, 2005). ECOLE was designed as a mixed methods intervention study in which the researchers aimed to enhance secondary students' positive emotions and achievement, and reduce students' negative emotions in the school subjects German, biology, and physics based on an instructional approach relying on SRL principles. The study was based on principles of self-regulated learning derived from the models described by BOEKAERTS (1992) and ZIMMERMAN (2000), including the enhancement of cognitive, affective, and social aspects of students' learning. Learning environments were created to support SRL, such as a free choice of learning tasks and material (differentiated by the type of cognitive, emotional, or social activation and achievement level; choice of learning partner; choice of learning order; and the duration of the learning phases). In addition, students were asked to compile a short portfolio to support them in documenting, observing, and reflecting on their learning process (GLÄSER-ZIKUDA & GÖHRING, 2007). [23]

The study was based on a quasi-experimental mixed methods design in which standardized questionnaires, achievement tests, video-based classroom observation, semi-structured interviews, and diaries (for students and teachers) were combined to analyze emotional and cognitive aspects of students' learning processes and achievement outcomes. While the quantitative approach was designed to test the effectiveness of the intervention with the aim of reducing students' negative emotions and enhancing students' positive emotions and achievement, the aim of the qualitative part of the study was to better understand how the implementation of the ECOLE instruction happened, as well as how it was experienced by the students and teachers. [24]

The ECOLE intervention study was conducted in the 2001-2002 school year. The sample for the whole ECOLE study consisted of 1,010 students from 8th and 9th grades (females n=511; males n=499) from 37 classes of all three school types of German secondary schools: Gymnasium (n=405), Realschule (n=493), and Hauptschule (n=112). The sample is a convenience sample and sample sizes differ due to the different willingness of schools to participate in the study. Students were aged from 13 to 15 years. The student sample sizes for each subject were: biology (n=519), German (n=456), and physics (n=480). In total, 37 teachers participated in the study.

In the quantitative part of the study, standardized instruments were used to measure individual (e.g., emotion, motivation, self-concept), social (e.g., school climate), and instructional aspects (e.g., instructional quality). In general, the instruments were of good to very good quality (GLÄSER-ZIKUDA et al., 2005). The researchers differentiated between long versions of questionnaires to measure traits (e.g., school enjoyment), and short questionnaires to measure states (e.g., emotions in single lessons). Traits were only measured twice, while states were measured several times. The shorter questionnaire was developed to ensure that students would complete it at the end of a lesson. Applying these instruments allowed us to test the hypothesized effects on different cognitive and emotional variables, and to test for differential effects regarding subgroups of students (e.g., gender) and the various instructional phases. In the qualitative part of the study, different participants (students and teachers), and methods (interviews, video observation, diaries) were used parallel to the quantitative measures. [26]

The control groups started in the first part of the school year. Five biology teachers, seven physics teachers and five German teachers taught classes (control groups). Afterwards, the same teachers were trained during a weekend course to apply the ECOLE instructional approach in a different class at the same school (treatment groups). The treatment groups were taught based on the ECOLE approach in the second part of the school year. All teachers taught the same instructional content with the same learning goals over a period of 12-18 lessons in the control and treatment groups. The ECOLE approach was applied in three teaching units, one each in biology (topic: ecology), physics (topic: electricity), and German (topic: grammar and punctuation). Teaching was divided into two instructional phases for each unit: The first phase consisted of an initial teaching phase, which was highly student-centered with no performance pressure and no grades. At the end of this first phase, students received individual feedback from the teacher based on a formative short test. In the second phase of the ECOLE intervention, teachers focused on individually supporting student learning by offering different and adaptive types of learning tasks. In addition, the students were presented with effective learning strategies for test preparation and strategies for coping with test anxiety. This intervention study with pre-, post-, and follow-up measurements was conducted using a mixed methods intervention design (CRESWELL & PLANO CLARK, 2018) in order to test for the hypothesized effects and to gain an in-depth understanding of the ECOLE approach. We simultaneously conducted the quantitative and qualitative parts of the study during the same phase of the research process. The quantitative and qualitative parts of the study were of equal importance. The qualitative and quantitative data were analyzed independently in a first step and combined during the interpretation of results (CRESWELL & PLANO CLARK, 2011). [27]

Qualitative and quantitative data were collected at the beginning, during the course of the study, and at the end. Therefore, we were able to analyze different perspectives (students', teachers', researchers') on the research topic, and to collect data before (long questionnaire, interview), during (video observation), and after the instructional process (long and short questionnaires, interviews, and diaries). Time, and therefore different measurement points, was a core aspect of the study. [28]

In the quantitative part, three main measurements in the treatment and control groups were conducted before the intervention (or instructional unit), after the intervention (instructional unit), and six weeks after completing the intervention (instructional unit). We analyzed the data with quantitative procedures to test for the hypothesized effects of the intervention. In the qualitative part, we conducted interviews with teachers before the intervention to find out more about their beliefs, teaching preferences, and the teacher-student relationship in the classes they taught. This information was essential to interpreting unexpected instructional effects, for example, and was also used for further quantitative analyses. In addition, selected instructional lessons were observed with video recordings made during the intervention to collect information about the actual implementation of ECOLE instruction. Finally, we conducted interviews with selected students before and after the entire intervention or instruction (in control groups) to investigate implementation quality in more detail. For example, students reported how well prepared they were for the achievement tests, as well as how teachers organized the lessons and used specific instructional material. Furthermore, in a second step we combined qualitative and quantitative data to analyze students' emotions and teacher competencies (GLÄSER-ZIKUDA & FUß, 2008). Data from interviews with students were transcribed and analyzed with structured and scaled qualitative content analysis, which yielded ordinal scaled data for perceived instructional quality. These qualitative data were transformed into quantitative data and used together with data from a standardized questionnaire measuring learning emotions of the same students for further quantitative analyses. [29]

In the quantitative analyses, we tested the hypothesized effects of the ECOLE intervention on students' emotions and learning outcomes in the three school subjects. The interviewed students were theoretically sampled based on the criteria of high and low achievers as well as boys and girls. The aim of the analysis was to gain a broader and deeper understanding of their perceptions of the school subjects and their instruction, as well as the emotional and cognitive aspects of their learning processes and outcomes. For example, students reported in interviews how they perceived the various instructional phases and the learning material, such as tasks in physics (GLÄSER-ZIKUDA & GÖHRING, 2007). The same students were observed during the intervention in selected lessons (one lesson in the first instructional phase and one in the second) to obtain further insight into their individual emotional experiences. Not only cognitive and motivational but also behavioral and expressional dimensions of students' emotions were examined (MAYRING, GLÄSER-ZIKUDA & ZIEGELBAUER, 2005). [30]

We conducted interviews with teachers at the beginning of the intervention to gain information about their basic teaching styles. After each lesson, all teachers described the lessons in terms of instructional conditions (teaching methods, teaching media, etc.) during the overall instruction (control groups) and in the intervention phase (treatment groups). The teachers recorded the data in a semi-structured diary that was used to document and control implementation quality (DURLAK, 2016). Both the interviews and the diaries were used to describe implementation quality and to control for teacher effects (GLÄSER-ZIKUDA & FUß, 2008). [31]

The mixed methods design of this intervention study comprises various types of methods and data (qualitative and quantitative), different types of analysis (qualitative and quantitative), as well as different perspectives (students', teachers', and researchers'). By collecting qualitative data (e.g., the interviews), we were able to take a more subjectively oriented approach to analysis. Thus, by integrating the qualitative data, additional information could be added to the interpretation of the quantitative data from the questionnaires. Furthermore, qualitative data were used to validate the quantitative data. Finally, qualitative data from the interviews with students were analyzed in a scaled content analysis, which allowed us to quantify qualitative data and compare them with quantitative data (ibid.). The ECOLE intervention encompassed multiple factors including teachers, students, and classes as well as school types, subjects, and instructional phases and methods. Therefore, the benefit of combining quantitative and qualitative research methods in the intervention study seems not only reasonable but also necessary to describe, understand, and explain the processes and effects of the ECOLE intervention. [32]

In the study SelBer (Selbstgesteuertes Lernen in der BERufsbildung; Self-Regulated Learning in Vocational Education), the development of SRL among a total of 160 students was recorded over an entire school year as part of implementing a learning environment conducive to SRL (MEJEH & HELD, 2022). One goal of the study was to conduct an analysis of vocational learners' learning processes, focusing not only on dispositional but also on situational aspects of learning. Another goal was to demonstrate the effectiveness of the implemented learning environment in supporting vocational learners' SRL. Furthermore, it was of utmost importance to continuously monitor the implementation of the intervention at the school level. In the quantitative part of the study, the following questions were addressed:

In the qualitative part of the study, the following questions were investigated:

We selected the study as an example for the application of mixed methods in the field of learning and instruction for two reasons. SelBer can be classified as a typical classroom study since the action mechanisms of an educational intervention are explained. Unlike ECOLE, however, the focus in SelBer was more strongly on how SRL changes over time. At the same time, however, how the detected action mechanisms can be understood and, if necessary, how they come about were also considered very important. The purpose of the study was to resolve the gap between empirical observations at the micro-level and general theoretical statements at the macro-level, for which a combination of quantitative and qualitative methods is of great benefit (KELLE, 2015). The weaknesses and limitations of one methodological approach can be balanced by the strengths of the other approach (JOHNSON, ONWUEGBUZIE & TURNER, 2007). In terms of mixed methods research, the goal in this study was therefore a more holistic analysis of a classroom intervention, bringing together what was originally thought of as disparate concepts, understanding and explaining, in terms of scientific theory, without being tied to an underlying research paradigm (KELLE, 2017). [35]

To answer the research questions posed in the SelBer study, a complex mixed methods design was required. The design is described in Figure 3.

Figure 3: SelBer research design³⁾. Click here to download the PDF file. [36]

After the introduction of the intervention at the vocational school (July 2019), a pretest was undertaken at the beginning of the school year in the form of a detailed questionnaire survey and interviews (August 2019; LQ). From September 2019, coaching sessions between vocational learners and teachers were audio recorded monthly (LC; qual.), while at the same time, from this point onward, short-term effects (SQ) and written weekly planning and self-reflection activities (WpSr) were collected. Interviews with selected learners and teachers were conducted in September 2019 and July 2020 (I; qual.). A post-test was undertaken at the end of the school year (July 2020; FL-LZE, quan.). [37]

Two questionnaires were used to collect quantitative data: a detailed questionnaire for the vocational learners from the intervention and control classes administered at two measurement points (LQ) one year apart, and an abbreviated questionnaire for vocational learners from the intervention and control classes administered at a total of 40 measurement time points over the course of one year (SQ). Semi-structured interviews were used to collect qualitative data. Interviews were conducted with vocational learners and teachers from the intervention and control classes at a total of two measurement points over the course of one year (I). Learning coaching sessions between vocational learners and teachers from the intervention classes were collected at a total of eleven measurement time points over the course of one year (LC). Weekly planning and self-reflection sessions involving the vocational learners in the intervention classes were collected at a total of 40 measurement points over the course of one year (WpSr). The total of 40 measurement time points was achieved by measuring the development of SRL every week for one school year. It was thus possible to analyze the development of the individual SRL components (cognition, emotion, motivation) over the course of one school year using different growth models. [38]

Time, and thus the timing of the survey, was a central component, because both short- and long-term intervention effects were measured. However, as the study goals also included capturing students' and teachers' experience and descriptions, a triangulation of procedures was required. For this reason, we decided on a pre-post-parallel mixed methods design. Qualitative and quantitative data were collected at the beginning, during, and at the end of the study. Thus, different perspectives on the subject under investigation were recorded, both during and after the process (KUCKARTZ, 2014). In quantitative terms, in addition to the two main measurement points at the beginning and end of the school year, weekly time series measurements were made. In the qualitative part, interviews with selected vocational students and teachers were conducted. In addition, coaching sessions between students and teachers were audio recorded, while learners were also encouraged to complete weekly self-reflection and learning plans. [39]

The aim of the time series analysis was to be able to more precisely explain the students' learning processes. Based on the quantitative results, extreme cases in the sample were identified for the qualitative analyses, i.e., vocational learners who exhibited either very strong or very weak use of various self-regulatory learning strategies. Interviews were then conducted with the selected learners and their learning coaching conversations were analyzed. The goal of the interviews was to gain a deeper understanding of the vocational school students' processes of learning development and their learning environment. Moreover, the analysis of the self-reports (reflections and weekly planning) provided in-depth insight into students' perceptions of the learning environment. To combine both methodological strands of the project, the interview questions were aligned to the variables collected in the time series measurements. For example, learners were questioned about using their planning and organizational strategies in the questionnaires, while they were then asked to describe their working and learning strategies within a 4-week time cycle in the interviews. [40]

Before elaborating on the added value of combining quantitative and qualitative research in the SelBer study, an important distinction must first be made. Due to the previously addressed goals of the study, a mixed methods research approach seems not only reasonable but also necessary, as the research questions were aimed at different epistemic goals: First, while the weekly measurements were used to analyze the effects of the intervention on the development of SRL, the actors' perception of this development was elicited in the interviews. Second, how the intervention was realized at the school was addressed through the questions on implementation, and thus, which opportunities and challenges arise when a classroom intervention is inductively co-developed with actors in the practice field. [41]

Thus, when evaluating the SelBer measure as an inductively developed intervention, in addition to analyzing learners' learning processes, an essential component was to also keep implementation at the school level in mind (treatment check). This approach is very similar to those of design-based research or action research, since findings regarding the implementation of the intervention were generated during the evaluation process and these were continuously discussed between practitioners and researchers. Due to the complexity of the design, particularly in surveying the development of SRL over time, questions about the study's feasibility were raised. Therefore, the standardized collection of data was closely linked to the intervention. Among other things, this also meant that any adjustments in the research design could always be mirrored and critically discussed in terms of their concrete implementation at the school. [42]

An important prerequisite for demonstrating the effectiveness of the intervention is an approach where both qualitative and quantitative methods are incorporated. On the one hand, there is the challenge that self-reports in the form of questionnaires are only of limited significance when self-regulation components are addressed. Therefore, due to certain learning processes, specific individual learners were interviewed to enable the researchers to not only make causal statements (e.g., based on the degree of regression in the growth model) but also to capture the perception and experience of the participants in this regard. On the other hand, it was important for the researchers in this project to identify individual components of the learning environment that students assessed as important for their learning processes. Thus, it with the questionnaires was possible to identify how the learners planned their learning process in the classroom. However, it was only by also considering the reflection documents and weekly planning that it became possible to understand how learners reflect on their planning processes. [43]

5. Conclusion and Future Perspectives for Mixed Methods Research on Learning and Instruction

By describing two examples of intervention studies based on mixed methods designs in research on learning and instruction, we illustrated the strengths of mixed methods research in approaching complex educational phenomena such as instructional and learning processes. In both studies presented, it is evident that within the context of the offer-and-use model (HELMKE, 2015), a more holistic analysis of learning and instruction became possible by combining qualitative and quantitative methods. For ECOLE, this was done by recording the different actors' perception of the instructional setting. This allowed the researchers not only to show the development of the students' subject-specific learning process (observational normativity), but also to discern how students and teachers experienced this development (observed normativity). Similarly, this also became clear in the SelBer study: While the development of SRL was measured weekly, the coaching sessions and reflection documents were used to capture contextual information about instruction that is often left out in monomethod classroom studies (PRAETORIUS et al., 2021). Based on the two examples, we also raise the question of which challenges arise in mixed methods research on learning and instruction and which implications these have for future research. [44]

First, researchers who wish to apply mixed methods have a wide range of opportunities to do so. In research on learning and instruction, many of these opportunities have not yet been exploited. In particular, researchers in the field of learning and instruction face the challenge of sample acquisition because most studies are grounded on voluntary participation. Furthermore, more complex research designs incorporating a wide range of qualitative and quantitative methods are needed to better account for the multilayered and processual nature of learning and instruction. To date, a predominance of descriptive qualitative methods can be observed (e.g., qualitative content analysis; GLÄSER-ZIKUDA, HAGENAUER & STEPHAN, 2020), while reconstructive and interpretative approaches are used less often in mixed methods studies (SCHREIER, 2017). In the two studies ECOLE and SelBer, a postpositivist view was dominant. This was reflected, for example, in the quasi-experimental design (ECOLE) and the use of longitudinal quantitative methods (SelBer), while relying on semi-structured interviews followed by qualitative content analysis in the qualitative part of the study. By utilizing this approach, the researchers provided a descriptive illustration of social reality, although an in-depth interpretive and reconstructive analysis of social reality was not attempted. [45]

Second, and related to this first aspect, mixed methods as a third paradigm (BERGMAN, 2010; KELLE, 2014) has not yet been widely adopted on a structural level in educational science—at least in the German-speaking countries. In the German context, this is reflected, for example, by the fact that many professorships for qualitative or quantitative methods in educational science exist, but mixed methods professorships are very rare. At the level of the curriculum, mixed methods courses are seldom available, while courses on empirical quantitative and/or qualitative methods are integrated into most of the curricula. In the future, these challenges must be met by systematic training of (young) scientists, for example by offering university courses at the bachelor, graduate and postgraduate level, but also by systematically expanding methods centers (KALKSTEIN & MEY, 2021), which are explicitly related to qualitative, quantitative and mixed methods in empirical school and classroom research. Also, professorships for mixed methods in educational science need to be established. [46]

To conclude, it becomes clear from the arguments about the nature of the phenomenon learning and instruction and the studies presented that for implementing mixed method studies successfully, researchers need to understand quantitative, qualitative, and mixed methods research designs (TEDDLIE & TASHAKKORI, 2009). It is particularly important for researchers conducting applied research on learning and instruction to ensure that scientific knowledge is gained from practical work, while at the same time making sure that the generated knowledge is returned to practice. Research approaches such as action research, participative research or design-based research can be used towards these goals in a variety of ways (ALTRICHTER et al., 2018; PREDIGER et al., 2015). For example, in the SelBer study, it could be shown that the analysis of students' learning processes alone is insufficient for sustainable theory development (KELLE, 2015). Only by collecting and analyzing students' perceptions and descriptions did it become possible to identify the challenges and problems learners experienced as a result of the intervention. Thus, the combination of the two methodological approaches resulted in added value and can also be applied to future interventions in school and classroom research. At the same time, a crucial objective was to understand the implementation of the intervention at the school level. It became clear during the study that not only the joint development process between researchers and school actors was highly relevant, but also the impact of the developed intervention. This shows how fruitful mixed methods research can be for school research, since combining different methodological approaches allowed for a combination of elements from micro-theory (SRL) and macro-theory (offer-and-use model). [47]

1) Although the wording of these research questions is at odds with common style guides regarding anthropomorphic figures of speech in scientific writing, it is common practice in quantitative research. <back>

3) LQ = Long Questionnaire (pre/post measurement), SQ = Short Questionnaire (intermediate measurement), I = Interviews, LC = Learning Coaching, WpSr = Weekly planning/Self-reflection <back>

Altrichter, Herbert; Posch, Peter & Spann, Harald (2018). Lehrerinnen und Lehrer erforschen ihren Unterricht. Bad Heilbrunn: UTB/Klinkhardt.

Bergman, Manfred M. (2010). On concept and paradigms in mixed methods research. Journal of Mixed Methods Research, 4(3), 171-175.

Berliner, David C. (2005). The near impossibility of testing for teacher quality. Journal of Teacher Education, 56(3), 205-213.

Biesta, Gert J.J. & Stengel, Barbara S. (2016). Thinking philosophically about teaching. In Drew H. Gitomer & Courtney A. Bell (Eds.), Handbook of research on teaching (pp.7-67). Washington, DC: American Educational Research Association.

Blömeke, Sigrid (2000). Medienpädagogische Kompetenz. Theoretische und empirische Fundierung eines zentralen Elements der Lehrerausbildung. München: kopaed.

Böder, Tim & Rabenstein, Kerstin (2021). Qualitative Ansätze in der Schulforschung. In Tina Hascher, Till-Sebastian Idel & Werner Helsper (Eds.), Handbuch Schulforschung (pp.1-24). Wiesbaden: Springer VS.

Boekaerts, Monique (1992). The adaptable learning process: Initiating and maintaining behavioral change. Applied Psychology: An International Review, 41(4), 377-397.

Boekaerts, Monique; Pintrich, Paul R. & Zeidner, Moshe (2000). Handbook of self-regulation. San Diego, CA: Academic Press.

Caena, Francesca & Redecker, Christine (2019). Aligning teacher competence frameworks to 21st century challenges: The case for the European digital competence framework for educators (Digcompedu). European Journal of Education, 54(3), 356-369, https://onlinelibrary.wiley.com/doi/epdf/10.1111/ejed.12345 [Accessed: December 12, 2022]

Creswell, John W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Upper Saddle River, NJ: Merrill.

Creswell, John W. & Creswell, J. David (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

Creswell, John W. & Plano Clark, Vicki L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Creswell, John W. & Plano Clark, Vicki L. (2018). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage.

Diriwächter, Rainer; Valsiner, Jaan & Sauck, Christine (2005). Microgenesis in making sense of oneself: Constructive recycling of personality inventory items. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(1), Art. 11, https://doi.org/10.17169/FQS-6.1.520 [Accessed: April 04, 2022].

Durlak, Joseph A. (2016). Programme implementation in social and emotional learning: Basic issues and research findings. Cambridge Journal of Education, 46(3), 333-345.

Erickson, Frederick (1986). Qualitative methods in research on teaching. In Merlin C. Wittrock (Ed.), Handbook of research on teaching (pp.119-161). Washington, DC: American Educational Research Association.

Fend, Helmut (2008). Schule gestalten. Wiesbaden: VS Verlag für Sozialwissenschaften.

Flick, Uwe (2017). Qualitative Sozialforschung. Eine Einführung. Hamburg: Rowohlt.

Fraillon, Julian; Ainley, John; Schulz, Wolfram; Friedman, Tim & Duckworth, Daniel (2020). Preparing for life in a digital world. IEA international computer and information literacy study 2018. International report. Cham: Springer Open, https://link.springer.com/book/10.1007/978-3-030-38781-5 [Accessed: December 11, 2022].

Gläser-Zikuda, Michaela & Fuß, Stefan (2008). Impact of teacher competencies on students' emotion—a multi-method approach. International Journal of Educational Research, 47(2), 136-147.

Gläser-Zikuda, Michaela & Göhring, Anja (2007). Analyse und Förderung selbstregulierten Lernens auf der Grundlage des Portfolio-Ansatzes – ein Forschungsprogramm in der Sekundarstufe I. Empirische Pädagogik, 21(2), 174-208.

Gläser-Zikuda, Michaela & Järvelä, Sanna (2008). Application of qualitative and quantitative methods to enrich understanding of emotional and motivational aspects of learning. International Journal of Educational Research, 47(2), 79-83.

Gläser-Zikuda, Michaela; Hagenauer, Gerda & Stephan, Melanie (2020). The potential of qualitative content analysis for empirical educational research. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 21(1), Art. 17, https://doi.org/10.17169/fqs-21.1.3443 [Accessed: May 02, 2022].

Gläser-Zikuda, Michaela; Fuß, Stefan; Laukenmann, Matthias; Metz, Kerstin & Randler, Christoph (2005). Promoting students' emotions and achievement—instructional design and evaluation of the ECOLE-approach. Learning and Instruction, 15(5), 481-495.

Gläser-Zikuda, Michaela; Seidel, Tina; Rohlfs, Carsten; Gröschner, Alexander & Ziegelbauer, Sascha (2012). Mixed Methods in der empirischen Bildungsforschung. Münster: Waxmann.

Gopalan, Maithreyi; Rosinger, Kelly & Ahn, Jee B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218-243, https://journals.sagepub.com/doi/full/10.3102/0091732X20903302 [Accessed: December 12, 2022].

Guba, Egon G. (1990). The alternative paradigm dialogue. In Egon G. Guba (Ed.), The paradigm dialogue (pp.17-30). Newbury Park, CA: Sage.

Hagenauer, Gerda & Gläser-Zikuda, Michaela (2019). Mixed-Methods. In Marius Harring, Carsten Rohlfs & Michaela Gläser-Zikuda (Eds.), Handbuch Schulpädagogik (pp.801-812). Münster: Waxmann.

Hammersley, Martyn (2008). Paradigm war revived? On the diagnosis of resistance to randomized controlled trials and systematic review in education. International Journal of Research & Method in Education, 31(1), 3-10.

Helmke, Andreas (2015). Unterrichtsqualität und Lehrerprofessionalität. Diagnose, Evaluation und Verbesserung des Unterrichts. Seelze: Klett-Kallmeyer.

Helsper, Werner & Klieme, Eckhard (2013). Quantitative und qualitative Unterrichtsforschung – eine Sondierung. Einführung in den Thementeil. Zeitschrift für Pädagogik, 3, 283-290, https://doi.org/10.25656/01:11938 [Accessed: December 12, 2022].

Hmelo-Silver, Cindy E.; Duncan, Ravit G. & Chinn, Clark A. (2007). Scaffolding and achievement in problem based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99-107.

Hsiao, Cheng (2007). Panel data analysis—advantages and challenges. TEST, 16(1), 1-22.

Järvelä, Sanna & Bannert, Maria (2021). Editorial. Temporal and adaptive process of regulated learning—what can multimodal data tell?. Learning and Instruction, 72, Art. 101268, 1-2.

Johnson, R. Burke (2015). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156-173.

Johnson, R. Burke & Christensen, Larry B. (2019). Educational research: Quantitative, qualitative, and mixed approaches. Thousand Oaks, CA: Sage.

Johnson, R. Burke; Onwuegbuzie, Anthony & Turner, Lisa (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133.

Kalkstein, Fiona & Mey, Günter (2021). Methoden im Zentrum! Methoden ins Zentrum? Potenziale und Grenzen universitärer Methodenzentren für die Erweiterung der qualitativen Methodenausbildung. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 22(2), Art. 26, https://doi.org/10.17169/fqs-22.2.3736 [Accessed: May 02, 2022].

Kelle, Udo (2014). Mixed Methods. In Nina Baur & Jörg Blasius (Eds.), Handbuch Methoden der empirischen Sozialforschung (pp.153-166). Wiesbaden: Springer VS.

Kelle, Udo (2015). Mixed methods and the problems of theory building and theory testing in the social sciences. In Burke Johnson & Sharlene Hesse-Biber (Eds.), Oxford handbook of multi and mixed methods research inquiry (pp.594-605). Oxford: Oxford University Press.

Kelle, Udo (2017). Die Integration qualitativer und quantitativer Forschung – theoretische Grundlagen von "Mixed Methods". Kölner Zeitschrift für Soziologie und Sozialpsychologie, 69(2), 39-61.

Kirschner, Paul A.; Sweller, John & Clark, Richard E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.

Klieme, Eckhard (2019). Unterrichtsqualität. In Marius Harring, Carsten Rohlfs & Michaela Gläser-Zikuda (Eds.), Handbuch Schulpädagogik (pp.393-408). Münster: Waxmann.

Kuckartz, Udo (2014). Mixed Methods. Methodologie, Forschungsdesigns und Analyseverfahren. Wiesbaden: Springer VS.

Kuhn, Thomas S. (2012 [1962]). The structure of scientific revolutions. Chicago, IL: Phoenix.

LeCompte, Margarete & Preissle Goetz, Judith (1982). Problems of reliability and validity in ethnographic research. Review of Educational Research, 52(1), 31-60.

Lincoln, Yvonna S. & Guba, Egon G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Lorenz, Ramona & Endberg, Manuela (2019). Welche professionellen Handlungskompetenzen benötigen Lehrpersonen im Kontext der Digitalisierung in der Schule? Theoretische Diskussion unter Berücksichtigung der Perspektive Lehramtsstudierender. MedienPädagogik, 19, 61-81, https://www.medienpaed.com/article/view/695 [Accessed: December 12, 2022].

Mackenzie, Noella & Knipe, Sally (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193-205, https://www.iier.org.au/iier16/mackenzie.html [Accessed: December 12, 2022].

Mejeh, Mathias & Hagenauer, Gerda (2021). Mixed Methods. In Tina Hascher, Till-Sebastian Idel & Werner Helsper (Eds.), Handbuch Schulforschung (pp.1-20). Wiesbaden: Springer VS.

Mejeh, Mathias & Held, Tanja (2022). Understanding the process of self-regulated learning: An intervention study to promote self-regulated learning in vocational schools. Vocations and Learning, 15(3), 1-38, https://link.springer.com/article/10.1007/s12186-022-09298-4 [Accessed: December 12, 2022]

Mishra, Punya & Koehler, Matthew J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.

Morgan, David L. (2007). Paradigms lost and paradigms regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.

Moss, Pamela A. & Haertel, Edward H. (2016). Engaging methodological pluralism. In Drew H. Gitomer & Courtney A. Bell (Eds.), Handbook of research on teaching (pp.127-247). Washington, DC: American Educational Research Association.

OECD (2021). 21st-century readers. Developing literacy skills in a digital world. Paris: PISA, https://doi.org/10.1787/a83d84cb-en [Accessed: August 4, 2021].

Onwuegbuzie, Anthony J. & Leech, Nancy L. (2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8(5), 375-387.

Praetorius, Anna-Katharina; Martens, Matthias & Brinkmann, Malte (2021). Unterrichtsqualität aus Sicht der quantitativen und qualitativen Unterrichtsforschung: Methodische Ansätze, zentrale Ergebnisse und kritische Reflexion. In Tina Hascher, Till-Sebastian Idel & Werner Helsper (Eds.), Handbuch Schulforschung (pp.1-20). Wiesbaden: Springer VS.

Prediger, Susanne; Gravemeijer, Koeno & Confrey, Jere (2015). Design based research with a focus on learning processes—an overview on achievements and challenges. ZDM Mathematics Education, 47(6), 877-891.

Reusser, Kurt (2018). Allgemeine Didaktik – quo vadis?. Beiträge zur Lehrerinnen- und Lehrerbildung, 36(3), 311-328.

Schreier, Margrit (2017). Kontexte qualitativer Sozialforschung: Arts based research, mixed methods und emergent methods. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 18(2), Art. 6, https://doi.org/10.17169/fqs-18.2.2815 [Accessed: May 02, 2022].

Schunk, Dale H. & Greene, Jeffrey A. (Eds.) (2018). Handbook of self-regulation of learning and performance. New York, NY: Routledge.

Seidel, Tina (2014). Angebots-Nutzungs-Modelle in der Unterrichtspsychologie. Integration von Struktur- und Prozessparadigma. Zeitschrift für Pädagogik, 60, 850-866, https://doi.org/10.25656/01:14686 [Accessed: December 12, 2022].

Seidel, Tina & Shavelson, Richard J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77(4), 454-499.

Tashakkori, Abbas & Teddlie, Charles (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In Abbas Tashakorri & Charles Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp.3-50). Thousand Oaks, CA: Sage.

Teddlie, Charles & Tashakkori, Abbas (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Los Angeles, CA: Sage.

Volet, Simone & Vauras, Marja (Eds.) (2013). Interpersonal regulation of learning and motivation. Methodological advances. Milton Park, Abingdon: Routledge.

Winne, Philip H. (2019). Paradigmatic dimensions of instrumentation and analytic methods in research on self-regulated learning. Computers in Human Behavior, 96, 285-290.

Winne, Philip H. & Hadwin, Allyson F. (1998). Studying as self-regulated learning. In Douglas J. Hacker, John Dunlosky & Arthur C. Graesser (Eds.), Metacognition in educational theory and practice (pp.277-304). Mahwah, NJ: Erlbaum.

Zimmerman, Barry J. (2000). Attaining self-regulation: A social cognitive perspective. In Monique Boekaerts, Paul R. Pintrich & Moshe Zeidner (Eds.), Handbook of self-regulation (pp.13-39). San Diego, CA: Academic Press.

Zimmerman, Barry J. & Schunk, Dale H. (Eds.) (2011). Handbook of self-regulation of learning and performance. New York, NY: Routledge.

Mathias MEJEH is senior researcher at the Institute for Educational Science at the University of Bern. His main research interests are self- and co-regulated learning, inclusive education and social network analysis. In this regard, he is also interested in the combination of qualitative and quantitative research approaches within the framework of mixed methods research.

University of Bern
Institute of Educational Sciences
Department of School and Teaching Research
Fabrikstrasse 8
CH-3012 Bern, Switzerland

Gerda HAGENAUER is a full professor in the School of Education and the Department of Educational Science at the University of Salzburg. Her main research interests are emotions, motivation and social relationships in teaching and learning at school, in teacher education and in higher education. She is also interested in mixed methods research.

University of Salzburg
School of Education and Department of Educational Science
Erzabt-Klotz-Straße 1
A-5020 Salzburg, Austria

Michaela GLÄSER-ZIKUDA is a full professor and chair of the Research and Teaching Unit School Education and Instructional Research at the University of Erlangen-Nürnberg. Her main research interests are emotions and self-regulation in school and higher education, instructional quality, and teacher and higher education. She is specifically interested in qualitative research methods and mixed methods research.

University of Erlangen-Nürnberg
Faculty of Humanities, Social Sciences, and Theology
Institute of Educational Science, School Education and Instructional Research
Regensburger Straße 160
D-90478 Nürnberg, Germany

Mejeh, Mathias; Hagenauer, Gerda & Gläser-Zikuda, Michaela (2023). Mixed methods research on learning and instruction—meeting the challenges of multiple perspectives and levels within a complex field [47 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 24(1), Art. 14, http://dx.doi.org/10.17169/fqs-24.1.3989.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License