Fifth Annual Meeting of Qualitative Psychology: Qualitative and Quantitative Approaches to Learning and Instruction. First meeting of the Special Interest Group No. 17 of the European Association for Research in Learning and Instruction (EARLI): Mixed Methodology in Psychological Research. Freudenstadt, Germany, October, 21-24, 2004, Organized by Leo Gürtler, Günter L. Huber & Mechthild Kiegelmann (Center for Qualitative Psychology, Tübingen, Germany)

Abstract: This conference report gives an overview of the fifth conference of the Qualitative Psychology Initiative and the first meeting of the European Association for Research in Learning and Instruction (EARLI) interest group (No. 17), that took place in Freudenstadt, Germany from 21-24 October 2004. The conference was organized by the Center for Qualitative Psychology (Tübingen). This year the main focus of the conference, which was attended by researchers from a wide spectrum of professions, was mixed methods as a research strategy in psychology. The main issue under discussion was whether a new paradigm is needed to resolve the contradiction between qualitative and quantitative approaches to doing research. This report attempts to give a résumé of the individual contributions and the conference as a whole, to put the workshop in context, and to provide a view of the trends in qualitative research in the field of psychology.

Key words: qualitative research, qualitative psychology, research design, methodology, mixed methods, conference, workshop, methods, psychology, networking

The Center for Qualitative Psychology (CQP) has been organizing an annual international meeting on qualitative psychology since the year 2000. The aims and mission statement of the CQP are published on their website and also presented in KIEGELMANN, HELD, ERTEL, and HUBER (2000). This conference report provides an overview of the fifth conference and the first meeting of the Special Interest Group No. 17, Qualitative and Quantitative Approaches to Learning and Instruction of the European Association for Research in Learning and Instruction (EARLI), held in Freudenstadt, Germany from 21-24 October 2004. Since many of the topics in the various sections of the conference overlapped, a joint conference program was compiled. [1]

For many years, researchers from various professions, from younger scientists to more established representatives of qualitative research approaches in psychology, have participated in the annual meeting on qualitative research approaches in psychology. The meeting is international and this year included participants and presenters from Austria, Switzerland, the Netherlands, the USA, Australia, South Korea, Spain, and Finland. In keynote lectures and plenary sessions all participants came together to discuss the meeting's topic in detail. Specific work and results were presented and discussed in the working groups. [2]

The 2004 conference focused on "mixed methods" as a research strategy in psychology. The main issue was whether a new paradigm is needed to resolve the gap between qualitative and quantitative approaches. This report gives a résumé of the individual contributions and the conference as a whole, while putting the workshop in context and providing a view of the trends in qualitative research in the field of psychology. [3]

The conference adopted a structure that had been successful at previous meetings, which gave the working groups more time for discussion and more detailed dialogs on their issues. For the timetable and the compositions of the working groups at the 2004 conference see Appendix A. [4]

All participants' contributions addressed their thoughts and arguments concerning their selection of research methods and the organization of their research designs. These decisions are often constrained by research questions, the circumstances in the field of research, and sometimes by limited funds and resources. However, from our point of view, decisions should not be restricted by a failure to consider the epistemological and theoretical implications involved in the use of different methods of psychological research. [5]

This report will briefly summarize the various topics that were presented and aims to follow the conference as it unfolded, beginning with the topic of the keynote address, triangulation. Following a summary of the keynote address the individual presentations will be briefly outlined, emphasizing the central lines of discussion. For more detailed information on the various contributions please see the more comprehensive descriptions in Appendix B at the end of the report. [6]

One of the main questions that arose during the conference was: is a new paradigm needed to resolve the contradictions between qualitative and quantitative approaches? [7]

As guest speaker, Uwe FLICK gave an introduction to the subject of "Triangulation as a framework for mixed methods research in psychology." He began with DENZIN's (1978) definition of triangulation. According to DENZIN, triangulation refers to the use of different kinds of data and/or different theoretical and methodological perspectives. This can be done using different methods and also within one method. Moving on to the issue of choosing between different qualitative and quantitative approaches, FLICK pointed out four modes of dealing with different paradigms: (1) strict rejection of "the other" approach, (2) domination or subordination of one or the other approach, (3) mixing methods, and (4) triangulation. For FLICK triangulation is more than simply mixing methods. It can begin at the design stage, consist in the use of different methods, and may involve linking results or assessing the quality of research. In each case, triangulation gives equal weight to the various perspectives, thus reflecting and integrating the underlying approaches. Triangulation thus always addresses different levels, i.e. the level of theory or methods, alone or together. It can therefore lead to reliable results, differing results and contradictory results, all of which can be reflected upon productively. FLICK demonstrated this with an example about trust in helping relationships, using different methods to throw light on it from both a subjective and an institutional point of view. He emphasized the importance of choosing and using methods purposefully. He concluded by showing a checklist for selecting research methods in a purposeful manner. The issue of whether triangulation should be defined as a new paradigm remains an open question. FLICK's view is that the flexibility of qualitative research would suffer if a new paradigm with rules about how triangulation should work were to be built up and that this would restrict us to a specific use of triangulation (see for more information FLICK 2004). [8]

The first work group with Siegfried HOPPE-GRAFF, Nicole LAMM-HANEL and Hye-On KIM investigated the topic of the potentialities and limitations of mixing methods in research with diaries and questionnaires. The research team investigated M.L. HOFFMAN's theory (1975) of parental influence on moral internalization. One problem of this type of research seems to lie in the criteria of quality, such as objectivity of observations or generalization of results, both of which are important aspects of research that have to be considered. Anyway, they often mean something different according to the research design, and they are not equally important each time. To give an example, instead of using objectivity in qualitative research, inter-subjectivity and transparency are better choices most of the time. [9]

The second work group centered on mixed methods in psychological studies, began with a presentation on "Coherence in knowledge communication: How do on-line groups communicate?" by Karin SCHWEIZER, Manuela PÄCHTER and Bernd WEIDENMANN. The study investigated collaborative learning in the context of a virtual seminar and analyzed how learners collaborate on different types of group tasks that used mainly a jigsaw puzzle approach. The combination of qualitative and quantitative approaches seemed to be very useful for investigating communication and collaboration in the virtual seminar. In the following presentation, Anne A. HUBER showed how she gained insight into to the issue of how to add qualitative insight to quantitative findings in a study on cooperative learning. She combined achievement tests, qualitative data, and questionnaires. Achievement tests were performed after each learning session over a period of several weeks. The qualitative data were used to better understand students' reflections of this special learning period. Danielle VERSTEGEN did research on instructional design focusing specifically on the role of iteration in an empirical study on the specification of training simulators. She used both subjective and objective sources of data. The results offered greater understanding of the problem, but the question remained as to when and how iteration is desirable. [10]

The third group worked on the subject of variations in mixed methods approaches and concentrated from the start on the question "Is a new paradigm necessary to resolve the contradiction between qualitative and quantitative approaches?" In his paper entitled "Six alternatives to mixed methods in qualitative research" Gary SHANK began by addressing the problem that qualitative and quantitative methods are at cross-purposes. His main question remained: How can we simultaneously resolve issues of meaning so as to test hypotheses on one hand, while at the same time trying to keep issues of meaning open and indeterminate? He concluded that qualitative researchers are best served by resisting the temptation to pursue mixed methods approaches. Instead, he suggested concentrating on alternatives within the domain of qualitative research. In the next presentation, Philipp MAYRING offered an opposing view and presented some ways of mixing qualitative and quantitative methods in theory and practice. He argued that it is possible to gain a deeper understanding for promoting reflective discourses in research from a dialectical position and suggested that this could serve as a paradigm for mixing methods. As an integrated model for practical research, qualitative content analysis allows the collection of qualitative data, both deductive and inductive coding of the data and qualitative as well as quantitative data analysis. The third presentation by Gillian M. BOULTON-LEWIS and Lynn A. WILSS reported on a study in which they had aimed to maximize the analysis of interview data. The goal of the survey arose by initially using phenomenography to understand the conceptions of formal learning of Aboriginal university students. This led them to employ an interpretive-descriptive approach to analyze learning in that context. They concluded that the congruencies and dissonances they found could only be detected if the different methods were combined. [11]

The fourth work group focused on the combination of qualitative methods. The session commenced with a presentation on research and intervention by interviews and learning diaries in in-service teacher training designed by Günter L. HUBER and Jürgen W.H. ROTH. These authors used learning diaries and interviews and concluded that the diaries could be used for comparison with the findings of the interviews and that the combination helps to access valuable information which can be used to plan intervention strategies for implementation in the professional development curriculum. Hannu SOINI and Tuulikki TUOMINEN-EILOLA investigated peer group consultations in studying experience. The presenters pointed out that the rhythm of peer group consultations consists of a constant change between withholding and releasing in the group. The combination of methods is mirrored in the structure of the peer group consultations and leads to several gains. In these authors' opinion, peer group consultations allow multiple positions, multiple perspectives and multiple choices for conceptualizing. [12]

Work group five concentrated on combinations of qualitative and quantitative methods beginning with a presentation by Nicole TORKA. In her paper on "Mixed methods: Challenging work and organizational psychology—the case of employee commitment" TORKA presented a combination study from research on commitment using short, informal, semi-structured interviews and observations to investigate the meaning of commitment from a blue-collar worker's perspective. The questionnaire supported the outcome of the qualitative explorative study and also supported the qualitative results. TORKA also pointed out that the "walking and talking on the floor" method shed new light on classical concepts of commitment. Following TORKA's presentation Antonio Medina RIVILLA and Ma. Concepción Domínguez GARRIDO reported on an investigation into the subject of "Teacher education for interculturality: complementarity of questionnaires and group discussions," in which they had tried to find a balance between "objectivity" and "subjectivity." The final synthesis of their analysis produced a series of ideas about culture and teaching. [13]

Work groups 6 and 7 worked on the issue of access to individual experience by mixed methods. Sabine K. LEHMANN-GRUBE and Katrin SOMMER presented their paper on the "Content analysis of task-solutions as a means to generate achievement scores." They investigated teaching students' capability to relate everyday techniques to scientific and technical concepts. A pretest and posttest were carried out and the data were analyzed by both qualitative and quantitative methods. The main results revealed that samples differ according to level of terminology and that the required knowledge has a smaller effect on the level of terminology. Michaela GLÄSER-ZIKUDA from Germany contributed research about the interaction between emotions and learning in elementary school classrooms and the specific contribution of qualitative-quantitative methods. The design covered a multi-modal approach of semi-structured interviews, diary logs, and video observations to investigate relevant emotions. GLÄSER-ZIKUDA noted that multivariate analyses, single cases, process surveys, and mutual validation of different data were fruitful approaches for gaining insight into what students actually feel in learning settings. [14]

Leo GÜRTLER explored how students experience humor in the classroom. Subjective theories (questionnaires with open-ended questions) were investigated. The responses obtained were analyzed for prototypes by formulating and testing sequential hypotheses. The four investigated groups (school type over sex) differed not only for word count (quantitative information), but by using qualitative sequence hypotheses as discriminators, it was possible to classify the response, i.e. the grouping factor (group membership), better than chance. This was tested via a permutation test (linear discriminant analysis). In her study, KIEGELMANN reflected on the psychology of breaking silence and how individuals can be supported to resist being silenced. The aim was to compare qualitative and quantitative results. She used a paper and pencil questionnaire with 150 participants and interviews on responses to social expectations and silencing in respect to silence about an affair within a family or neighborhood. The results of both the qualitative and quantitative analyses indicate that there is a difference between breaking silence in neighborhoods and doing so in families. [15]

Work groups 8 and 9 attempted to gain a deeper understanding of findings obtained by mixing methods. Jeannette BISCHKOPF and her colleagues combined qualitative and quantitative research strategies to analyze the experience of aversive tension in different patient groups. The study followed the combination model of a mixed methodology approach, using the sequence model in which a qualitative approach is applied in a pre-study, and further quantitative analyses building on the qualitative results are used later. Magdalena MAJOREK from Switzerland investigated sleep disorders and nightmares in children with ADHD (Attention Deficit Hyperactivity Disorder). The aim of the study was to explore sleeping patterns and the running themes of dreams of children with ADHD. An integrated approach to collecting data including questionnaires, reports, and guided interviews was employed. The data were analyzed using functional analysis, triangulation, and created Grounded Theory. [16]

Silke-Birgitta GAHLEITNER from Germany presented an ongoing study in the field of trauma research entitled "Following the process—using mixed methods to capture clients' experiences of change in therapy and counseling." In contrast to quantitatively oriented mainstream psychotherapy research, her study focused on clients' subjective experiences. Data collection includes semi-structured interviews and quantitative questionnaires. GAHLEITNER put up for discussion a number of problems that can arise at the interface between qualitative and quantitative approaches. Regula ZUEGER-CÁCERES from Switzerland explored participatory development projects and their contribution to poverty alleviation. She used qualitative methods and factor analysis. ZUEGER-CÁCERES interpreted the emerging factors as operants with functional segmentations. They act as representatives of the inner realities of the people who were investigated. [17]

Carmen RICOY LORENZO, Ramón PÉREZ PÉREZ and Tiberio Feliz MURIAS from Spain investigated the idea of the press as an educational resource in adult education. Their research objectives were to determine what kind of didactic resources are available in adult education. The methods they employed were descriptive research combined with case studies. The questionnaire, interviews, and document analysis produced various results. It might be expected that using the press would offer advantages. However, in what ways the press itself is willing to support adult education and whether joint activities with the press are possible remains to be investigated. As the last contribution, Leo GÜRTLER and Günter L. HUBER from Germany gave a very vivid introduction to practical aspects of mixed methods with AQUAD 6 in a presentation entitled "Combining software for qualitative and quantitative analysis." [18]

The chief aim of the conference was to explore "mixed methods" as a research strategy in psychological research. Both the keynote speaker and many of the presentations led to a broader and deeper understanding of the possibilities afforded by combining different research methods and strategies and making them accessible within one survey. The various presentations generated many interesting ideas and a wealth of material for discussion, resulting in new questions for further work. As mentioned above, one of the critical questions was: Is it necessary to establish a new paradigm or is it enough just to use mixed methods as required? [19]

No consensus was reached on the issue of whether it is necessary or expedient to develop a new paradigm to overcome the distinctions between qualitative and quantitative approaches. In the course of a lengthy discussion in which many important subjects were touched upon, it became evident that the questions at issue could not be answered without addressing the deeper epistemological underpinnings of the different approaches. This applies both to the theoretical epistemological level and to the teaching of research methods at universities. The question as to the need for a new paradigm evolved into a question regarding the tension between the need for structures to provide a firm basis for purposeful procedures on the one hand, and the need for flexibility on the other. At the same time, the freedom required in order to be able to adapt methods to both the process and the subject of research in the individual case cannot be optimally exploited without structures and rules as guidelines for procedure. [20]

It was agreed that a paradigm might impose too narrow a definition and fail to leave sufficient room for creative triangulations and combinations of methods, whereas the question as to the suitability and formulation of "rules" remained open. Here a consensus was reached that the dialectical aspect, on which most participants were agreed, requires further consideration and exploration. Whether that implies that one should follow SHANK and tend towards multiperspectivity within qualitative or quantitative, or, as MAYRING proposed at the meeting, tend rather towards a dialectical investigation along the lines of the individual inquiry and with the resources available while paying attention to the epistemological basic orientation, or as FLICK proposes, to make a purposeful choice and use of methods—possibly with the help of his checklist—is another question. The clarification of this issue remains a work in progress. [21]

Furthermore, this question leads to the overarching question of whether quantitative and qualitative methodologies (or research strategies) are opposites and in conflict or—as many presenters implicitly suggested and we also propose—that qualitative and quantitative are complementary. It would seem fruitful to consider the idea that they are complementary, since qualitative differences do exist between qualitative and quantitative methodologies and these can be reasonably attributed to their being complementary. On the other hand it makes less sense to compare qualitative and quantitative with the objective of finding the superior method than it does to identify what fits best for this sample or with this research question. [22]

In contrast, using the two paradigms as complementary always provides an opportunity to compare results and thus allows the emergence of new perspectives. In fact, both qualitative and quantitative do overlap to a great extent with other methodologies. It is not possible to interpret statistical analyses without utilizing at least some qualitative interpretation. The same applies for qualitative findings. Many textbooks on qualitative content analysis refer to quantitative methods of handling data (e.g. to determine the congruence of coding with KAPPA or RAND indices). In other literature, it is more difficult to find the implicit quantitative assumptions behind qualitative discussions. This problem alone merits an article. At times the distinction between qualitative and quantitative seems to be arbitrary and more a linguistic artifact than a real and clear objective differentiation. [23]

However, as mentioned above, this requires a broader argument based on the use of language. This touches theory of science, the formulation of hypotheses, and theory building. Until now, the use of language is seldom explored in terms of its implications for research decision making and subsequent actions. The roots of the discussion are of an epistemological nature and not simply a question of applying techniques from one of the paradigms. An example can be demonstrated by means of the measurement model of GIGERENZER (1981). The model deals with the interactions between methods, subject matter, and research subject. It is cited mainly in quantitative literature, if at all. However, it can easily be adapted to qualitative as long as the numeric system is not essential but can be fully replaced by a semantic system that works solely on the level of words and their interpretations, i.e. qualitative methodology. The elements of the model as cited by GIGERENZER (1981) are: (1) representativeness, (2) unambiguousness, and (3) remarkableness (for content). The first element is independent of qualitative or quantitative as it addresses the question as to whether a research method is appropriate to catch the phenomenon at all. The second element points to the fact that if something is investigated, it should be clear that the object that is proposed for survey is actually surveyed. The third element can then be divided into statistical significance (numeric system) and semantic significance (proposed new semantic system). Thus, this very short example should demonstrate that for all practical purposes, the distinction between the paradigms is not as clear as it often is postulated. However, it shows how qualitative and quantitative overlap substantially, without postulating any kind of identity. [24]

However, this means that this discussion should not be carried on at conferences dedicated to the combination alone, but also in actual research contexts, in university teaching settings, and within areas that initially seem to deprecate the other method. The question of paradigms leads to the question of how the requirement for certain underlying structures for doing "good" research and beliefs about how research can be done purposefully correspond with each other. This ambivalence is strongly associated with the need for structure, the existence of clear structures ("cooking recipes," if indeed they do exist at all!) on how to investigate social contexts. But this does not mean that arbitrariness is meaningful. The liberty, i.e. the variability of realizing one's own investigation with varying methods according to needs, wants, and resources, is not limited. On the other hand, the absence of limits does not mean that "anything goes." On the contrary, every research action has to be reasoned. However, some kinds of rules are necessary if the phenomenon is to be caught sufficiently. The question of the validity of research is still relevant today, and the question of rules has to be left open at present. A good starting point still seems to be to take a look at the processes and usage of a dialectical positioning of qualitative and quantitative methodologies. [25]

The next conference of the Center for Qualitative Psychology will take place in Velden on the shores of Wörthersee, near Klagenfurt (Austria) from October 21–23, 2005. The main topic of the workshop will be Generalization in Qualitative Psychology. Further information is available on the center's website. [26]

Meeting of EARLI-SIG #17 Qualitative and Quantitative Approaches to Learning and Instruction and the Workshop Mixed Methods in Psychological Research

Diaries and Questionnaires: Potentialities and Limitations of Mixing Methods (Siegfried HOPPE-GRAFF, Germany, Nicole LAMM-HANEL & Hye-On KIM, South Korea)

HOPPE-GRAFF et al. investigated M.L. HOFFMAN's theory (1975) of parental influence on moral internalization with diaries and questionnaires. One of the basic propositions of HOFFMAN's theory is that parents' responses to transgression can be classified as predominantly power assertive, love withdrawing, or inductive. Transgressions lead to parental activities that aim to change the behavior of the child. The goal of the study was to check HOFFMAN's conclusions, which were based on parents' beliefs and memories. The empirical observations combined quantitative and qualitative approaches. The latter were operationalized qualitatively by collecting diary entries on, e.g., mother-child interactions in disciplinary encounters and important incidents in family life. In addition, parents' beliefs were assessed by using a questionnaire with closed and open items (quantitative approach). HOPPE-GRAFF pointed out that one problem of this type of research is how to define criteria of quality, such as objectivity of observations or generalization of results, which are important for the two paradigms although they have different meanings in their respective fields. To realize that aim, he proposed to test the mothers' diary entries for objectivity by identifying points of agreement between different observers. Case studies (e.g., by combining diary entries with video records) enhanced the understanding of the results, but only by establishing a special relationship with the observed families that enabled an appropriate interpretation of them. One of the main results of LAMM-HANEL's (2003) work points out the necessity of a modification of HOFFMAN's theory. She was able to propose a new classification system of parents' responses. But more interestingly, mothers' beliefs and their actions do not largely correspond with each other. On a group level beliefs can be generalized, but not on an individual level. In the end, diary records are not subjective because they can be classified by different observers (~ 80% agreement).

Coherence in Knowledge Communication: How Do On-line Groups Communicate? (Karin SCHWEIZER, Germany, Manuela PAECHTER, Austria & Bernd WEIDENMANN, Germany)

The study investigated collaborative learning in the context of a virtual seminar. It analyzed how learners collaborate on different types of group tasks, mainly using a jigsaw puzzle approach. Learning and working in a group is a complex process in which different cognitive processes have to be performed. These cognitive processes are expressed in part by the group members' joint actions and their discourse. One of the aims of the study was to investigate how group members shape their discourse and construct meaning in different communication environments.

Qualitative methods were employed to analyze the group members' discourse. The discourse was investigated within a framework of problem solving. Moreover, the groups' performance was also analyzed by content analysis. In order to compare the different communication settings it was necessary to use a quantitative approach. A similar procedure was applied for the analysis of group performance. The number of arguments in an essay was compared to the total amount of possible arguments. Then the percentage of arguments in all essays compared to all possible arguments was calculated. These quantitative data could be used for inferential statistics in order to compare the discourse behavior in various communication settings. This combination was very useful for the investigation of communication and collaboration in the virtual seminar.

The results indicate that the characteristics of discourse in on-line groups differ from those in face-to-face groups. Chat groups produce much less turns than face-to-face groups and video conference groups. Yet, the proportion of coordinated turns in relation to elaborations and other content-related contributions reached sometimes more than 80%. Taking both results into account, it is obvious that members of chat groups need to spend much more effort to coordinate their discourse by verbal means than members of other communication outlets.

How to Add Qualitative Depth to Quantitative Findings in a Study on Cooperative Learning (Anne A. HUBER, Germany)

ANNE HUBER described a project on cooperative learning and how qualitative research is a valuable way to enhance and to deepen quantitative findings. In the focus of her research stands the so called "WELL-method" ("Wechselseitiges Lehren und Lernen"—"Common Teaching and Learning"; HUBER 2004), a cooperative learning design based on alternating the roles of teaching and learning on a peer to peer level. WELL is a special form of cooperative learning with the characteristics of (1) task specialization (learners become experts for a part of the subject matter), (2) strategy instruction on how to learn and how to process the material, and (3) three learning phases (becoming experts, teaching each other, and supporting deep processing). The approach has its roots in the jigsaw puzzle, which is extended here to the "Partnerpuzzle" (jigsaw puzzle), in which students become experts for different learning domains. This ensures that they are able to teach each other later on. This strategy fosters deep processing as well as collaboration. It can be combined with various learning methods, such as structural-lay techniques. In this sense, the Partnerpuzzle can be regarded as a WELL method.

The 2 x 2 design of her study consisted of class level (two grades: 7th and 8th) and a Partnerpuzzle with or without strategy instructions. The study was undertaken in biology lessons in a secondary school. Consequently, her research question focused on the roles of strategy instruction and class level, and their respective influence on achievement, intrinsic motivation, and perceived competence.

Following her research questions, HUBER gave an overview on the theoretical background information on class level (that determines the cognitive ability of the learners) and strategy instructions (the cognitive-elaborative perspective). Then, she explained her hypotheses.

The procedures were teaching, questionnaires, and tests on the subject after each learning session over a period of several weeks. For measurement, various scales were chosen for cognitive ability, achievement, intrinsic motivation, and perceived competence. Qualitative data were collected via incidental observations in the classes and by open questions afterwards in the last week. This was used to support reflection of students on "what was good or bad about learning." The quantitative parts of the data were analyzed in a 2 x 2 ANOVA.

Results: Class level had a positive effect on cognitive ability for all cases (five units). Class level had a positive effect on achievement in two thirds of the cases. Thus, both factors play important roles in learning and achievement. Strategy instructions also play an essential role in achievement. This was true for delayed class tests in all cases and less obvious for immediate short tests. The impact of strategy instructions on intrinsic motivation was found to be stronger in the 8th than in the 7th grade (interaction effect). The same goes for perceived competence with slight alterations (not an interaction effect in all cases).

Qualitative findings suggest that especially "8th grade students without strategy instructions had a difficult time, e.g., staying on task or on time. Coded answers give evidence that these students did not like the many tests and they did not mention at all that the method helped them. Due to this combination of methods, quantitative findings can be understood much better by combining multiple sources of data.

Iteration in Instructional Design: An Empirical Study on the Specification of Training Simulators (Danielle VERSTEGEN, The Netherlands)

Danielle VERSTEGEN did research on instructional design with special focus on the role of iteration. Starting with a short history and situation of the topic in the research field, she gave a simplified view of instructional design: Analysis–Design–Production. In practice however, tasks are much more difficult. Information can be incomplete and insecure, designs can be ill-structured, and consequences cannot be foreseen. Furthermore, different parties are involved that sometimes give contradictory messages. To summarize, she emphasized that instructional design should not only be improved but also regarded as an iterative process. Her research question therefore was oriented toward the support of instructional designers to foster the effectiveness of the forthcoming products. Her research domain was the specification of training simulators.

Exploring the literature, VERSTEGEN found that all authors confirm the iterative characteristics, although there is a lack of systematic research on this issue. Prescriptive models, but also descriptive ones, do not provide enough information or clear guidelines on how to do justice to the iterative characteristics. Data on the descriptive level come from interviews, think-aloud studies, and case studies with experts dealing with iteration. Requirements are, as VERSTEGEN stated, e.g. reality (case, time-span), representative subjects, and authentic setting.

VERSTEGEN presented results from different studies. The first one tried to collect information for the enhancement of the optimization process of instructional design. Both subjective and objective data sources were used. These consisted of questionnaires, notes, and log files including information on the number of iterations. Further possibilities can be found in using experts, logs of peer discussion forums, feedback, video analyses of meetings, and presentations. However, these data have some limitations. One example is log files. It is unclear if log-on time equals the time on task or whether the opening of help windows really means that information is read. Thus, there is only a limited control over events that can lead to iteration.

The second study focused more on controlled settings and on planning interventions to evoke iteration. Peer reviews, contact with experts, discussion groups via chat, and new information were provided. The types of data were the same as in the first study.

The third study narrowed the focus to single cases (N = 5) that differed in size and complexity. Workshops were also held with stakeholders and facilitators. Observations and comments were less formal. Systematic instructional design models were applied to help to structure and to manage iteration and associated processes.

To conclude, iteration seemed to be unavoidable, but there also seemed to be no relationship between quality and number of iterations. However, great variances could be found regarding the number of iterations between subjects and cases. One may achieve a facilitation of the whole process by supporting especially the management of iteration, planning decision points and reviews (break points), and comparing alternatives with each other. Managing also points to the need to focus on strategy to recognize and to identify possible triggers for iteration. VERSTEGEN ended her presentation by looking at future research questions such as when and how iteration is desirable.

SHANK addressed the problem that qualitative and quantitative methods are at cross-purposes to each other and raised the question, how can we simultaneously resolve issues of meaning so as to test hypotheses on one hand, and at the same time try to keep issues of meaning open and indeterminate in order to allow a qualitative inquiry process to unfold? Quantitative research strives to test truth claims in order to determine their plausibility. Under the best of circumstances, these truth claims can be stated as hypotheses. When we have hypotheses, or are striving to get toward hypotheses, there are several conditions that have to be met. The most important condition, for our purposes, is that of meaning. Otherwise, we might find ourselves in the situation of not knowing how to interpret our findings. In qualitative research, we must avoid pre-determining meaning at all costs. Qualitative research enhances our understandings and insights into a situation or phenomenon, and these conditions are grounded in meaning. Therefore, if we try to mix qualitative and quantitative methods, we are essentially combining oil and water. SHANK points out that qualitative researchers are best served, at least for now, by foregoing the temptation to pursue mixed methods approaches. He suggests concentrating instead on the basic reasons why we might want to explore a mixed methods approach in the first place, and see if there are alternatives within the domain of qualitative research proper to address those reasons. In his complex paper he gives six concrete examples of qualitative alternative research strategies for three reasons: the enhancement reason, the grounding reason, and the discovery reason.

Maximizing Data Use: Mixed Qualitative Methods (Gillian M. BOULTON-LEWIS & Lynn A. WILSS, Australia)

BOULTON-LEWIS and WILSS presented a study with the aim to maximize the analysis of interview data. The goal of the survey arose by initially using phenomenography to understand the conceptions of formal learning of Aboriginal university students. This led to an interpretive-descriptive approach to analyze learning in that context. BOULTON-LEWIS cited PATTON (1991) that both methods used are seen as representatives of qualitative research, concretely coming from phenomenology, with the aim to foster understanding. She went on to compare both methods for their methodological roots, advances, and limitations. In her research example, phenomenography was used to describe conceptions of formal learning whereas the constant comparison method was applied to investigate reasons for learning, informal learning, and strategies for formal learning. As for results, students could be allocated to the categories that were most typical for them. Conclusions could be drawn especially about the relationships between the four different research parts of formal learning, reasons, informal learning, and strategies.

Results showed that for conceptions of formal learning: (1) acquiring knowledge, (2) understanding, and (3) personal growth are important. Reasons for learning are (1) paying for study, (2) indigenous issues (improving conditions, being a model, going back and helping), and (3) personal development. For hierarchical strategies that could be identified, the following can be named: (1) Focusing and rehearsal, (2) organization and memory strategies, and (3) elaboration and monitoring. In general, it was striking that half the sample reported using strategies that did not match their own conception of formal learning. BOULTON-LEWIS concluded that these congruences and dissonances were only to be found by combining the different methods.

In his contribution MAYRING began by introducing four conflicting paradigms in psychology: natural science versus human science, the positivism debate, the constructivism debate and the qualitative turn. He outlined three positions that have developed in the attempt to overcome the controversies about these paradigms: (1) the pragmatic position; (2) the solution oriented, dialectical, process oriented position; (3) the deconstructivistic position (allowing multiple solutions). MAYRING argued that the dialectical position is the most promising in terms of gaining a deeper understanding and for promoting reflective discourses in research and suggested that in one sense it could serve as a paradigm. In the mixed methodology approaches we can distinguish between combination models (e.g. sequence models) and integration models (e.g. triangulation). As an integrated model, qualitative content analysis (see MAYRING 2000a, 2000b) allows collection of qualitative data, both deductive and inductive coding of the data and qualitative as well as quantitative data analysis. As an example of a further combination model he presented a study on "coolness." In the first explorative part, qualitative data were collected. These were then analyzed by grounded theory, developing a model of "coolness." In a second step quantitative data were collected, which were used to test the model by LISREL.

Research and Intervention by Interviews and Learning Diaries in In-Service Teacher Training (Günter L. HUBER & Jürgen W.H. ROTH, Germany)

HUBER and ROTH explored in-service teacher training in Germany. In-service training is divided into two phases: (1) university studies, state examination, and (2) learning to teach, called the "Referendariat"; mentoring; and attending a teacher seminar. Tandem projects try to establish peer-to-peer intervention and collaboration via dyads. Their goals are active learning and implementation of cooperation, e.g., by mutual observation and informal discussions. The discussions are summarized in a learning diary, the topic of the survey. Seminars are structured for exchange of experiences, identification of open questions (e.g., "which support is available from whom?"), and discussion. The learning diary contains two parts: private and public. Additionally, self-reflection and external-reflection are parts of the diary. As another access to the field, interviews were conducted at the beginning of the training and six months later. They focused on students' learning processes as teachers and were divided into six topics (e.g., knowledge of effects).

Data were analyzed with AQUAD 6. In comparing cases, the authors presented examples of two students who differed in tolerance of ambiguity and need of structure. Further comparisons on case level were students' own learning experiences as well as their evaluation of their own learning. HUBER and ROTH concluded that the students' self-descriptions (in the interviews) seem to organize their learning experiences according to their personality styles, i.e., tolerance of ambiguity and need of structure. Starting from these first results, the diaries can be used to control the findings of the interviews. The presenters showed diary entries to demonstrate this combination of different perspectives.

Finally, it was remarked that this combination gives access to valuable information for the planning of intervention strategies for implementation in the professional development curriculum.

Peer Group Consultation in Studying Experience (Hannu SOINI & Tuulikki TUOMINEN-EILOLA, Finland)

SOINI and TUOMINEN-EILOLA presented a study on peer group consultation (PGC) in studying experience. PGC sessions cascade into personally meaningful experience, collaboration in producing material given in experience, collaboration in search for meaningful focus in material for its presenter, and commonality of the foci. The presenters pointed to the rhythm of PGCs that consists of a constant change between withholding and release in the group. Withholding stands for maintaining ambiguity and ambivalence whereas release diminishes the former two. Further characteristics are that as long as ambiguity is predominant, the material expands and the definition of the foci of case is to be explored instead of material being fixed and releasing the foci. The task of supervisors is to structure the process and underlying rhythms.

The combination of methods is mirrored in the structure of the PGC. Main emphasis lies on epistemic functions and roles. PGC are preferred because they not only contain the same functions as with a pair in face-to-face consultation, but also provide the opportunities for multiple perspectives and external observation. Further roles are presenter, consultant, and observer. Settings are different according to the overall topics consultation, definition, and discussion.

As one of the results, the presenters noted that a group provides (1) multiple positions for distributing epistemic functions of consultation, (2) multiple perspectives for facilitating abundant material, and (3) multiple choices for conceptualizing the presented material, and (4) a common and careful definition of the issue is possible.

Mixed Methods: Challenging Work and Organizational Psychology—the Case of Employee Commitment (Nicole TORKA, The Netherlands)

In her paper on "mixed methods: challenging work and organizational psychology—the case of employee commitment" TORKA presented a combination study from research on commitment. Remarking on the fact that in organizational research qualitative research is still attacked as unscholarly and thus almost nonexistent, she presented her research on employment contracts and employee commitment in four Dutch metal companies. Using short, informal, semi-structured interviews and observations to investigate the meaning of commitment from a blue-collar worker's perspective she found that the workers' subjective perspective differs dramatically from the concept of commitment employed in normative measurements of commitment. Moreover, as TORKA pointed out, the "walking and talking on the floor" method, shed new light on classical concepts of commitment. For example, as with temporary contracts, atypical contracts are often seen as detrimental to commitment. TORKA found no differences between so-called atypical and typical workers. Another finding of interest for the subject of the conference was also the fact that the questionnaire supported the outcome of the qualitative explorative study and supported the qualitative results.

Teacher Education for Interculturality: Complementarity of Questionnaires and Group Discussions (Antonio Medina RIVILLA & Ma. Concepción Domínguez GARRIDO, Spain)

In their investigation of methodological complementariness RIVILLA and GARRIDO tried to find a balance between "objectiveness" and subjectivity. On the one hand, this objectiveness comes from a quantification of the interviewees' answers. On the other hand, subjectivity and flexibility correspond to the discussion groups. The results bring out a new educational intercultural model that creates for teachers a complete personal and professional development. There is a triple complementariness between (1) inferior values and their meaning globally; (2) estimated percentage and the media of all dimensional questions; and (3) the sum of open questions, texts, and 1500 teachers' answers. The group paid attention to three different parameters: coincidence, analogy, and dissonance. The final synthesis of their analysis generates several ideas, for example: formation and compromise with interculturality, intercultural teachers' reality as an integral possibility of promotion and sharing experiences, and values from formational schools and communities.

Content Analysis of Task-Solutions As a Means to Generate Achievement Scores (Sabine K. LEHMANN-GRUBE & Katrin SOMMER, Germany)

LEHMANN-GRUBE and SOMMER shifted the attention towards two research questions: instruction and prerequisite knowledge of the "Soxhlet," an apparatus for extraction of ethereal oils in alcohol. Instruction was surveyed for enhancement of students' capability to relate everyday techniques to scientific and technical concepts. The second, prerequisite knowledge, points out the necessity in teaching to transport not only knowledge, but also understanding, in this case understanding of complex chemistry engineering apparatus. Methodologically, a pretest and post-test were assigned. In the open pre-test, participants were asked to describe separation methods, to name them, and to explain their mechanisms with an example. The post-test consisted of explanation of functioning, identification of separation methods, examples in everyday experience and combining various methods that are integrated in the Soxhlet, and the comparison of examples and apparatus. In the time between the pretest and post-test, worksheets that informed the students about processes in the apparatus and everyday analogies were applied.

The post-test data were analyzed for best answers, best solutions, and different levels of terminology. The texts were also coded and classified to form categories. LEHMANN-GRUBE gave examples of lexical definitions (context, independent, context-bound, simplified) to demonstrate the coding. She also provided examples of how students describe the Soxhlet after they observed the processes. Categories were weighted to allow aggregation of different measures. Further quantitative analyses tried to combine the relationship between the answers with multivariate regression models that would be hardly possible with qualitative methods. From the main results it can be seen that samples differ according to level of terminology and prerequisite knowledge has a smaller effect on level of terminology.

Final conclusions pointed to a low-road and high-road transfer according to SALOMON and PERKINS (1989). SALOMON and PERKINS define low-road transfer as knowledge that is learned unintentionally, implicit, often acquired by model learning, and learned by reinforcement. In contrast, high-road transfer can be characterized by abstraction resulting from explicit instruction. It is guided by meta-cognitive activities, decontextualization, and fosters epistemic structures to cope with situations if automatic solutions fail or are blocked.

Emotion and Learning in Elementary School Classrooms (Michaela GLÄSER-ZIKUDA, Germany)

GLÄSER-ZIKUDA used a qualitative-quantitative approach to research students' learning emotions in instruction. Emotions and learning have various theoretical foundations such as self-regulation theory, learning emotions, anxiety research, and well-being. GLÄSER-ZIKUDA presented a model that tried to integrate these theories. Important aspects point to quality of instruction and social climate (teacher-student-student interactions). Her research questions were: (1) the relationship of emotions to instructions, and (2) the specific contribution of qualitative-quantitative methods.

The design covered a multi-modal approach of semi-structured interviews, diary logs, and video observations to investigate relevant emotions such as interest, well-being, anxiety, or boredom.

Beginning with the results, GLÄSER-ZIKUDA presented descriptions of positive as well as negative emotions targeted at teachers' behavior, instruction, and social climate. As a frequent emotion, boredom can be named for teacher-centered and pleasure for student-centered interactions. Video-observations demonstrate that clearly structured presentations were related to interest, while satisfaction of students was related to positive feedback coming from the teacher.

Quantitative results supported the findings in the sense that teacher competencies were strongly associated with students' emotions. In contrast, the correlations of other dimensions such as self-regulated learning or social climate were weaker. In particular, the teachers' ability to categorize (diagnostic competence) students was an emotionally important aspect for students. In general, well-being seemed to be an essential factor in instruction and a crucial indicator of the quality of teaching. However, anxiety is more person-related and depends more on individual cognition and personal abilities.

Qualitative results enhance the quantitative ones. With them it was possible to give detailed descriptions and to combine theory and empirical data to a greater extent to derive interventions for the conception of teacher training.

GLÄSER-ZIKUDA ended her presentation with an overview of future perspectives. These included the differentiation between age, gender, school type, and various instructional settings. On the methodological side, she noted that multivariate analyses, single cases, process surveys, and mutual validation of different data were fruitful approaches to better understand what students actually feel in learning settings.

GÜRTLER explored how students experience humor in the classroom. This topic is worthy of research as on the one hand the ideal teacher is always humorous, while on the other hand recent approaches to quality of teaching (HELMKE 2003) rarely even mention humor as a social facilitator. Theories of humor are mostly researched in quantitative ways (e.g., factor-analytically). In his study, GÜRTLER investigated subjective theories (GROEBEN & SCHEELE 2000) following the two-phase structure (separation of inquiry of content and of structure) of the Research Program of Subjective Theories. All answers were coded in AQUAD 6 and analyzed for prototypes. Prototypes were realized by applying methods described in OLDENBÜRGER (1981). Sequential hypotheses were formulated and tested. A two-way ANOVA showed significant sex difference in word-production (sex versus school type). These differences were supported by a permutation test (via linear discriminant analysis) with the group vector sex versus school type. Thus, qualitative findings (e.g., boys write less, are more cool, give more "joke" answers, girls write more, are more empathetic, but also more critical) were absolutely in line with quantitative findings. Research restrictions arose from the observation that empirical frequency matrices of sequential hypotheses rapidly develop near singularity (see also LEHMANN 1995). This issue is rarely discussed in qualitative or quantitative research despite the fact that it is not an artifact at all.

Silence Breaking in Families and in Neighborhoods. A Comparative Study with Qualitative and Quantitative Data (Mechthild KIEGELMANN, Germany)

KIEGELMANN examined in her study the psychology of silence breaking and how individuals can be strengthened to resist silencing. The aim was to compare qualitative and quantitative results regarding the research questions: (1) Does group identification make a difference for silence breaking? (2) How do participants explain their silence or silence breaking? She used a paper and pencil questionnaire, with 150 participants and interviews, on responses to social expectations and silencing regarding breaking the silence about an affair within the family or neighborhood. The analysis was facilitated by SPSS and AQUAD. The quantitative comparison differentiated between the condition of the neighborhood and the condition in families. The qualitative study categorized information about psychological processes of silence breaking.

The results of both the qualitative and the quantitative analysis indicate that there is a difference between the silence breaking in the neighborhood and in the family. The different quality of the relationship could be a reason for the difference in the attitude about silence breaking. On the other hand, silence can be caused by the triviality of the problem, rather than because of the group identification. In the discussion KIEGELMANN points out that abilities such as social and emotional competence or bodily relaxation exercise control over language and social situation. However, the quality and importance of the relationship has to be added to previous results to understand silence breaking phenomena. Other reasons for silence can be conformity in groups, threat and active distancing. But triviality has to be added here: The discernment of others' needs obviously must reach a threshold of importance for individuals before they act.

Combining Qualitative and Quantitative Research Strategies for Analyzing the Experience of Aversive Tension in Different Patient Groups (Jeannette BISCHKOPF, Christian STIGLMAYR, Stephan SCHEUER, Viktoria ALBRECHT, Nancy PORZIG & Anna AUCKENTHALER)

Patients with borderline personality disorder report states of aversive inner tension, which they try to terminate by impulsive self-harming behavior. Building on the results of an interview study about subjects' use of the term "tension" (SCHEUER 2003), the experience of aversive tension was examined systematically in different patient groups. 117 participants (30 patients with borderline personality disorder, 30 patients with depression, 27 patients with anxiety disorder, as well as 30 healthy controls) were asked about their experiences of aversive tension using an open questionnaire. Qualitative content analysis (MAYRING 2000b) was applied for coding study participants' self reports. The qualitative analysis led to the differentiation of the experience of aversive tension into the following categories: cognition, emotional and physical aspects, action tendencies, behavior, and coping. The quantitative analyses revealed significant disorder-specific differences in the experience of inner tension. The study followed the combination model of a mixed methodology approach, using the sequence model in which a qualitative approach is applied in a pre-study, and further quantitative analyses are used afterwards built on the qualitative results.

Sleep Disorders and Nightmares of Children with ADHD—A Qualitative Study (Magdalena MAJOREK, Switzerland)

Sleep disturbance was once a diagnostic criterion for ADHD (Attention Deficit Hyperactivity Disorder). It was omitted in the DSM-IV but the latest research suggests that sleep disturbance is an important component of the syndrome, and that children with ADHD suffer from nightmares. In view of recent research findings demonstrating the impact of sleep phases to cognitive functioning one can postulate that sleep disturbance may play a causative role in ADHD. The aim of the study is to explore sleeping patterns and the running themes of dreams of children with ADHD. It is proposed to study six children diagnosed as ADHD with noticeable sleep disorders. The study will last twelve months and comprise three phases: assessment phase I, treatment phase, and assessment phase II. An integrated approach to collecting data will be used: (1) quantitatively, in the form of a questionnaire for the assessment of general behavior and attention, and parents' questionnaires to determine subjective sleeplessness; and (2) qualitatively, in the form of parents' weekly reports about the sleeplessness of their child, documentation of dreams and artistic products, and guided interviews. A functional analysis, triangulation, and created Grounded Theory will be applied to the data.

Following the Process: Using Mixed Methods to Capture Clients' Experiences of Change in Therapy and Counseling (Silke-Birgitta GAHLEITNER, Germany)

GAHLEITNER presented an ongoing study in the field of trauma research entitled "Following the process—using mixed methods to capture clients' experiences of change in therapy and counseling." Research findings show that the results of psychotherapeutic treatments of complex trauma remain unsatisfactory. In contrast to quantitatively oriented mainstream psychotherapy research, her study will focus on the clients' subjective experience. Data collection will be conducted with problem-centered interviews in combination with the semi-structured Client Change Interview, quantitative questionnaires, and include psychometric tests administered after the interviews. The data will be analyzed both qualitatively by content analysis in combination with a gender-specific approach, and quantitatively by statistical methods. GAHLEITNER brought up for discussion a number of problems that can arise at the interface between qualitative and quantitative approaches: (1) sampling, which should be usable for both qualitative and quantitative procedures; (2) the combination of the Client Change Interview (ELLIOTT, SLATICK, & URMAN 1991) with the problem-centered interview (see WITZEL 2000), so as to take into account results from a previous study on sexual abuse; and (3) the difficulty of finding a suitable mode of combining and interpreting the qualitative and quantitative results.

Regula ZUEGER-CÁCERES explored participatory development projects and their contribution to poverty alleviation. At first, she introduced the terms poverty, well-being, participation, and empowerment. She discussed these terms in the research context, the poor rural population in the Peruvian Andes. Methodologically, she used Q-methodology and factor analysis. The sample was collected according to Q-sort. For that purpose statements were grouped according to the themes in eight categories: power, enlightenment, wealth, skills, well-being, gender, affection, and request. Q-sample statements were then rank-ordered by participants ranging from agree to disagree.

Participants came from the Peruvian Andes. The projects were Food and Agriculture Organization of the UN (FAO), Farmer Field Schools and SANBASUR (Improvement of Rural Water and Wastewater Management). The author also gave examples of a gathering on poverty concerns. Answers were e.g., "It is my destiny to be poor," "I am ashamed at what I have become," or "The future lies in the education of our children."

The Q-sorts were then processed by principal component analysis (PCA) and Centroid FA (with rotation). ZUEGER-CÁCERES interpreted the emerging factors as operants with functional segmentations. They act as representatives of the inner realities of the people who were studied. The benefits of these factors were seen as being an indicator of change. The idea is that as soon as empowerment takes place, the inner realities of people should change too. Therefore, social changes should be mirrored in research instruments.

The Press As an Educational Resource in Adult Education. A Quantitative-Qualitative Methodological Perspective (Carmen RICOY LORENZO, Ramón Pérez PÉREZ & Tiberio Feliz MURIAS, Spain)

Carmen RICOY LORENZO et al. investigated the press as an educational resource in adult education. Research objectives were to determine what kind of didactic resources are available in adult education, but also to investigate and to describe to use of written material (the press) as a learning factor in teaching. Another goal was to explore and to concretize the support possibly achieved by using the press as a learning tool.

Methodological aspects were descriptive research combined with case study, and integrating qualitative as well as quantitative approaches. As research instruments, questionnaires, interviews, and a study on didactic resources were chosen. The sample consisted of 110 teachers and 512 students.

Concerning the results, the questionnaire, interviews, and document analysis offered different results. Teachers and students named in varying degrees school workshops, vocational training, secondary training, and basic training as educational environments for using the press.

A main result was that didactic resources were emphasized as important in adult education. Educational advantages could be expected by using the press. However, how the press itself is willing to support adult education or whether joint activities are possible together with the press remains to be investigated. To cite some examples, predominant didactic aids were written, graphical, audio, and visual material. However, most parts of the sample use the press sometimes as an aid (teachers to a greater extent than students). Students tend not to use the press more than just occasionally and only a small number of teachers do not use the press at all.

In sum, the classic aids are still written materials in teaching. The most frequent symbolic media are books, notebooks, and maps. This supports the idea of an integration of the different media to combine them in the curriculum as well as outside of the curriculum. The press, as LORENZO et al. stated, is not implemented on a regular basis in teaching, but rather only occasionally. The functions of the press can be summarized by discussion and debates, critical sense development, analysis of news, and self-creations such as advertising, interviews, and articles. The press may help especially in the task of what is called transfer: building a bridge between knowledge and its application within extra-curricular situations.

Combining Software for Qualitative and Quantitative Analysis: Examples from an Empirical Study (Leo GÜRTLER & Günter L. HUBER, Germany)

In "combining software for qualitative and quantitative analysis" GÜRTLER and HUBER gave an introduction into practical aspects of mixed methods. Beginning with the method of truth tables to identify typical configurations in qualitative data, HUBER explained the basic principles of this analysis to find types within qualitative data. The method is rooted in the works of BOOLE and is the basis of every electronic chip or processor. Here, it is called qualitative comparison. This binary logic was at first prepared by RAGIN (1987) for the social sciences and later developed into fuzzy logic. GÜRTLER went on with some demonstrations of what is possible with quantitative analyses to combine them with qualitative data that are coded in AQUAD 6. With examples from interviews on teacher's thinking on humor, qualitative data (i.e., persons, codes) can easily be visualized by multidimensional scaling to develop heuristics for differences and similarities. Furthermore, a randomization test on hierarchical clustering (OLDENBÜRGER 1986) showed significant results beyond random and legitimated further analyses with hierarchical cluster algorithms (agglomeration). Going further with examples from the Boolean approach, he suggested calculating at least one or two implicants (as the output of the RAGIN method is called) in each case to really gain benefits from this analysis if used on qualitative data.

Elliott, Robert, Slatick, Emil & Urman, Michelle (2001). Qualitative Change Process Research on Psychotherapy: Alternative Strategies. Psychologische Beiträge, 43(3), 69-111.

Gigerenzer, Gerd (1981). Messung und Modellbildung in der Psychologie. München: Reinhardt.

Groeben, Norbert & Scheele, Brigitte (2000, June). Dialog-Konsens-Methodik im Forschungsprogramm Subjektive Theorien [9 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal], 1(2), Art. 10. Available at http://www.qualitative-research.net/fqs-texte/2-00/2-00groebenscheele-d.htm [Date of Access: June 06, 2005].

Helmke, Andreas (2003). Unterrichtsqualität. Erfassen – bewerten – verbessern. Seelze: Kallmeyer.

Hoffman, Martin L. (1975). Moral Internalization, Parental Power, and the Nature of Parent Child Interaction. Developmental Psychology, 11, 228-239.

Huber, Anne A. (2004). Neue Lernformen für neue Anforderungen an Lernende in Schule und Erwachsenenbildung: Der Ansatz des "Wechselseitigen Lehrens und Lernens (WELL)". Zeit-Schrift für interdisziplinäre Bildung und praktische Philosophie, 6-7, 42-50.

Kiegelmann, Mechthild; Held, Josef; Huber, Günter L. & Ertel, Irmentraud (2000, June). Das Zentrum für Qualitative Psychologie an der Universität Tübingen [24 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal], 1(2), Art. 14. Available at: http://www.qualitative-research.net/fqs-texte/2-00/2-00kiegelmannetal-d.htm [Date of Access: June 08, 2005].

Lamm-Hanel, Nicole (2003). Mutter-Kind-Interaktionen in Überschreitungssituationen: Kritik, empirische Überprüfung und Weiterentwicklung von M. L. Hoffmans Taxonomie mütterlicher Erziehungsmaßnahmen. Dissertation, Universität Leipzig.

Lehmann, Günter (2002). Statistik: Einführung in die mathematischen Grundlagen für Psychologen, Wirtschafts- und Sozialwissenschaftler. Heidelberg: Springer.

Mayring, Philipp (2000a). Qualitative Inhaltsanalyse. Grundlagen und Techniken (7th ed.). Weinheim: Beltz.

Mayring, Philipp (2000b, June). Qualitative Content Analysis [28 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal],1(2), Art. 20. Available at: http://www.qualitative-research.net/fqs-texte/2-00/2-00mayring-e.htm [Date of Access: July 07, 2005].

Oldenbürger, Hartmut-A. (1981). Methodenheuristische Überlegungen und Untersuchungen zur "Erhebung" und Repräsentation kognitiver Strukturen. Dissertation; Mathematisch-Naturwissenschaftliche Fachbereiche der Georg-August-Universität Göttingen.

Oldenbürger, Hartmut-A. (1986). Does a Tendency to Group Pupils or Attributes Exist Within Teacher's Cognitions/Judgments? In Miriam Ben-Peretz, Rainer Bromme & R. Halkes (Eds.), Advances of Research on Teacher Thinking (pp.186-200). Lisse: Swets und Zeitlinger.

Patton, Michael J. (1991). Qualitative Research on College Students: Philosophical and Methodological Comparisons with the Quantitative Approach. Journal of College Student Development, 32, 389-396.

Ragin, Charles (1987). The Comparative Method. Moving Beyond Qualitative and Quantitative Strategies. Berkeley, CA: Berkeley University Press.

Salomon, Gavriel & Perkins, David N. (1989). Rocky Roads to Transfer: Rethinking Mechanisms of a Neglected Phenomenon. Educational Psychologist, 24(2), 113-142.

Scheuer, Stephan (2003). "Ich fühle mich angespannt". Eine explorative Studie zum Erleben eines alltäglichen Phänomens. Unpublished Thesis; Fachbereich Erziehungswissenschaft und Psychologie der Freien Universität Berlin.

Silke-Birgitta GAHLEITNER wrote her doctoral thesis on gender-specific modes of coping with sexual abuse at the Institute of Clinical Psychology of the Free University Berlin. She received her training in psychotherapy in Vienna (Austria) and is a licensed clinical social worker in Germany (ZKS). She is now working as a Professor for Psychology and Clinical Social Work at the University of Applied Sciences Ludwigshafen and as a lecturer at the first Clinical Social Work Masters degree program in Germany. Her main areas of work are: psycho-social counseling, psychotraumatology and trauma therapy, gender research and qualitative research methods.

University of Applied Sciences Ludwigshafen
Maxstrasse 29 - D-67059 Ludwigshafen
Germany

Leo GÜRTLER works as a psychologist at the Department of Educational Psychology, School for Education, University of Tübingen. He is a member of the Center for Qualitative Psychology and wrote his doctoral thesis on inner and outer perspectives on humorous actions in school and adult education. Vipassana Meditation was investigated as a supportive technique to develop insight into one's own thoughts, feelings, and actions to foster humor. His actual research interests are the Research Program Subjective Theories (German: "Forschungsprogramm Subjektive Theorien") and the finding of resilience mechanisms for recovery from drug addiction. He is at the end of his training as a family therapist.

Institut für Erziehungswissenschaft (School for Education)
Abteilung Pädagogische Psychologie (Department of Educational Psychology)
Universität Tübingen (University of Tübingen)
Münzgasse 22-30
72070 Tübingen
Germany

Gürtler, Leo & Gahleitner, Silke-Birgitta (2005). Conference Report: Fifth Annual Meeting of Qualitative Psychology "Qualitative and Quantitative Approaches to Learning and Instruction" / First Meeting of the Special Interest Group #17 of the European Association for Research in Learning and Instruction (EARLI) "Mixed Methodology in Psychological Research" [26 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(3), Art. 33, http://nbn-resolving.de/urn:nbn:de:0114-fqs0503330.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License

Thursday, Oct. 21, 2004
Arrival, welcome, and introductory session	6:00 pm
Friday, Oct. 22, 2004
Plenum	09:00-11:00 am
Work groups	09:30 - 10:30 am	Group 1
Work groups	11:00 - 12:30 am	Group 2
Work groups	02:30 - 04:00 pm	Group 3
Open discussion	04:00 - 04:30 pm
Work groups	05:00 - 06:30 pm	Group 4
Saturday, Oct. 23, 2004
Invited Address	09:00 - 10:30 pm
Work groups	11:00 - 12:30 pm	Group 5
Work groups	02:30 - 04:00 pm	Group 6, Group 7
Work groups	04:30 - 06:00 pm	Group 8 & 9
Sunday, Oct. 24, 2004	09:00 - 12:30 pm

Group 1: Potential of mixed method approaches
1.1	Diaries and Questionnaires: Potentialities and Limitations of Mixing Methods (Siegfried HOPPE-GRAFF, Germany, Nicole LAMM-HANEL & Hye-On KIM, South Korea)
Group 2: Mixed methods in psychological studies
2.1	Coherence in Knowledge Communication: How Do Online Groups Communicate? (Karin SCHWEIZER, Germany; Manuela PÄCHTER, Austria, Bernd WEIDENMANN, Germany)
2.2	How to Add Qualitative Depth to Quantitative Findings in a Study on Cooperative Learning (Anne A. HUBER, Germany)
2.3	Iteration in Instructional Design: An Empirical Study on the Specification of Training Simulators (Danielle VERSTEGEN, The Netherlands)
Group 3: Variations of mixed methods approaches
3.1	Six Alternatives to Mixed Methods in Qualitative Research (Gary SHANK, USA)
3.2	Maximizing Data Use: Mixed Qualitative Methods (Gillian M. BOULTON-LEWIS & Lynn A. WILSS, Australia)
3.3	Mixing Qualitative and Quantitative Methods (Philipp MAYRING, Austria)
Group 4: Combining qualitative methods
4.1	Research and Intervention by Interviews and Learning Diaries in In-Service Teacher Training (Günter L. HUBER & Jürgen W.H. ROTH, Germany)
4.2	Peer Group Consultation in Studying Experience (Hannu SOINI & Tuulikki TUOMINEN-EILOLA, Finland)
Invited address:
Triangulation as a Framework for Mixed Methods Research in Psychology (Uwe FLICK, Germany)
Group 5: Combinations of qualitative and quantitative methods
5.1	Mixed Methods: Challenging Work and Organizational Psychology—the Case of Employee Commitment (Nicole TORKA, The Netherlands)
5.2	Teacher Education for Interculturality: Complementarity of Questionnaires and Group Discussions (Antonio Medina RIVILLA & Ma. Concepción Domínguez GARRIDO, Spain)
Groups 6 and 7: Access to individual experience by mixed methods
6.1	Content Analysis of Task-Solutions As a Means to Generate Achievement Scores (Sabine K. LEHMANN-GRUBE & Katrin SOMMER, Germany)
6.2	Emotion and Learning in Elementary School Classrooms (Michaela GLÄSER-ZIKUDA, Germany)
6.3	How Students Experience Humor in the Classroom (Leo GÜRTLER, Germany)
7.1	Silence Breaking in Families and in Neighborhoods. A Comparative Study with Qualitative and Quantitative Data (Mechthild KIEGELMANN, Germany)
Groups 8 and 9: Deeper understanding of findings by mixing methods
8.1	Combining Qualitative and Quantitative Research Strategies for Analyzing the Experience of Aversive Tension in Different Patient Groups (Jeannette BISCHKOPF, Christian STIGLMAYR, Stephan SCHEUER, Viktoria ALBRECHT, Nancy PORZIG & Anna AUCKENTHALER)
8.2	Sleep Disorders and Nightmares of Children with ADHD—A Qualitative Study (Magdalena MAJOREK, Switzerland)
8.3	Following the Process: Using Mixed Methods to Capture Clients' Experiences of Change in Therapy and Counseling (Silke-Birgitta GAHLEITNER, Germany)
9.1	Assessing Empowerment with Q-methodology (Regula ZUEGER-CÁCERES, Switzerland)
9.2	The Press As an Educational Resource in Adult Education. A Quantitative-Qualitative Methodological Perspective (Carmen RICOY LORENZO, Ramón Pérez PÉREZ & Tiberio Feliz MURIAS, Spain)
9.3	Combining Software for Qualitative and Quantitative Analysis: Examples from an Empirical Study (Leo GÜRTLER & Günter L. HUBER, Germany)