A Description of the Uses of Content Analyses and Interviews in Educational/Psychological Research

Abstract: Psychology in general, and educational psychology in particular, has traditionally adopted the methods of positivistic science; that is, it employs experimental research methodology and statistical tests of significance. However, more and more psychologists are beginning to use and to appreciate qualitative research methodologies. These methodologies include ethnographies of classrooms, in the case of educational psychology, content analyses of research participants' verbal or written responses to problem solving tasks, in the case of cognitive psychology, and so forth. This paper presents a description of the uses of content analyses and interviews with participants in educational psychology research. The purposes of the research described in this paper were to examine how undergraduate students would apply knowledge to case study problems and to find out how students felt about the use of case studies as an instructional tool. To achieve these aims, content analyses were conducted on students' written responses to cases and individual interviews were conducted with students.

The methods typically employed in psychological research tend to be those borrowed from positivistic science, usually physics. Thus, experimental research methodology and statistical tests of significance tend to form the core of psychologists' store of methods. The use of such methods has a long history in psychology. Recently, however, some psychologists have begun to utilize qualitative research methodologies, such as ethnographies and interviews, that allow them to offer a richer description of the phenomena being studied. [1]

The aims of the study herein described were threefold. First, the study sought to characterize and describe college students' written analyses of case studies in order to determine how they applied knowledge acquired from lectures and textbooks to solve the dilemmas presented in the cases. Second, the study sought to explore students' attitudes concerning the effectiveness of case studies as an instructional method. Third, the study sought to discern if analyzing case studies affected students' level of self-regulation. The research described in this paper took place within the framework of educational psychology. [2]

Students' written analyses of case studies were the primary sources of data used to describe and characterize students' application of knowledge. Case studies were the vehicles whereby students were given the chance to apply their knowledge. Case studies involve students in authentic or hypothetical problem situations that may reflect the problems faced in the discipline being studied. Cases are narrative in nature, tend to be ambiguous, and may call for multiple solutions. Unlike multiple-choice tests, and other objective measures, cases provide a way for students to put their knowledge to use. Thus, examination of students' written analyses should provide a means to discern whether students understand and can use the knowledge they have acquired from lectures and textbooks. If they do apply their knowledge, to what degree do they do so? That is, do they simply repeat what the textbook or the instructor has said, or do they demonstrate understanding by applying concepts in creative and insightful ways. To examine how students applied their knowledge content analyses were conducted on their written responses. [3]

In order for case-based instruction to be effective, students must have the requisite knowledge, and must be willing to put forth sufficient effort and persistence, and to self-regulate their learning (BLUMENFELD, SOLOWAY, MARX, KRAJCIK, GUZDIAL, & PALINSCAR, 1991). ERTMER, NEWBY, and MacDOUGALL (1996) have argued that these characteristics, while not unique to case-based instruction, may be particularly necessary in a case-based learning environment in which students engage "in complex and inherently ambiguous learning tasks" (p.721). ERTMER et al. further suggest that students' approaches and responses to case-based instruction rely upon their level of self-regulation. ZIMMERMAN (1994) defines self-regulation as "the degree that individuals are metacognitively, motivationally, and behaviorally active participants in their own learning process" (p.3). [4]

ERTMER et al. (1996) have stressed the connections between case-based instruction and students' levels of self-regulation. ERTMER et al. argue that students who are classified as high self-regulators, who are not familiar with the material being taught (for example, educational psychology) or with the instructional method (for example, case studies), will achieve academic success because they will consistently and persistently employ effective study and learning strategies. Students classified as low self-regulated learners who find themselves in the same situation as their high self-regulating peers may well falter by not employing effective study and learning strategies. ERTMER et al. suggest that by observing students who are classified as high or low self-regulators will allow researchers to discern what attributes, attitudes, and approaches these students bring to case-based instruction. The presence of these attributes, or lack thereof, may allow researchers "to identify enabling instructional conditions and strategies that enhance the performance of all kinds of learners in a case-based environment" (p.721). [5]

The content analyses conducted on the students' written responses to the cases were focused upon, in part, determining how high and low self-regulating students approached cases. It might be expected that high self-regulators would apply their knowledge in creative ways and not be off put by the ambiguity inherent in most case studies. In contrast, it could be expected that low self-regulators would not apply their knowledge in creative ways and would be put off by the ambiguity found in cases. Furthermore, it was expected that high self-regulators would apply theory in a deep manner. That is, high self-regulators would employ a theory from the textbook or lecture and use that theory like a tool to ameliorate the problem in the case. For instance, a high self-regulator might read a case study and after reflecting on the situation in the case decide that concepts from behavioral psychology would help solve the problem. Therefore, this student would take concepts such positive reinforcement or punishment and apply them to the problem in the case. A low self-regulator, in contrast, could be expected to simply state that a particular case study is an example of behavioral psychology and not employ the relevant concepts to the problem in the case, or do so superficially, with little or no reflection. [6]

RIDLEY (1991) has argued that self-regulation rests on a continuum: On the low end, individuals tend to be unreflectively automatic, and on the high end, individuals tend to be reflectively intentional. Students on the low end of the continuum react "to the situation [classroom situation] with unexamined and habitual thoughts ..." (p.33). Given the differences in approaches to learning found among high and low self-regulated learners, and given that case-based instruction tends to ask students to engage in complex and ambiguous learning activities—attempting to solve cases—that usually require them to call upon previous experiences, and that may require novel solutions, what differences may be found in knowledge use by students classified as either high or low self-regulators? Do they approach such problems intentionally and reflectively? Or do they approach such problems automatically and unreflectively? And does case-based instruction move students along the continuum from the low end (unreflective and automatic) to the high end (reflective and intentional)? ERTMER et al. (1996) have asserted that "self-regulated learning skills are thought to enhance students' approaches to case-based instruction," and "case-based instruction is thought to promote and support the development of self-regulation skills" (p.723). [7]

The subject matter selected for the current study was educational psychology. This domain was selected because it provided a clear link between theory and practice. That is, many of the theoretical and conceptual issues raised in educational psychology have real-world correlates. For example, the psychological concept of operant conditioning (reinforcing or rewarding behavior) lends itself readily to real-life situations. [8]

Thirty eight undergraduate college students who were enrolled in Introduction to Educational Psychology were asked to participate in the study. The researcher was also the instructor for the course. The course met once a week and was designed to offer a survey of various psychological theories as they apply to educational issues. It was from the 38 students that six students were selected and whose case analyses were examined and who were interviewed. The decision to focus on six students was made due to the fact that over the course of the semester all of the students would have read and analyzed four case studies, providing far too much data to be analyzed in a reasonable time. The six students were selected based upon their performance on the Motivated Strategies for Learning Questionnaire (MSLQ). This instrument attempts to assess students' level of self-regulation. Three students who scored low on the MSLQ and three students who scored high were selected in order to discern if there were differences in the way these two groups of students approached the cases. [9]

At four points in the semester all students were required to complete case analyses that called for them to employ psychological theories that were introduced in the textbook and lectures. The cases that students read and analyzed were adapted from existing case studies. The topics covered were learning theories, cognitive psychology, social learning theory, motivation theory, and behavioral theory. These topics were selected by the researcher from the course syllabus. Three of the cases were concerned with educational issues such as teaching method, low self-esteem among children, and motivation to learn. The fourth and final case was a transfer case. That is, it did not take place in a classroom, nor did it involve education, per se. [10]

It was with the assignment of each case that the researcher had to make a decision. Undergraduates tend to like to be told what to do; that is, they want to know what is expected of them. If very prescriptive instructions were given, it was likely that students would follow those instructions and not think about or reflect upon the problems faced in each of the cases. The researcher was interested in how students would spontaneously use their knowledge. Thus, the instructions given to students were deliberately vague so as not to constrain their responses. The students were also told that they could use any materials they deemed relevant to the task at hand. The materials included lecture notes and textbooks. [11]

All students received feedback in the form of letter grades and written comments pointing out misuse of conceptual or theoretical material and suggestions as to how the next analysis could be improved. Grammatical errors were pointed out as well. [12]

Following their analyses of the third case, semi-structured interviews were conducted with the six selected students to examine their attitudes concerning the effectiveness of using case studies in the classroom. The students were asked for their opinions concerning the use of cases as an instructional method, and whether working on cases was an effective way to learn theory. Interviews were conducted by the researcher and were audiotaped with the students' permission. [13]

Students' written case analyses were analyzed using a coding scheme devised by the researcher. The goal of the content analysis was to provide an overall description of how students analyzed the case studies. This description includes the ways in which students' knowledge manifests itself in their case analyses. Constructing the coding scheme to be used for the analysis proved to be one of the most labor intensive and time consuming aspects of the study. The number of codes ranged from 36 to the final number of five. As the codes were being developed, the researcher had to continually reflect upon the purpose of coding; was coding going to be done to capture everything students wrote, or was coding going to capture a small number of concepts. As the number of codes increased and, consequently, became unmanageable, it became evident to the researcher that a smaller number of codes would be most effective. The table on the following page contains definitions and examples of each of the codes used to analyze students' responses. The researcher believed that these five codes offered a clear representation/description of how students respond to cases. [14]

Because cases may involve more than one problem, it was decided that problem identification would be helpful in discerning whether students perceived more than one problem. Only by identifying the relevant problem in the case and recognizing the consequences of that problem could a student go about constructing a reasonable solution. The solution process involved making explicit the connections between theory and the problem at hand. A student analyzing a case would demonstrate understanding of theory and theory use by utilization of examples and by using theory to extract a problem or interpret a problem in relation to a particular theory. It is one thing to interpret or represent a problem in terms of theory, it is another thing to be able to use a theory to solve a problem. Being able to discuss or describe the outcomes resulting from the application of theory would provide another indication that students had some understanding of theory.

Table: Descriptions of the Concepts Used to Analyze Students' Case Study Responses [15]

Once the final set of codes was developed, two graduate students served as independent coders. The two coders and the researcher read and coded each of the 24 case analyses generated by the six students in the study. Both of the coders were trained to use the coding system by coding practice cases prior to doing the actual coding. The percent agreement was calculated between the two coders and between each of the coders and the researcher. There was little agreement among the three coders. There are two possible explanations for this. [16]

First, one of the coders had taught the course, educational psychology, many times before and was familiar with the material. The other coder had not taught the course before and, thus, was unfamiliar with the material. Furthermore, the researcher was also the instructor to this class, giving him a degree of familiarity not only with the material, but with the students, that the other coders could not have had. Having presented the material to the students, and having interacted with them throughout the 15 week semester, the researcher was in a unique position to better understand the students' written case analyses. [17]

Second, the coding system that was used proved to be too unwieldy for the purposes of the study. Fewer codes may have allowed for better agreement. It was decided by the researcher to focus on only five codes in the analyses of students' data. The five codes were chosen because it was felt they would best answer the questions put forth by the study. Importantly, it was decided that the researcher's coding of the data would be the only coding used. As noted previously, the researcher was in the best position to analyze the students' responses. [18]

Given the lack of agreement among the three coders after all of student cases had been coded, the researcher was faced with a decision: Should all three coders be used or only the researcher's coding? After consulting with colleagues, the researcher decided that only his codes would be used to interpret students' responses. This decision was based upon the fact that, as noted previously, the researcher was in a better position to interpret students' responses than the other two coders because he had presented the material and was familiar with the students. For instance, a student may have written something in her or his analysis that was discussed in class; thus, only the researcher would know the context and meaning of that statement. Two other coders, both equally knowledgeable about educational psychology, could have been chosen to code the data. However, it became a matter not only of knowing the content of the course and the codes, but knowing the students, and only the researcher had knowledge of the students. [19]

As the current study was being planned the researcher had decided to conduct interviews over the course of the semester. This decision was based upon methods used in a similar study (ERTMER et al., 1996). Interviewing the students multiple times would have provided information on how or if their attitudes and approaches to cases changed over time. As the semester and the study commenced it became clear that getting the students to come for interviews was almost impossible. Thus, as the semester drew to a close, the researcher arranged to meet with each of the six selected students individually. Accommodating the students' schedules as well as my own proved to be a challenge. [20]

The goal of the interviews was to get a sense of students' beliefs and attitudes about the use of case studies. Three primary questions were asked during each interview, with follow-up questions concerning students' beliefs about cases affecting their motivation to learn and their confidence that they understood that which they had learned, and if they believed analyzing case studies was an effective instructional strategy. [21]

The data provided by the interviews proved to be the most informative of the study. However, a potential problem in interviewing the students was that the instructor qua researcher conducted them, thus creating a situation in which the students might not be as candid as hoped. It quickly became apparent that that would not be a problem. Over the course of the 15 week semester a rapport had been established between the researcher and the students, thus allowing the students to feel comfortable discussing their feelings about the cases. [22]

That the students felt free to express themselves makes the interview data rich in that, through their candidness, the students provided the researcher with insights into what they thought about this method of instruction. For instance one student reported that "It was like being a poor auto mechanic and having access to a brand new set of ... tools ... here take your pick!" Another student replied "I don't think the case studies made that much difference." [23]

In this section I will offer a brief overview of the main results of the study. First, I will discuss changes in students' MSLQ scores. Second, I will discuss the differences in students' case study analyses. Third, I will discuss students' attitudes toward case studies. [24]

One of the questions asked in this study was whether working on cases would increase students' level of self-regulation. Overall, four of the six students who were the focus of this study increased their MSLQ scores slightly from pretest to posttest. However, only one student changed his score significantly to change his classification from high self-regulation to low self-regulation. It was concluded that for this small number of students, working on cases had a minimal effect upon their level of self-regulation. Moreover, it became clear from interviews that many of the students displayed characteristics of both high and low self-regulated learners. [25]

The second question asked in this study was whether there are individual differences to be found in the ways high and low self-regulating students go about solving case study problems. Of the six students participating in this study, three based their case analyses and solutions mainly on the formal, conceptual knowledge acquired in the textbook and lecture material, with little elaboration. Two of these students were classified as low self-regulated learners, and one student classified as a high self-regulated learner. The other three students relied less on the textbook and lecture material and were guided by, as reported by them in interviews, their instincts and intuitions. Two of these students were classified as high self-regulated learners, and one classified as a low self-regulated learner. Two of these students, one of whom was the low self-regulated learner, called upon their real-life experiences and approached cases as practical, authentic problems. To wit, they called upon their experiences and formal knowledge to analyze and solve the cases, and recognized the benefits of working on cases in terms of understanding and using theory. [26]

The third question asked in this study was focused upon students' attitudes and beliefs about case studies as an instructional method. Each student reported to the researcher that cases were good at giving him or her a chance to use the knowledge they had acquired in the course. Of interest are the differences to be found among students' use of knowledge to analyze cases. All of the students applied theory to the cases. However, despite recognizing the opportunity afforded by cases to apply their knowledge, perhaps some students simply choose not to link theory to the cases in any deep way. It is these students, possibly, who perceived cases as academic exercises, and, wanting a good grade, were reluctant to use theory in a reflective or creative way, thus making their analyses superficial. One student reported that case studies had little effect on how she studied the material. Another student explained that she felt "underconfident" about working on cases. Perhaps it is this sense of "underconfidence" that led this student to rely on the textbook and lecture material. [27]

It is hoped that this brief description of qualitative research conducted in educational psychology was informative. Qualitative research is challenging research. Data do not fit easily into tables and categories. Once a researcher has identified a topic to study, the most important question then becomes how to study it. The choice of research methodology should be guided by the research questions asked. In this instance it was important to know how students analyzed case studies and what they thought about them. Content analyses and interviews were chosen as the best methods to arrive at answers to these questions. [28]

Blumenfeld, P.C.; Soloway, E.; Marx, R.W.; Krajcik, J.S.; Guzdial, M. & Palincsar, A. (1991). Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist, 26, 369-398.

Ertmer, P.A.; Newby, T.J. & MacDougall, M. (1996, Fall). Students' responses and approaches to case-based instruction: The role of reflective self-regulation. American Educational Research Journal, 33(3), 719-752.

Ridley, D.S. (1991). Reflective self-awareness: A basic motivation process. Journal of Experimental Education, 60(1), 31-48.

Zimmerman, B.J. (1994). Dimensions of academic self-regulation: A conceptual framework for education. In D.H. Schunk & B.J. Zimmerman (Eds.), Self-regulation of learning and performance: Issues and educational implications (pp. 3-21). Hillsdale, NJ: Erlbaum.

School of Education
5D28 Forbes Quadrangle
University of Pittsburgh
Pittsburgh, Pennsylvania 15260
USA

Faux, Robert (2000). A Description of the Uses of Content Analyses and Interviews in Educational/Psychological Research [28 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(1), Art. 26, http://nbn-resolving.de/urn:nbn:de:0114-fqs0001265.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License

Concepts	Definition	Example
Problem identification	Diagnosis of a problem(s) presented in a case	James is being disruptive in his math class.
Consequence of problem(s)	The consequences that may arise from the identified problem(s)	Students and teacher are distracted.
Theory application	The introduction of theoretical material and linkage of material to the problem(s) at hand	Operant Conditioning. Positively reinforce James when he behaves correctly.
Example(s) of theory application	Example(s) of how the chosen theory would be applied to the problem(s)	Teacher praises James when he behaves correctly.
Outcomes	The state of affairs that would obtain when a theory is applied	James' disruptive behavior will cease.