Volume 19, No. 1, Art. 9 – January 2018
Conference Report:
Lakshmi Balachandran Nair
Scientific Integrity in Qualitative Research (SCIQUAL) Seminar 2017. Utrecht University, Utrecht, the Netherlands; September 13-14, 2017; organized by Lakshmi Balachandran Nair, Methodology and Statistics Department, Utrecht University
Abstract: The Scientific Integrity in Qualitative Research (SCIQUAL) 2017 seminar focused on the basic rules of good scientific practice and researchers' commitment to (or lack thereof in) adhering to these rules. Especially in the case of qualitative research, where there is a lack of standardized measures to ensure the quality of the methods, scientific integrity is a fuzzy concept and a big concern. To add on to this, increasing demands to publish or perish compel researchers to produce strong, concrete, evidence-based contributions at an alarmingly fast pace. Other factors like financial constraints, competition, etc. might also tempt scientists to achieve success swiftly through the use of unfair research practices. This is an alarming trait since good science is supposed to be credible, authentic, trustworthy, and ethical. SCIQUAL 2017 brought our attention to a few topics (e.g. reflexivity, ethical climate, deviant cases) which comes under the umbrella term of scientific integrity.
Key words: scientific integrity; qualitative research; ethics; reflexivity; publication bias; deviant cases; glocalized methodology
Table of Contents
1. Introduction to the SCIQUAL 2017 Seminar
2. Presentations
2.1 What is scientific integrity?
2.2 How can we evaluate scientific integrity?
2.3 What are the elements affecting scientific integrity during a qualitative study?
2.3.1 Reflexivity
2.3.2 Ethical climate
2.4 What are the elements affecting scientific integrity after a qualitative study?
2.4.1 Deviant cases
2.4.2 Publication bias
2.5 Is following rules the best way to ensure scientific integrity?
2.6 How can we teach scientific integrity to students?
3. Group Discussions
3.1 "We know what does not work. But we do not know what works"
3.2 "Think about mundane activities and build a methodology based on it"
4. Concluding Remarks
1. Introduction to the SCIQUAL 2017 Seminar
The Scientific Integrity in Qualitative Research (SCIQUAL) seminar 2017 was organized with the aim of contemplating the current and future ethical climate in qualitative research. Integrity by definition assumes ethical reflection, self-discipline, and self-critical assessment on the part of researchers. By maintaining the reputation and respectability of science, integrity helps in sustaining a meaningful dialog among researchers themselves as well as between researchers and society. Thus, integrity plays a major role in the development of science. [1]
Most of the discussions on scientific integrity happen at the level of plagiarism and fake data. Many articles are being withdrawn1) following allegations of cooked-up results. However, scientific integrity expands beyond these two malpractices. It involves respect and regard for individuals, groups, and institutions, maintaining the obligations towards the research community, declaration of conflicts of interest, and commitment to research rigor, as well ensuring the transparency of scientific communication (HAMMERSLEY, 2017). There have been several articles examining different aspects of scientific integrity (for instance, SNEE, 2008; WILLIAMS, BURNAP & SLOAN, 2017) pertaining to different research areas and methods. This exposes a gap of similar works in qualitative research. Therefore, SCIQUAL 2017 is an attempt at such a reflection on scientific integrity and its various dimensions in a qualitative research process. [2]
I will first give a brief description of the seminar presentations, organized under the subthemes: what scientific integrity is, how it can be evaluated, what the factors affecting integrity during and after a study are, whether we should follow rules to ensure integrity, and how to teach integrity to students (Section 2). This will be followed by an account of the two group discussions (Section 3), which will lead us to the final, concluding remarks (Section 4). [3]
The seminar consisted of ten presentations, each one followed by a question and answer session. The main topics that were discussed included what scientific integrity is, how it could be evaluated, what the elements affecting scientific integrity during and after a qualitative study are, whether following rules is the best way to ensure scientific integrity, and finally how integrity can be taught to students of qualitative research. [4]
2.1 What is scientific integrity?
The first presentation was given by Martyn HAMMERSLEY. He stated that contemporary work on scientific integrity often conceptualizes a set of procedures, rules, or principles that abide by good research practices in the field. In contrast, the older conceptualization involves situationally appropriate value principles (MACFARLANE, 2009). HAMMERSLEY made a distinction between intrinsic (epistemic) and extrinsic research values, the former relating to the production of knowledge (for example the truth, justifiability, relevance, feasibility, and honesty of the research tasks) and the latter pertaining to ethical constraints in the pursuit of research (HAMMERSLEY & TRAIANOU, 2012). [5]
According to HAMMERSLEY, proceduralism and ethical regulations might pose a threat to the intrinsic integrity of research. When researchers focus too much on adhering to codes or committee approvals, the pursuit of value-relevant knowledge can be compromised (see also MERTENS & GINSBERG, 2009). Furthermore, focusing too much on the political and practical purposes and procedures of a research project can lead to the researcher having conflicting requirements and disengaging from academic dialog. The situation changes from one of academic engagement to one of rebuttal of criticism, which is counterproductive. HAMMERSLEY concluded that in this age, it is essential to be committed to scientific integrity; however procedural means are not the answer to the related threats. [6]
2.2 How can we evaluate scientific integrity?
Three of the presentations focused on ways to evaluate scientific integrity in qualitative research. Gillian SYMON discussed how contemporary publication pressures might contort the best practice and evaluative norms of qualitative research. SYMON argued that qualitative researchers may feel pressurized to accept and follow the guidelines dictated by reviewers, commentators, and journal editors of prestigious journals in order to achieve publication. She emphasized that this standardization is a threat to the diverse and eclectic nature of qualitative research practice. She pointed out that maintaining integrity in the face of institutional pressures for conformity is the problem at play here. In the second presentation focusing on this topic, Hubert VAN PUYENBROECK explained how, as a qualitative-oriented researcher, he makes choices within the spectrum of qualitative research methods and simultaneously opts for a valuable research framework. He referred to the concepts of scientific integrity such as trustworthiness, truthfulness, and accountability within a framework of qualitative research. [7]
The next presentation was given by Lakshmi Balachandran NAIR. She showed that iterative approaches like grounded theory methodology rely on several cycles of comparing empirical phenomena with theoretical inclinations until saturation is reached. The characteristic messiness of such iterative research can often make it difficult for the customers (i.e., readers, reviewers, and editors) to audit the research procedures—that is to say, to perceive and appreciate the chain of evidence and the boundaries between various stages of the study. NAIR examined the degree of sophistication and transparency of reporting in grounded theory methodology, namely the extent to which authors transparently account for research procedures so that reviewers, editors (intermediate customers of the article), and readers (end customers) are able to follow the whole research process. NAIR then built on literature in auditing to propose a new perspective to facilitate the appreciation of iterative research procedures by these customers, called "second party auditability." [8]
2.3 What are the elements affecting scientific integrity during a qualitative study?
Marit Helen HEM explicated how vulnerability of the researcher could be a potential source of reflexivity in qualitative research. From literature and her own experience, HEM opines that research participants are often considered potentially vulnerable and therefore protectable (DICKSON-SWIFT, JAMES, KIPPEN & LIAMPUTTONG, 2007). Given the situation, it is surprising that there is little thematization and contextualization of researcher's vulnerability within the field of research ethics. Using her long-term fieldwork on an acute mental health unit, HEM showed how the researcher's position changed during her interaction with a patient (whom she referred to as "William"). From an interested researcher grateful for the opportunity to study the situation, the researcher eventually became confused about her relationship with the patient/research participant. This made her focus on her reflexivity (self-observation; BRINKMANN, 2012). HEM focused on the importance of researcher vulnerability and suggested how self-observation and reflexivity are important in a qualitative research context. In particular, HEM suggested that the researcher should understand how his/her own self is both the subject and object in the process of observation (ibid.). She talked of three types of self in this context—the personal self of the researcher, the self as understood by the researcher, and the self the researcher becomes through the gaze of participants. She suggested that the researcher could use this awareness of self as a source of knowledge of oneself, the participants, and the relationship between both. [9]
In her presentation, Bareerah HOORANI examined the role of ethical climate in how PhD and postdoc students in Switzerland ensure transparent qualitative research. By assessing the organizational ethical context using a deductive ethical index, she looked into both self-focused moral reasoning and other focused moral reasoning among Swiss academics. The results indicated that a more ethical research climate does indeed lead to more transparent reporting. [10]
2.4 What are the elements affecting scientific integrity after a qualitative study?
This work by Michael GIBBERT and co-authors given in the seminar (see also GIBBERT, NAIR, WEISS & HOEGL, 2014) explored the role of including outliers or deviant cases in ensuring scientific integrity and rigorous theory building. Through an analysis of all articles published in five major management journals over twenty years, the authors found that less than 5 percent of all articles published in that period mention outliers at all, and only five articles use them for theory building. Drawing on the methodological literature as well as concrete examples from their database, GIBBERT et al. provided a roadmap for rigorous theory building based on outliers by discussing which outliers are most promising theoretically and how this potential can best be used. The effective gains and losses of theory development from outliers were discussed in terms of exemplar articles: their primary value resides in providing the starting point for theoretical progress, by extending theories' purchase on empirical realities. [11]
In his presentation, Alrik THEIM discussed how deliberate suppression of research findings or publication bias could not only be due to vested interests of authors, editors, or project sponsors, but also due to methods used. THEIM discussed this in the context of qualitative comparative analysis (QCA), where the biases occur due to data supplementation (use of artificial data along with empirical data), theory confirmation (manual selection of implicants based on theoretical expectations), and model simplification (use of minimal number of implicants). THEIM explained his empirical project where he and a co-author attempted to replicate QCA studies to further explore this issue. [12]
2.5 Is following rules the best way to ensure scientific integrity?
Giampietro GOBO discussed the two conflicting roles of methodology: the political (external) and the procedural (internal). While the political role helps science in legitimizing its results, the procedural role helps researchers during the research process of generating knowledge. From the times of Galilei, researchers have disrupted overt methodological rules for organizational reasons. GOBO suggested that setting up new roles and evaluation criteria might have the same effect as these former rules. As an alternative, he proposed a more sustainable methodology based on organizational constraints, ad hoc activities, everyday conventions, and tacit agreements. [13]
2.6 How can we teach scientific integrity to students?
Gerben MOERMAN discussed the role of integrity in teaching research methods. For him, most social research method books and courses discuss integrity quite implicitly (as a chapter or a section) rather than explicitly and extensively. He made explicit that the inherent subjectivity and reflexivity in qualitative research makes the pursuit of integrity even more difficult. MOERMAN gave an example of a teaching approach where he extensively discussed instances of integrity transgression and fraud in the context of a fundamental discussion on realism, reflexivity, and the crisis of representation. MOERMAN then discussed two specific cases of scientific misconduct that happened in the context of Dutch academia: the case of data faking by social psychologist Diederik STAPEL and the case of anthropologist Mart BAX fabricating evidence in over 60 papers. [14]
3.1 "We know what does not work. But we do not know what works"
During the first group discussion, the participants discussed threats to integrity and how we can move forward as qualitative researchers. The focus of the discussion was then on why we are looking for a strong methodology and whether we are fixating too much on methodological integrity. A participant mentioned how qualitative research, although not the most prominent methodology in such fields, is often conducted in life and medical science research. At the same time, interviews conducted in a medical or clinical setting are often smaller in number and are subjected to lesser rigorous standards when compared to social sciences. However, this has not been perceived as a problem in these fields so far. The discussion was about why it is perceived as a huge problem in the field of the social sciences. [15]
The participants also put forth that to judge someone else's work based on one's own perceptions of integrity is a big problem. There is a lack of communication between different camps in qualitative research. A good way to resolve this would be to revisit the concepts and find common ground. Some pragmatic solutions offered involved the use of an external person to analyze the integrity of a study. Archiving qualitative data for checking and analyzing by future researchers was also suggested as another way of ensuring integrity. The latter suggestion raised further questions on whether it is advisable or useful to analyze or use someone else's data. [16]
3.2 "Think about mundane activities and build a methodology based on it"
In the second group discussion, the participants mainly focused on the concept of sustainable methodology raised by GOBO. The core idea was to discuss the revisiting of traditional concepts from a practical point of view. The discussion called out for a decolonization and subsequent glocalization of the concept of methodology characterized by a community of practices rather than a rigid set of rules. It was not to suggest that researchers abandon methodology, but rather use situational ethics and contextual factors to guide decision making in a research process. [17]
The SCIQUAL seminar addressed a range of issues relating to scientific integrity as well as the need for defining it in terms of concepts like transparency and auditability. The participants and the presenters brought different perspectives to the table. This ensured active deliberation during the group discussion session. Despite the different opinions, all attendees agreed upon scientific integrity being an important issue to focus on in qualitative research. As renowned chemistry researcher and President of John Hopkins University Ira REMSEN (1904, p.9) commented: "Absolute accuracy, absolute fidelity, absolute honesty are the prime conditions of scientific progress." [18]
As a further step to SCIQUAL, the organizer and some of the participants are planning to offer a workshop entitled "Research Ethics and Integrity in Qualitative Research: Opening Perspectives at the Forthcoming European Congress of Qualitative Inquiry (ECQI) 2018" at Leuven, Belgium. [19]
1) See http://www.retractionwatch.com [Accessed: November 2, 2017]. <back>
Brinkman, Svend (2012). Qualitative inquiry in everyday life: Working with everyday life materials. London: Sage.
Dickson-Swift, Virginia; James, Erica; Kippen, Sandra & Liamputtong, Pranee (2007). Doing sensitive research: What challenges do qualitative researchers face? Qualitative Research, 7(3), 327-353.
Gibbert Michael; Nair, Lakshmi Balachandran; Weiss Matthias & Hoegl Martin (2014). Oops, I've got an outlier in my data—What now? Using the deviant case method for theory building. Presentation, Academy of Management Annual Meeting, Philadelphia, PA, USA, August 1-5, 2014.
Hammersley, Martyn (2017, in press). Is phrónēsis necessarily virtuous? In Nathan Emmerich (Ed.), Virtue ethics in the conduct and governance of social science research. Bingley: Emerald.
Hammersley, Martyn & Traianou, Anna (2012). Ethics in qualitative research. London: Sage..
Macfarlane, Bruce (2009). Researching with integrity: The ethics of academic inquiry. London: Routledge.
Mertens, Donna & Ginsberg, Pauline (Eds.) (2009). The handbook of social research ethics. Thousand Oaks, CA: Sage.
Remsen, Ira (1904). Scientific investigation and progress. Science, 19(470), 1-11.
Snee, Helene (2008). Web 2.0 as a social science research tool, https://pdfs.semanticscholar.org/336d/7b8706ea7b870eb36e2e81be16a001f50048.pdf [Accessed: October 3, 2017].
Williams, Matthew; Burnap, Pete & Sloan, Luke (2017). Towards an ethical framework for publishing twitter data in social research: Taking into account users' views, online context and algorithmic estimation. Sociology, 1-20, http://doi.org/10.1177/0038038517708140 [Accessed: October 3, 2017].
Dr. Lakshmi Balachandran NAIR is assistant professor specializing in qualitative research methods at Utrecht University, the Netherlands. Her work explores the rigor, transparency, and impact of research methods (qualitative, specifically) in management and other social sciences. Lakshmi's articles on methodological sophistication of research methods have been published or accepted for publication in journals (Journal of Business Research, Qualitative Research in Psychology, Scientometrics, Journal of Management Inquiry) and proceedings (Academy of Management Proceedings), as well as other publications (Sage Research Methods Cases). Lakshmi has also co-authored a book where she contributed case studies on companies undergoing strategic innovation ("Strategic Novelty: A Definitive Guide to Outlier Strategies", Financial Times Press, 2015).
Contact:
Dr. Lakshmi Balachandran Nair
Methodology and Statistics Department
Utrecht University
Sjoerd Groenmangebouw, Padualaan 14
3584 CH, Utrecht, the Netherlands
Tel.: 0031682259649
E-mail: l.balachandrannair@uu.nl
URL: http://www.lakshmibnair.com/
Nair, Lakshmi Balachandran (2018). Conference Report: Scientific Integrity in Qualitative Research (SCIQUAL) Seminar 2017 [19 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 19(1), Art. 9, http://dx.doi.org/10.17169/fqs-19.1.2964.
Creative Commons Attribution 4.0 International License