To Use or Not to Use: Computer-Assisted Qualitative Data Analysis Software Usage among Early-Career Sociologists in Croatia

Abstract: In this article we critically analyse the usage of computer-assisted qualitative data analysis software (CAQDAS) among early-career sociology researchers in Croatia. In Croatia, the CAQDAS community is very small and is dominated by problems of great expectations coming from early-career researchers. This is elaborated through a case study that addresses issues of spread and availability of CAQDAS, opportunity for its usage, reasons why early-career researchers decide to use it or not and grounds for their decisions for use of a particular software package. The perceived advantages and limitations of CAQDAS are analysed and some misconceptions about CAQDAS are contextualised and related to the dominant quantitative research framework in Croatian sociology. The broadening of the number of qualitative researchers in the sociological community in Croatia, together with more educational programmes on CAQDAS, which would highlight reflexive usage of software, open perspectives for strengthening of qualitative research in Croatia.

Key words: computer-assisted qualitative data analysis software; CAQDAS; Croatian sociology; case study; mixed-methods approach; interviews; online survey

"The first and foremost point to make about the use of computers in qualitative analysis is that computers do not and cannot analyze qualitative data."
(ROBERTS & WILSON, 2002, §21)

Over the last thirty years the development of computer-assisted qualitative data analysis software (CAQDAS) has gone through many changes—from the development of separate software development projects in the mid-1980s and early 1990s, the connecting of the CAQDAS community in the 1990s, towards the rise of the meta-perspectives in the 2000s and finally to developments of CAQDAS 2.0, that is, the trend towards integrating qualitative data analysis software with Web 2.0 collaborative online platforms (DI GREGORIO, 2010). In this latter phase, one can notice that the number of users is rising (based on the number of different CAQDAS courses) and there are more non-academic users (commercial agencies, public sector, NGOs) of the programs (MANGABEIRA, LEE & FIELDING, 2004, p.172). Nevertheless, although the field is being better established, there is still scepticism about the use of CAQDAS. This scepticism points to the overall "positivistic epistemological position" in the creation of software. In this way, ROBERTS and WILSON (2002, §5-6) argue that "[c]omputing technology assumes a positivistic approach to the natural world that sees it as being composed of objects that humans can study, understand and manipulate", but "the goal of qualitative researchers is to try and see things from the perspective of the human actors". Therefore, the main concern is that implicit assumptions of the software architecture will interfere with the qualitative research process and will result in the loss of shades of meaning and interpretation that qualitative data bring. [1]

The other concern tackles the way that computers mediate interaction between the researcher and the qualitative data (WEITZMAN according to BRINGER, JOHNSTON & BRACKENRIDGE, 2004, p.250). There is an implicit assumption that interaction with the data is somehow more "natural" when paper and pencil are used. However, the fact remains that computer mediation is present even when using the most basic word-processing software and is nowadays an unavoidable part of interaction with the data. In addition, there are many debates on the issue of how to assess quality in qualitative research, and to what extent concepts of validity, reliability and generalisability are applicable to qualitative research (FRIESE, 2010; GIBBS, FRIESE & MANGABEIRA, 2002; GOBO, 2008), and how (and if) CAQDAS contributes to these processes. [2]

In the last couple of years, one could notice the beginning of CAQDAS usage in the Croatian social science community. In sociology, this emerging community is based, on the one hand, on a few experienced researchers who use qualitative methods in their research, as well as, on the other hand, on researchers at the beginning of their careers who have opted to use qualitative methods in their PhD research projects. What is interesting is that the number of younger researchers who use qualitative methods in their PhD research is on the increase and consequently the interest in CAQDAS is growing. [3]

However, this interest is mediated by different factors influencing potential software users' choice of whether to use CAQDAS or not. There is an issue of availability of the resources needed for software use: the cost of the software package itself, but also the availability of educational resources and peer advice. Furthermore, there is a dilemma regarding the potential costs and benefits of software usage which influences the motivation to put the initial effort into learning how to use it. Therefore, we conceptualise the overall environment of CAQDAS appropriation as consisting of three main factors: 1. methodological trends within the sociological research community; 2. attainability of resources; and 3. perceived cost and benefits that the software could bring to data analysis and to the overall research process. [4]

In the following sections we will first outline the historical context of the development of qualitative methods in the sociological field in Croatia. Secondly, after giving an overview of methods used in our research we present the results through two main issues—the spread and reasons for (not) adopting CAQDAS and experiences with CAQDAS usage. These issues are further elaborated in the discussion and conclusion of the article. [5]

Although qualitative methods have been used in Croatian sociology since the 1970s, the usage has been peripheral. Indeed, there was and still is reluctance toward the acceptance of qualitative methods alongside their quantitative counterpart.¹⁾ As the research by VUČKOVIĆ JUROŠ (2011, p.166) shows, in the period between 2000 and 2009 in seven Croatian social science journals only 43 out of the total number of published articles used qualitative research methods: "More than half of these articles were sociological, while others were mostly published from the perspectives of social work, pedagogy and related disciplines". She did not mention the total number of articles. An estimation based on a sample of volumes for every journal included in research by VUČKOVIĆ JUROŠ shows that approximately 1,700 articles were published from 2000 to 2009. Therefore, we can conclude that only approximately 2-3% of published papers were based on qualitative methods. [6]

The conference Qualitative Transitions in Rijeka in 2010 was the first larger event that gathered together Croatian researchers and their European guests in order to discuss pertinent issues of qualitative research in Croatia. The establishment of a qualitative methods strand of the Croatian Sociological Association in February 2011 is also a contribution to the further development of qualitative research. [7]

The orientation towards quantitative methods is also evident in the university curriculum.²⁾ The oldest sociology department in Croatia—Department of Sociology at the Faculty of Humanities and Social Sciences, University of Zagreb—has focused primarily on quantitative methods, while considerably less attention has been given to qualitative research. The reason for this is that the department was established in 1963, during the heyday of quantitative methods in social sciences worldwide. Additionally, the founder of the department, Rudi SUPEK, was a psychologist by education. Because Croatian psychology was then an established discipline³⁾ with a hard-sciences background and orientation, SUPEK used it as a paragon while developing the Department of Sociology. His book "Ispitivanje javnog mnijenja" ["Public Opinion Research"], a quantitatively oriented methodology textbook published in 1961, made a huge impact on methodological developments at the Department. Nowadays, there are eight quantitative methods courses⁴⁾ and only one qualitative methods course at the undergraduate and graduate levels ("Qualitative Research Methods").⁵⁾ [8]

In 1977, the Department of Sociology was established at the Faculty of Philosophy in Zadar, and in 2005 the Department began its new programme in sociology, within the framework of the Bologna process reform. The new programme was more open towards qualitative methodology as well. At the time we conducted this research, at the undergraduate and graduate levels, the following courses in qualitative research methods were taught: "Qualitative Research Methods", "Qualitative Field Research", "Visual Analysis" and "Media Research and Methods".⁶⁾ The Department of Sociology at "Studia Croatica" (Center for Croatian Studies), University of Zagreb, is a relatively new sociological department, established in 1996. At the time of concluding our research it offered only one course in qualitative methods ("Qualitative Research Methods") at the undergraduate and graduate levels although the special research line in cultural anthropology in the same programme offers an insight into qualitative methods as well. The Department of Sociology at the Faculty of Philosophy, University of Split, was established in 2005 and currently offers the highest number of qualitative research courses (undergraduate and graduate levels included): "Qualitative Methodology", "Media Research and Methods", "Qualitative Analysis in Social Sciences", "Culturological Research and Methods", "Ethnographic Methods in Sociology" and "Ethnographical Approaches in Sociology". In 2012, the Croatian Catholic University (established in 2010) opened the Department of Sociology, which centers on quantitative research in sociology, but also offers courses in qualitative methods such as "Qualitative Methodology" at the undergraduate level, and "Qualitative Methods: In-Depth Techniques", at the graduate level. [9]

Until recently, the PhD programme in sociology ("Postgraduate Doctoral Programme in Sociology", "PDSS" with the Croatian acronym) was taught at the national level at the University of Zagreb, with the cooperation of all universities and several research institutes.⁷⁾ Regarding qualitative research methods, it includes two courses—"Grounded Theory" and "Mixed Methods Research". In 2011, the University of Zadar opened a joint degree doctoral programme with the University of Teramo, Italy, in "Sociology of Regional and Local Development" that explicitly stresses qualitative methods. [10]

The indication that qualitative methods are on the rise in Croatia, especially amongst the younger generations of researchers, can be found in their research methods choices. Since 2008, the number of defended PhD theses in sociology which have used qualitative methods in research equals the number of theses based on quantitative methods.⁸⁾ It can be hypothesised that this choice in methods could have something to do with the financial constraints that most young researchers encounter.⁹⁾ [11]

Within the sociology programmes at Croatian universities, CAQDAS is taught only at PhD level at the University of Zagreb, as part of the "Grounded Theory" course, and at the University of Zadar in the second year of the master's programme. It can be observed that although more and more young researchers use CAQDAS and qualitative methods for their own projects, both are not adequately present in Croatian higher educational context.¹⁰⁾ This situation echoes an article on qualitative research in Slovenia by ADAM and PODMETNIK, who elaborate on the marginal status of qualitative methods in Slovenian social sciences: "This marginality is above all caused by weak institutionalization and insufficient inclusion of qualitative approaches in (post)graduate curricula"(2005, §13). The fact that the younger sociological departments outside of Zagreb are more open towards qualitative methods can be seen as a positive move. Nevertheless, the central role of the University of Zagreb in sociology in Croatia cannot be ignored and thus more adequate inclusion of qualitative methods in its curricula should further strengthen the rise of qualitative methods, as the growing interest in qualitative research is not being addressed by the courses offered at this time.¹¹⁾ [12]

"Paradoxically, the 'softer' a research strategy, the harder it is to do."
(YIN, 1994, p.16)

As noted earlier, CAQDAS has not been present within sociology curricula and, up until 2012, with several exceptions was not in use among senior sociologists or beyond the academic context. It should be mentioned that the first author of this article periodically held introductory CAQDAS workshops for postgraduate sociology students at the Faculty of Humanities and Social Sciences in Zagreb. Since 2008, there have been four such workshops; three of them were organised as a part of the "Qualitative Research Methods—Grounded Theory" course and one was organised at the same faculty as part of a postgraduate sociology students' conference in September 2010. Recent generations of PhD students who attended CAQDAS workshops were the first to have been exposed to software learning. It should be mentioned that the majority of these PhD students also work as researchers in (sociology) departments at either faculties or institutes across Croatia. Therefore, they are the focus of our research. As this was the first research on CAQDAS in Croatia, the research was of an exploratory character. We aimed to describe the spread of CAQDAS usage (the latest data used is from March 2012) and then to address factors influencing CAQDAS appropriation: contextual influences structuring the opportunities for CAQDAS adoption and researchers' views on costs and benefits of software usage. The aim of a detailed analysis of a decision-making process was to find out whether the decision to use CAQDAS or not is grounded in arguments that are related to CAQDAS capabilities to facilitate research processes. This we will address as the "internal" reasons, as opposed to "external" reasons, such as software availability and affordability and presence of CAQDAS users within the professional network. [13]

In order to assess the situation with CAQDAS usage in Croatian sociology, we conducted a case study on usage patterns among early-career sociologists in Croatia. The case study consisted of desk research, two online surveys of postgraduate sociology students (in 2010 and 2012) who attended CAQDAS workshops, and semi-structured interviews. Thus, our research process had two phases: In the first one (June-September 2010), we conducted desk research, a short online survey, and interviews with early-career researchers who have used CAQDAS in their research, and who were selected using the snowball method. During the second phase (February 2012), the online survey was repeated, two additional generations of postgraduate sociology students were included in our sample and additional desk research was conducted. [14]

Our design was based on an approach which aims to grasp the ideas and values behind decisions and participant evaluation of the social process, experience or outcomes (KARDORFF, 2004; PATTON, 2002; RITCHIE, 2003, pp.29-30). We decided to use a pragmatic mixed-methods approach to match methods to aspects of the researched issue and also in order to achieve more credible analysis by methods triangulation (FLICK, 2004; PATTON, 2002, pp.556-559). The matching of methods and issues is summarised in Table 1. The rationale behind the usage of a particular method and the details on procedure are explained in the following paragraphs. [15]

The desk research was primarily needed to provide the historical context (presented in the previous section) and to put the beginnings of the CAQDAS usage in Croatia within the perspective of general methodological trends. Although some insight into the spread of CAQDAS use was attained through desk research, most data was gathered through surveying PDSS students, that is, early-career researchers. We conducted the online survey among the PDSS students who attended four CAQDAS workshops from 2008 to 2011 in order to see whether those who have been exposed to software learning adopted it in their research afterwards. Furthermore, we wanted to determine whether someone else within their professional peer-group uses the software, so as to map CAQDAS users beyond the PDSS student population. The surveys were not anonymous because they were used as a source for approaching participants for the interviews through the snowball method. Later on this provided us also with an option for longitudinal comparison. The names are confidential and will never be used in research reporting or otherwise disclosed. This was made clear to respondents before participating in the online survey and also when approached for the interviews. As they are the only sociologists who had a chance to receive CAQDAS training in Croatia, they represent the population in our research. In addition, the survey was used to explore the reasons for (not) adopting software and grounds for deciding for a particular software package.¹²⁾

Table 1: Research themes and methods overview. Please click here to access the PDF document [16]

The short online survey was carried out via the Google Docs application and e-mailed to postgraduate sociology students of the PDSS who attended the CAQDAS workshop during their doctoral programme. The first two generations of students who took part in these workshops (2008 and 2009) were invited to complete the online survey in June/July 2010 and all four generations of students were sent a survey invitation in February 2012. As noted earlier, they are the only sociologists who had any kind of CAQDAS training in Croatia. Out of 31 respondents who received the questionnaire in 2010, 18 answers were received. In 2012, 47 survey invitations were sent and 31 students responded. Eleven persons answered on both occasions, so we could track the change (if there was any) in their CAQDAS usage over time. The questionnaire contained 17 questions and addressed the following issues: 1. spread of CAQDAS (software availability, whether respondents use it and if they know someone else who does); 2. opportunities for its usage; 3. reasons they decided to use it or not; and 4. grounds for their decisions to use a particular software package (for the themes they covered see Table 1). [17]

In order to find those researchers who actively used (or/and still use) CAQDAS in their research, we approached survey respondents who said they used CAQDAS and, using the snowball method, found two additional participants. We thus had a sample of six researchers with whom we conducted semi-structured interviews, sometimes referred to as the interview guide approach (PATTON, 2002, pp.343-349). The interview guide consisted of three parts: 1. three questions on the decision-making behind choosing the software package; 2. four questions about the methods used in the research facilitated by software (in all cases these were PhD theses); and 3. questions on the software usage practices and impacts on research process. The latter consisted of questions on how they used the software, whether the software influenced the research process (and in what way) and whether they felt that software determined any of the crucial research aspects, for instance, conceptualisation (see Table 1).The initial interview questions sequence and wording were adapted according to the progress of the particular interview, but interviewers took care that all the issues from the guide were explored (pp.343-344). The interviews lasted from 35 minutes to one hour and 15 minutes and were conducted in Croatian. They were recorded digitally and transcribed by F4 software by the interviewers; as additional tools we used MAXQDA and Excel. [18]

Thus, to explore the early-career researchers' usage of CAQDAS, we coded the interviews starting from three broad themes in the interview guide (SALDAÑA, 2009, pp.66-70): reasons behind choosing the package, research methods used by respondents and the software usage practices and impacts on the research process. In an iterative process of concept-driven and data-driven thematic coding (GIBBS, 2007, pp.44-45) we arrived at a more refined codebook covering different particular issues referred to by respondents. We coded different aspect of the themes (SALDAÑA, 2009, pp.139-145), exploring the data further. For example, we coded different reasons behind choosing a particular package. With regards to the experiences with CAQDAS usage, we applied evaluation coding (PATTON, 2002; SALDAÑA, 2009, pp.97-101), differentiating between advantages, obstacles and other kinds of impact on research process, which is reported in Section 4.2. [19]

In Section 4.1 we present both the survey and interview results concerning the spread of and reasons for (not) adopting CAQDAS (see Table 1). Questions regarding reasons for not using CAQDAS were included in the surveys, as survey respondents included those students who had not used CAQDAS after the workshops. In interviews, on the other hand, the focus was on the reasons for using it, as all interviewees were software users. [20]

Additionally, through interviews we wanted to gather more detailed insight into user experiences with CAQDAS, which is the theme of Section 4.2. We were interested in issues regarding software preference, methodological choices and type of empirical data (e.g. textual, graphic, interviews, multimedia, etc.). Furthermore, we wanted interviewees to reflect on whether the software influenced the research process and, if it did, in which ways this was evident. [21]

Regarding the spread of CAQDAS, the survey showed that there are very few institutional CAQDAS licences in Croatia.¹³⁾ In 2010, there were two project-related multiple-user licences (with two respondents working within the same research team), while one researcher worked with an individual licence that was funded by the research project. In 2012, there were two institutional licences and one project-based multiple-user licence. Institutional licences were bought by two faculties (Faculty of Humanities and Social Sciences and Faculty of Law—Study of Social Work, both at the University of Zagreb) and this indicates that future generations of students at these institutions will probably come into contact with CAQDAS during their graduate studies.

The number of respondents who have colleagues using CAQDAS increased during the research period. At the same time, the number of students who actually used CAQDAS in their own research after the workshop was rather low. But all in all, cumulatively the number of users is increasing. [23]

The reasons why students did not use software included opportunities, availability and other perceived obstacles. As far as opportunities for CAQDAS usage are concerned, both in 2010 and 2012 the majority of respondents confirmed that they would use CAQDAS if the institution where they worked had a licence (Figure 3). Both times many students answered "maybe", which indicates that the decision to use or not to use CAQDAS is not often made in advance, but is made according to the particular research project situation. On both occasions approximately half of the respondents stated that there was neither a need nor an opportunity to use CAQDAS after the workshops, which can be attributed partly to the fact that a number of the respondents had not started their PhD research projects yet.

Figure 3: Would the respondents use CAQDAS if their institution had a licence? [24]

Furthermore, we wanted to understand the reasons of those students who decided not to use CAQDAS were, even though they had an opportunity to do so. We provided two answers—software prices and learning effort—expecting them to be the most prominent, which was confirmed by the results. We also gave an option of an open-ended answer which provided us with two particular project-related answers. Regarding the relation of opportunity and actual usage, it can be reported that those who have an opportunity to use CAQDAS mainly decide not to use it because of the perceived costs—whether financial or the "cost" of time needed for the initial learning effort which is regarded to be too high when weighted against the perceived benefits of software usage. [25]

Reasons for adopting CAQDAS were not part of the survey, but were included in the interviews. These were mentioned in the discussion on why interviewees chose a particular software package. They were asked to compare the option of using CAQDAS to alternatives (e.g. paper-and-pencil, word-processor). Most of the interviewees pointed towards various ways in which software facilitates the research process making it "more convenient", "quicker", "more systematic" and "easier". The most often mentioned reasons were "convenience in data analysis", "helps document research process", "data systematisation", "visibility of coded segments". Additionally, one of the interviewees mentioned two not so obvious reasons: "a kind of fashion and curiosity of work with this type of software"(interviewee I5, par.6¹⁴⁾). The most interesting reason mentioned, by the same interviewee, was that the software usage gives an aura of credibility to the research:

"With qualitative data it is always a question of their value in relation to quantitative data. So I think that this concerns usage of software, however awkward it may sound, but in a way this gives credibility to it. When you say that you have done qualitative data analysis in NVivo, and some people have never even heard of it, then it gives it some kind of relevance, although I think this certainly cannot be the only reason for using it" (I5, par.6). [26]

We will turn back to the issue of credibility in Section 4.2.1, as other respondents mentioned this point while discussing their experience with software usage. [27]

In the survey, we had two follow-up questions for those participants who said that they used CAQDAS in their research. First, we asked which software package they used and afterwards we asked the open-ended question: "Why did you choose that particular software?" In Table 2, we present merged answers from both the open-ended survey question and from interviewees' accounts in answer to the interview question: "Why did you use [software name] in particular?" followed by the additional question: "Besides the reasons you mentioned, were there any additional reasons for your decision to use [software name] in particular?" By thematic coding of the interview answers we arrived at the codes presented in Table 2. Because we used the survey as a pool for subsequently recruiting interviewees, selected survey respondents were also interviewed. Answers from two additional persons we did not interview from the 2012 survey, fit into the category framework that we inductively created from the interviews; these are presented besides interviewees in Table 2 (S1 and S2).

The most frequently used programs were Atlas.ti, MAXQDA and NVivo. However, the characteristic of a specific software package, being either the properties of the software packages or the price/availability, seem to be of less importance compared to the influence of the learning and work environment. Among the particular technical capabilities, respondents mentioned a portable/USB installation (MAXQDA; I1), good PDF support (Atlas.ti; I3) and a more secure single MAXQDA project file in comparison to the external file storage in Atlas.ti, which caused data loss for a colleague (I4).¹⁵⁾ When asked about the background of their decisions regarding the software, four of our respondents mentioned that they had come across CAQDAS within the educational context (two of them abroad, in the UK and USA), which was the most important guideline in their software choice afterwards for their own research projects. In addition, most of the interviewees mentioned the importance of colleagues' recommendations. To illustrate this point we will quote one of our interviewees:

"Firstly, I had access to it because I saw that other people used it and then they organised a couple of workshops on how it is used. So basically it was something that I was best acquainted with and everybody else said that it was great. [pause] Therefore, I haven't seen any others, but everybody else claimed that this one was better than any other" (I3, par.6). [29]

So it seems that the decision to use the software was more influenced by the factor of availability (most importantly, of introductory-level education or peer assistance with its usage), than clear preconceptions of its capabilities or clear expectations, i.e. in which way the software can add to the quality of their analysis and research report. [30]

Although there was a small number of respondents who had the opportunity to use CAQDAS in their research (given that many of the students had only begun their PhD programme), it seems that there is some basis for a tentative conclusion. It can be said that when students decide not to use CAQDAS it is primarily because they do not perceive that there are clear and valuable benefits to spending their time learning the software. It can be argued that the reasons for not using CAQDAS are obstacles "internal" to the software: if software was easier to learn and more user-friendly, they would change their decision. On the other hand, those who did use CAQDAS mostly used "external" references (e.g. colleagues' recommendations or even "a fashion", as one interviewee mentioned) when deciding on a specific software package. None of the survey respondents mentioned any argument related to methodology, research approach or research credibility or validity. Only in interviews, when explicitly asked to think about advantages of software usage, did respondents mention arguments regarding advantages "internal" to the software itself. [31]

Taking into account the arguments for and against CAQDAS usage, we have to note that their implications partly depend on usage patterns. Arguing that typical modes of users' approaches can be identified, MANGABEIRA et al. (2004) found four types of users of software.¹⁶⁾ We can say that in Croatia we are still experiencing beginner's difficulties; the community of users is still in the making. Our sample of six interviewees cannot be used as a basis for any kind of typology, of course. Nevertheless, we can notice a tendency of our respondents to take, as MANGABEIRA et al. (p.170) call it, "loyalists" attitudes, because the initial experience with one of the software packages seems to be most important for the decision on the specific software package. When someone learns to use one software package, of course, learning another can easily be seen as the loss of precious time, so we can speak of a kind of "community of followers" of a specific software package. They rarely had the time or motivation to explore various capabilities and to compare differences between several software packages.¹⁷⁾ [32]

The results of the online survey confirm these findings; the survey showed that the most important influence for deciding whether to use software or not was the advice of colleagues and initial introduction to software within a PhD course. The survey also showed that many students believed that they would use CAQDAS if programs were available at their institutions, even before they start actually working on their PhD theses, although a third of them were undecided. Nevertheless, the experience of our interviewees shows that, as far as PhD students are concerned, the decision to use software eventually has to be made with only limited information of software capabilities. Colleagues' recommendations, introductory educational workshops and reading/watching tutorials give an overview. However, only when he/she starts to work on the data he/she gets to know the positive and negative aspects of the program, especially in later phases when more advanced functions come into focus. [33]

In this light we should stress that it is important to make an informed choice regarding CAQDAS, as CARVAJAL (2002) has elaborated. One has to bear in mind that the advice of our colleagues sometimes tends to be biased. On the one hand, we sometimes find communities of users loyal to some specific software which would always recommend only that one and not another. On the other hand, there are many misunderstandings about CAQDAS that must be fought, which mainly "represent a quantitative approach to the use of computers in qualitative analysis" (§12), that we will also deal with in the following section. CARVAJAL stresses that courses on CAQDAS have to include more information and promote critical thinking and the analysis of software and its methodological implications for qualitative research. Thus, more education not only on CAQDAS is needed, but also on qualitative research methods in general. [34]

Analysing the respondents' answers, we arrived at the following advantages of CAQDAS usage: The advantage most often mentioned was "better data overview and access", with all but one interviewee referring to it (and some of them more than once). The other categories we found, all referred to by three different respondents, were: "easier coding and recoding", "facilitation of analytic process", "eases workflow and research material organisation" and provides "more serious/professional work outlook", while two gave a general evaluation that it is "simple" to work with software. [35]

One of the more interesting points the interviewees raised was that using software provided face value of doing "more serious" work. One interviewee compared work with CAQDAS to a paper-and-pencil approach:

"So these methods that I used before... Maybe there were other ways, but these that I was acquainted with, were really unpractical ... All these papers and cutting, and gluing them ... It was like some process in primary school. It was simply much less professional ... I think that using the software adds to a kind of professional outlook, of data presentation, too" (I5, par.26). [36]

Another issue stressed by our interviewees was that the usage of CAQDAS brings a kind of exactness to the analysis. This notion is found in a bit vague idea of getting closer to "the standards" of quantitative analysis, which are generally perceived as more exact, precise, reliable and reproducible:

"By using NVivo, I wanted to obtain, let's call it this way, a positivistic confirmation of my qualitative aspirations ... [laughter] So ... I am usually prone to other types of research and this was the first time that I personally worked on qualitative research" (I2, par.22). [37]

Or, more explicitly:"Atlas.ti helps with the exactness of data results" (I6, par.33).We find this "exactness argument" very intriguing, as it can be related to the ongoing discussion on whether time-saving "technical" advantages regarding data organisation, coding and retrieving could be regarded and intentionally used as tools for making the research more systematic, and conclusions more valid, and the research process more explicable and transparent.¹⁸⁾ From the perspective of the interviewees the perception of important others who evaluated their work was the primary reference with regard to this issue. They saw gain in the face value of using software. The fact that they used software, and not the issue of how they used it to improve the quality of the research, was perceived to have implications for the way the academic public evaluated their research. [38]

User-unfriendliness of software packages and learning time and effort were the two most prominent perceived obstacles found in our data. Some aspects of user-unfriendliness (e.g. clumsiness of some outputs, the need to find a workaround to get a specific analytic query, the need to supplement the work with paper-and-pencil/word-processor, to mention the most cited ones) were reported by all but one interviewee. Also, half of the interviewees reported some kind of technical obstacles, such as crashes of software. The second most prominent type of obstacle mentioned by four interviewees was connected to the amount of learning time needed in order to grapple with specific software. Related to this, we should mention that two interviewees explicitly stressed that the software they used was not intuitive. [39]

Beside these, two additional types of difficulties were mentioned (by two interviewees). We shall discuss them here, as we find them interesting from the analytical standpoint: too great expectations and inclination towards accepting a particular "software's analytic approach". Fresh users approached CAQDAS with the expectation that it would somehow analyse data for them, but then realised the differences: "Basically, it cannot do anything. It cannot do for you the most important things. Unlike SPSS which does a certain data processing" (I3, par.26). [40]

Given these expectations, it is understandable that some of the interviewees were not satisfied:

"I have expected more from NVivo and it drove me crazy for months. I thought to myself—why on earth haven't I taken a normal survey, put the results in SPSS, taken variables out and said, 'folks, here are the results'?" (I2, par.35) [41]

Of course, as our introductory quote reminds us, this simply cannot be the case. It is the researcher who analyses the qualitative data, not the software. That clash of expectations regarding who is the agent responsible for the analysis—researcher or software—was reflected in an account where an interviewee said that "NVivo's insistence on more lexical and semantic text analysis" led to the "thousands of pieces of nonsense that could not fit into any sensible analysis" (I3, par.31). This issue is further elaborated in the following section. [42]

When directly asked if software did somehow interfere with the research process or results, some of the interviewees insisted that it did not, while others mentioned obstacles they encountered when the software could not do something they wanted it to do (for instance, display more complicated selections of data in a clear and visible way). These situations were then faced as riddles to be solved, which were seen as a part of the process of learning to use the software. "For me gaining information and the analytical process are inseparable. That is, when I am trying to solve a particular problem then I am looking at whether Atlas is capable of doing it" (I3, par.79). [43]

As it presents learning of the software through taking steps in an analytic procedure, it can be interpreted as an instance of learning-by-use (CASTELLS, 1996, p.31) which is a typical feature of contemporary technology, as opposed to the learn-then-use approach, typical for older, less interactive technologies (e.g. TV). However, these unexpected moments of learning slowed down the analysis, as one interviewee noted. Some interviewees stated that they did not have enough time to "tinker" with software capabilities, such as tools for lexical analysis that they thought could be useful for broadening their overall analysis. On the other hand, this "tinkering" approach carries a risk of interfering with the research processes. An interviewee noted how such risks were recognised and intercepted:

"In the beginning I made attempts at tinkering, in order to see what the possibilities are (e.g. code line, some options with visuals), but then my tutor explicitly forbade such approach [laughter]. Which is OK, as I think that software has to facilitate the analysis, and not guide it, direct it" (I4, par.32). [44]

The opposite case was found in another interviewee account (I2, par.25) of using "NVivo's logic" that she learned from examples in manuals; although, when asked if software interfered with analytic processes, she claimed that it definitively did not. Students reacted negatively to the suggestion that software influences their research process. Obviously, this would suggest that they are not on top of their own research; writing the PhD thesis is a test of their competence for independent research. At the same time, throughout the interviews they did mention instances that, to various extents, contradicted their own claims. It seems that only when they faced disruptive moments—be it the need to find "how software can do it", or the mentor's intervention—did they consciously reflect on software's influence on the research process. [45]

We can conclude that our interviewees primarily perceived the benefits of software usage in terms of easier data-handling (how to organise the data, code it and retrieve it more efficiently) and in terms of a better ability to analyse the data in a more systematic way. They obviously concluded that they could benefit from using software, as they decided to use it in the first place, but they aligned their expectations and mastered software usage (and got better insights into its capabilities) during the research process itself. [46]

What we also found interesting when researching experiences of CAQDAS usage is the above-mentioned perception of the "higher professionalism" whilst using CAQDAS. MANGABEIRA et al. (2004, pp.169-170, p.175) also point out that using software can add additional legitimacy and credibility in the eyes of research audiences and that this could be especially attractive to early-career researchers trying to become accepted within the research community. In the Croatian context this can be partly explained by the contextual factors of the research methods tradition, discussed earlier in this article. Within the context of a strong and established quantitative tradition, qualitative researchers sometimes feel as though they have to explain themselves and prove that their research is as "serious" or "scientific" as that of their colleagues working within the dominant, quantitative tradition. Nowadays, a great part of social sciences statistics expertise involves mastering statistical software such as SPSS, R or others. If one knows how to get the software to do the analysis and read the output, one does not even have to know much about the mathematical background of the analysis. As software is responsible for the complex calculations behind statistical analyses, it is a key component of the research process. Software usage within the quantitative tradition is nowadays completely normalised and taken for granted. Analysing the data means reflexive usage of the computer software. By analogy, the very fact that the qualitative researcher is using software could be presented as doing "serious research", as opposed to doing something which resembles "some process in primary school". [47]

There are still not many users of CAQDAS in the Croatian sociological community, but the trend of its usage is on the rise. Mostly non-experienced users are interested in CAQDAS, as the education and user support is not yet institutionalised. It seems that when students decide not to use CAQDAS it is primarily because they do not perceive that there are some clear and valuable benefits for which they would be prepared to spend their time learning to use particular software. [48]

Young researchers, especially those who independently learn to use software, either by consulting their colleagues and/or by using manuals, sometimes have too great expectations of software. However, after initial enthusiasm they face the fact that software cannot analyse qualitative data for them. As there are not many users of CAQDAS in Croatia, some researchers point out the simple fact that using the software can give their research an aura of "more serious" work in front of some of their colleagues. In this way there exists some vague notion that software does not merely carry out procedures that would be used otherwise (coding and retrieving), yet simpler and quicker, but that it could "in a way give credibility" to the research, to cite one interviewee (I5). However, our interviewees did not explicate in which ways, exactly, this is achieved. We found only one specific reference directly relating software use with achieving the specific requirements of the method used (e.g. grounded theory approach). [49]

We argue, therefore, that accounts of "more exact" analysis have more to do with trying to legitimate the research within the research community where quantitative approaches have been dominant for a long time, especially within the departments which host the postgraduate sociology programme that most of them attended. Nevertheless, we found that the arguments about the specific benefits of software corresponded with the arguments which a number of authors used when discussing the possible gains from software in terms of credibility, validity and the overall quality of qualitative research. [50]

As our analysis shows, CAQDAS is a technology which is mostly learned by doing; the reflexivity of its usage is very much part of its usage and cannot be avoided. More experienced users can provide invaluable informed advice to younger researchers through educational workshops. Our research shows that users usually decide on whether to use software or not at the level of being informed on software (and gain more in-depth knowledge only while using it). The role of introductory education and advice could be seen as a key factor contributing to the spread of CAQDAS, as far as the Croatian case is concerned. There should be a caveat because the "community of users" often advocates usage of one particular software package. [51]

If we get back to our framework of decision-making on software usage from our introduction, it can be concluded that in Croatia, trends in research methods facilitate the growth of the CAQDAS user community. Availability, though, is the most important issue. The software is not easily available as prices are high and institutions do not want to acquire it mainly due to financial reasons. But availability of educational resources seems to be even more important as it shows that the initial contact with software is of the greatest importance for the decision to use it or not, and on deciding on the particular software package. Perceived costs and benefits of software usage seem to matter mostly on the level of overall impressions about learning effort. But elaborated expectations on gains and difficulties that software usage will bring to the research process are almost completely lacking or are exaggerated, as in the case of the expectation that software will do the analysis itself. We believe that for those who have the opportunity to use CAQDAS, the main reason for deciding not to use it is that the perceived costs of learning are regarded to be too high when weighted against the perceived benefits of software usage. [52]

This discussion on the potential that software has for enhancement in quality of qualitative research should be read as an attempt to inspire colleagues to be more reflexive in software usage, from the perspective of methodological concerns that are not always that obvious at a first glance while considering whether to use software or not. If the notion that software can add to quality of qualitative research would become more prominent, the decision-making process regarding software usage and package choice would probably be less influenced by "external" factors such as opportunity, price or, sometimes, academic trends. [53]

We would like to thank our colleagues who agreed to participate in this research. We would also like to thank Karin DOOLAN, Ph.D., University of Zadar, Croatia and reviewers of FQS for reading and commenting on an earlier version of this article.

1) There were cases when proponents of quantitative methods attempted to discredit qualitative methods politically as the research results of the latter were not in line with the then-current political agenda (see the case of JILEK, KOKOT & POTOČNJAK, 1981). <back>

2) Review of available courses was made according to programmes available at web pages of all researched departments of sociology (last update March 2012). In order to limit the scope of our research to those institutions that offer degrees in sociology, we have focused on departments of sociology only, while there are sociology chairs at other faculties as well. <back>

4) At the time of concluding our research (early 2012) the following courses were available: "Introduction to Statistics for Sociology", "Introduction to Statistics for Sociology I", "Introduction to Statistics for Sociology II", "Quantitative Research Methods", "Data Processing and Analysis", "Survey Method", "Selected Chapters of Statistical Analysis", "Construction and Evaluation of Measuring Instruments". <back>

5) It has to be mentioned that there are three other research methods courses that could not be regarded as either quantitative or qualitative methods courses. The first gives an introductory overview of social science methods at the undergraduate level ("Introduction to Social Research Methodology"). The second is taught at the graduate level, and is focused on broader epistemological issues ("Selected Issues in Epistemology of Social Sciences"). Additionally, there is an elective course "Research Project" in the second year of master's studies, where students plan and conduct their own research project, working as a team. Only beginning in 2011 did students have the chance to do a qualitative research project (using interview as a method of data collection). Previously, all research projects were quantitatively oriented. <back>

6) All university programmes in sociology that were analysed in this research were downloaded between February and March 2012. <back>

7) The programme was conducted in cooperation with the following institutes: Institute for Social Research in Zagreb (IDIZ), Institute for Development and International Relations (IRMO), Institute for Migration and Ethnic Studies (IMIN), and Institute of Social Sciences "Ivo Pilar". Unfortunately, during the writing of this article this cooperation was stopped and now the Faculty of Humanities and Social Sciences, University of Zagreb, is the only host. <back>

8) We conducted a short analysis of the research methods which were used in theses from 2008-2011, listed in the official online database Croatian Scientific Bibliography and in the "Collection of Masters and Doctoral Theses" of the National and University Library in Zagreb. We included all of the theses categorised in the bibliographic indexes under the field of sociology. Since 2008, 13 theses have used qualitative methods, 22 have used quantitative methods, seven have used mixed methods and 16 were desk research (systematic reviews or theoretical discussions which did not include primary empirical data collection). However, if we look only at the theses defended at the Department of Sociology, University of Zagreb, then we have an equal number of theses using qualitative and quantitative methods (nine), five using mixed methods and ten desk research theses. <back>

9) A brief overview of the institutional background of authors of defended PhDs shows that quantitative methods were used mostly by persons employed as young researchers at the Institute "Ivo Pilar" or the Institute for Social Research in Zagreb—institutions with an established infrastructure for big-N surveys. The costs of such research are far beyond the reach of young researchers working as research assistants on low-budget research projects or those who are not formally employed at academic institutions. <back>

10) Based on the data from the Faculty of Humanities and Social Sciences, many graduate students also use qualitative methods for their final graduate papers. Among research methods, interviews (in different iterations) are used most frequently. <back>

11) The same could be said regarding editorial policies of Croatian social sciences journals, which could encourage publishing of qualitative methods based papers and thus make this methodology more visible. <back>

12) We have limited the survey to those students who attended the workshops because our focus was on those who received the training. Other PhD students had a chance to attend workshops but did not show interest in participating in them. <back>

13) The survey covered respondents from academic institutions (e.g. universities, research institutes) who have a sociology curriculum and/or that conduct sociological research. <back>

14) All translations from Croatian to English are ours. Ellipses in quotes indicate faltering speech, and longer pauses are noted by [pause]. The codes of interviewees were given according to the interview schedule; thus the code I3 corresponds to the third interviewee, etc. The paragraph where the quote is situated in the interview transcript is indicated by the number next to the interviewee code. <back>

15) In Atlas.ti, documents are stored outside the main project file ("hermeneutic unit"in Atlas.ti terminology). Depending on the project setup, this can cause problems when transferring project files from one computer to the other. <back>

16) "Loyalists", "critical appropriators" (including the experts); "experienced hands", and "instrumental adopters". <back>

17) Only when interviewees considered some unique capability of the software as very important to their research process did they pay attention to "internal" factors, that is, software capabilities and structure. For example, one of the interviewees mentioned MAXQDA's portable installation as useful, because she could work both at the workplace and from home, which was very important to her. <back>

18) Of course, there is an on-going discussion on whether validity and reliability criteria are relevant to qualitative research, or if measurement of the quality of qualitative research is a completely wrong proposition from the start. For arguments regarding the possibility of CAQDAS improving qualitative research credibility, reliability and overall quality see: BRINGER et al. (2004), FIELDING and LEE (1998), FRIESE (2010), HWANG (2007), and SILVA and RAMOS (2010). <back>

Adam, Fran & Podmetnik, Darka (2005). Qualitative research in a changing epistemic context. The case of a small social science community. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(3), Art. 40, http://nbn-resolving.de/urn:nbn:de:0114-fqs0503400 [Date of access: January 8, 2013].

Bringer, Joy D.; Johnston, Lynne H. & Brackenridge, Celia H. (2004). Maximizing transparency in a doctoral thesis: The complexities of writing about the use of QSR*NVIVO within a grounded theory study. Qualitative Research, 4(2), 247-265.

Carvajal, Diógenes (2002). The artisan's tools. Critical issues when teaching and learning CAQDAS. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(2), Art. 14, http://nbn-resolving.de/urn:nbn:de:0114-fqs0202147 [Date of access: January 8, 2013].

Castells, Manuel (1996). The rise of network society: The information age: Economy, society and culture (Vol.1). Oxford: Blackwell.

di Gregorio, Silvana (2010). QDAS development and future trends. Presentation at the 3rd Annual European Workshop on Computer-Aided Qualitative Research 2010, Lisbon, http://prezi.com/82arnt4a5pyb/qdas-development-and-future-trends [Date of access: November 14, 2010].

Fielding, Nigel G. & Lee, Raymond M. (1998). Computer analysis and qualitative research. London: Sage.

Flick, Uwe (2004). Triangulation in qualitative research. In Uwe Flick, Ernst von Kardorff & Ines Steinke (Eds.), A companion to qualitative research (pp.178-183). London: Sage.

Gibbs, Graham R.; Friese, Susanne & Mangabeira, Wilma C. (2002). The use of new technology in qualitative research. Introduction to Issue 3(2) of FQS. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(2), Art. 8, http://nbn-resolving.de/urn:nbn:de:0114-fqs020287 [Date of access: January 8, 2013].

Gobo, Giampietro (2008). Re-conceptualizing generalization: Old issues in a new frame. In Peerti Alasuutari, Leonard Bickman & Julia Brannen (Eds.), The Sage handbook of social research methods (pp.193-213). Los Angeles, CA: Sage.

Hwang, Sungsoo (2007). Utilizing qualitative data analysis software: A review of Atlas.ti. Social Science Computer Review, 26(4), 519-527.

Jilek, Miroslav; Kokot, Mladen & Potočnjak, Željko (1981). Četiri priče s "Otoka mladosti" [Four stories from the "Youth Island"]. Zagreb: Centar društvenih djelatnosti SSOH.

Kardorff, Ernst von (2004). Qualitative evaluation research. In Uwe Flick, Ernst von Kardorff & Ines Steinke (Eds.), A companion to qualitative research (pp.137-142). London: Sage.

Mangabeira, Wilma C.; Lee, Raymond M. & Fielding, Nigel G. (2004). Computers and qualitative research: Adoption, use and representation. Social Science Computer Review, 22(2), 167-178.

Patton, Michael Quinn (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage.

Ritchie, Jane (2003). The applications of qualitative methods to social research. In Jane Ritchie & Jane Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp.24-46), London: Sage.

Roberts, Kathryn A. & Wilson, Richard W. (2002). ICT and the research process: Issues around the compatibility of technology with qualitative data analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(2), Art. 23, http://nbn-resolving.de/urn:nbn:de:0114-fqs0202234 [Date of access: January 8, 2013].

Saldaña, Johnny (2009). The coding manual for qualitative researchers. Los Angeles, CA: Sage.

Supek, Rudi (1961). Ispitivanje javnog mnijenja [Public Opinion Research]. Zagreb: Naprijed.

Vučković Juroš, Tanja (2011). Reporting on the issues of research rigour and ethics: The case of publications using qualitative methods in Croatian social science journals. Revija za sociologiju [Sociological Review], 41(2), 161-184.

Yin, Robert K. (1994). Case study research: Design and methods (2nd edition). Thousand Oaks, CA: Sage.

Petra RODIK works as an Associate Professor at the University of Zagreb, Faculty of Humanities and Social Sciences, Department of Sociology, where she earned her PhD in sociology in 2011. She introduced CAQDAS workshops within the postgraduate studies programme in sociology, since 2014 has been teaching a graduate-level "Selected Topics in Qualitative Research" course and is an active member of the qualitative methods strand of the Croatian Sociological Association.

Department of Sociology
Faculty of Humanities and Social Sciences, University of Zagreb
Ivana Lučića 3, 10000 Zagreb, Croatia

Jaka PRIMORAC works as a Research Associate at the Department for Culture and Communication at the Institute for Development and International Relations (IRMO) in Zagreb, Croatia. She holds a PhD in sociology from the Faculty of Social Sciences and Humanities, University of Zagreb. Her research interests intersect the fields of cultural and creative industries, cultural and creative labour, media and cultural policy and digital culture. She is also interested in the implications of the usage of qualitative research methods in sociological research.

Department for Culture and Communication
Institute for Development and International Relations
Lj.F.Vukotinovića 2/II, 10000 Zagreb, Croatia

Rodik, Petra & Primorac, Jaka (2015). To Use or Not to Use: Computer-Assisted Qualitative Data Analysis Software Usage among Early-Career Sociologists in Croatia [53 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 16(1), Art. 12,
http://nbn-resolving.de/urn:nbn:de:0114-fqs1501127.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License

	I1	I2	I3	I4	I5	I6	S1	S2
More convenient and flexible than other			x
Particular technical capability	x		x	x
It is used at the institution of affiliation			x			x
Got familiar with it during PhD programme			x		x	x	x
Colleagues' recommendation		x	x	x		x		x
Requested by project team	x
Price				x
Availability								x