Communities of Scholars and Mixed Methods Research: Relationships Among Fields and Researchers

Abstract: In this paper I explore processes of knowledge production and circulation within a specific research community: the self-identified community of mixed methods scholars—i.e., the group of researchers adopting a mixed methods approach and using the label mixed methods—during the phase of its emergence and institutionalization. I focus on citations within this community, considering that the act of citing is linked to the intention of scholars to position their work not only within a research area but also within a community contributing to that specific area. I employed strategies from citation network analysis (CNA) to understand the fields involved, as well as the structure of the community in relation to citation practices. I identified the most common subjects and methodological fields in which mixed methods are mentioned by isolating sub-communities and the most influential authors in the network. I discuss the implications of this network structure with regard to power relations and hegemony. This also includes the function of nodes which appear to be marginal, but are relevant in citation practices since these authors play a bridging role across the various sub-communities.

Key words: mixed methods; research communities; citation network analysis; knowledge circulation

6. Consolidating a Research Community: Citation Practices Among Mixed Methods Scholars in its Emerging Phase

Combining different methods in the social sciences is not truly innovative, if we consider research endeavors up to at least the mid-20th century, when social scientists—especially sociologists and anthropologists—would jointly adopt qualitative and quantitative strategies in their studies (see the many examples from the early 20th century in MAXWELL, 2016 for both the social and the natural sciences, and in PEARCE, 2012 for the social sciences). While research practices such as triangulation, combining methods or using multiple methods have always been common, the label mixed methods was not applied with the same connotation as we know it today. Mixed methods research as a "strand that has recently emerged and that explicitly aims to offer a framework for combining methods" (TIMANS, WOUTERS & HEILBRON, 2019, p.194) was increasingly adopted by scholars at the latest from the early 2000s onwards. Employing the common name mixed methods is only one indicator of the emergence of a definite community of scholars. By the same token, TIMANS et al. argued that this community is "the result of the position-takings of its producers" (p.196). [1]

TIMANS et al. identified an institutionalization process in mixed methods research within both the academic and the scientific field. First and foremost, this is evident in the large corpus of publications on method integration, including various textbooks, a handbook in its second edition (TASHAKKORI & TEDDLIE, 2003, 2010), and a reader (PLANO CLARK & CRESWELL, 2008). Many of the numerous general methodological textbooks also contain distinct sections on mixed research designs. Moreover, in 2007 a journal specifically dedicated to mixed methods (the Journal of Mixed Methods Research, JMMR) was founded, and contributions featuring mixed designs are now explicitly welcomed in many methodological journals. Furthermore, the Mixed Methods International Research Association (MMIRA) was established in 2013. Also, groups of mixed methods scholars participate in research associations in various fields, such as—to mention one—the special interest group in the American Education Research Association (AERA). [2]

Therefore, researchers in the mixed methods area form a group that is characterized by 1. a common methodological identity with a specific label, 2. a relatively precise time period for its emergence and institutionalization, and 3. characteristic relations to various fields of research, since mixed methods are merely an approach that could potentially be applied to an infinity of subjects. Given its relatively young age as a self-identified methodological specialty, mixed methods research can be considered a pertinent turf to explore communities of scholars as well as knowledge production and circulation, especially with regard to its emerging phase. Thus, after two decades have passed since the publication of the 2001 FQS special issue on "Qualitative and Quantitative Research: Conjunctions and Divergences" (SCHREIER & FIELDING, 2001), the exploration of mixed methods communities, especially during their emergence and institutionalization phase, is particularly significant for this special issue. [3]

Given these premises, in this paper I intend to explore the self-identified community of scholars involved in mixed methods, aiming at understanding the consolidation of a group of researchers within processes of knowledge production and circulation. In particular, I will look at citation structures in international publications and scientific journals while asking:

To answer these questions, I will consider articles in which mixed methods research is mentioned and that were published between 2003 and 2017—dates that I will explain later—and I will analyze citation structures with citation network analysis (CNA). When focusing on citation practices in social research, it is fundamental to keep in mind the meaning(s) attributed to the act of citing within the process of knowledge construction and its circulation. Indeed, it can be asserted that "researchers from the same specialty tend to cite each other in order to position their work in the field based on previous knowledge" and that "scientific knowledge is assumed to increment over time following a 'smooth path', the papers that introduce important new insights are cited until new results modify or contradict them" (CALERO-MEDINA & NOYONS, 2008, p.272). Moreover, publishing papers in scientific journals is part of the research evaluation system and scholars cite to acknowledge other papers and their authors, so that hierarchies of authority and power are constructed in scientific communities. The act of citing is closely related to the intention of researchers to position their work not only within a field of research but also within a community of scholars contributing to that specific field. For this reason, looking at citations could be helpful in attempting to better understand a research community and how knowledge is produced and circulated within it. [5]

The idea of positioning a researcher's work also resonates with the concept of situated knowledges (HARAWAY, 1988): This is a reminder from feminist theories that "knowledge can never be regarded as universal" (WESTMARLAND, 2001, §9), acknowledging the relevance of the context in which knowledge is produced as well as the positioned role of the researchers. A feminist perspective is crucial for my work, as I am conscious that knowledge is always value-laden. Being in the position of both constructing knowledge myself and of investigating processes of knowledge production, I need to be aware of these two viewpoints. From one side, using a reflexivity approach, I realize how my own vision is certainly related to who I am as a researcher—an early-career, queer, white person, coming from Italy and working at a university in northern Italy—with specific values; thus, who I am may affect the perspective I adopt in this study. From the other side, I acknowledge that the field I am exploring is itself value-laden—either explicitly or implicitly—hence, it is necessarily not neutral and is influenced by specific visions of the scholars participating in the communities of research streams that I consider within this study. With feminist theories, a lens is provided to look at citation practices, which are, on the one hand, a way to acknowledge being part of a research field with its values. On the other hand, citations are related to power dynamics which are not detached from socially structured forms of power as an interaction of categories of gender, race, and social class (HARDING, 1986, p.9). Therefore, when investigating citation practices, we need to also take into account these elements, considering that scholars' and researchers' levels of success and prestige are also likely to be shaped by these same structured forms of power. [6]

Moreover, as the focus here is on the legitimized ways of constructing and producing social scientific knowledge, I should also acknowledge the unequal distribution of power and resources in the framework of mixed methods as a situated field. For this reason, the new political sociology of science (NPSS) (FRICKEL & MOORE, 2006) can be considered a useful theoretical perspective on the topic, since it can be used in attempting to answer the question "What's political about science?" (and I would like to add "What's political about mixed methods?"). Similarly to feminist theories, following NPSS allows us to recognize "the contingent and constructed character of scientific knowledge but also insists that construction processes are neither random nor randomly distributed" (p.9). The following elements of NPSS are especially relevant for reflections on the topic of this paper.

Furthermore, NPSS is described as the "analysis of institutions and networks as they condition the availability and distribution of power in the production and dissemination of knowledge" (FRICKEL & MOORE 2006, p.8), with networks being considered "dynamic configurations of relationships among individual and organisational actors" (ibid.). I will specifically focus on the network dimension in this paper, identifying and describing specific networks related to the citing process and their relationships within the mixed methods community in the social sciences. [8]

As NPSS authors only marginally incorporate feminist theories on knowledge, I will argue here that an NPSS theoretical framework would benefit from a thorough discussion of the concept of situated knowledge, as both embodied and embedded. Knowledge can be regarded as embodied if we consider that it is anchored within the knowing subjects and thus is inseparable from the materiality of their bodies. Situated knowledge is also embedded knowledge, as it is situated in the dynamics of interaction, in language, in a physical context. Furthermore, as GHERARDI noted:

"[...] the practices of science—like any other social process—are situated in specific contexts of power/knowledge (Knorr-Cetina, 1981). Situated practices are both pre-reflexive (depending on unstated assumptions and shared knowledge for the mutual achievement of sense) and reflexively constitutive of the situated members' contexts from which they arise" (2008, p.517). [9]

Another central concept here is that of practice. In this paper I focus on citation practices as a means to connect knowing with doing (GHERARDI, 2000). It is indeed meaningful to recognize the significance of individual researchers' agency and their intentionality while conducting research, as building knowledge is necessarily not only theory but also praxis and the two cannot be truly separated. For the act of citing, this means that while we, as researchers, are involved in research endeavors, acknowledging the work of other scholars, we are both theorizing and doing (practicing the act of citing). [10]

The study I present here was conducted within a larger research project carried out in the context of my PhD dissertation, in which I focused on mixed methods articles published in international journals. I will start by presenting an overview of the mixed methods research community and its relation to various research fields (Section 2). I will then describe the methodological approach and strategies from citation network analysis (CNA) used in this study (Section 3). Next, I will present some results, starting with the identification of sub-communities in the mixed methods network (Section 4) and with a specific focus on the sub-community writing about mixed methods (Section 5). Finally, I will discuss findings from this study (Section 6) and draw some conclusions (Section 7). [11]

A common assumption within the self-identified mixed methods community, but also increasingly among social researchers more generally, is to consider mixed methods a particular sub-field within social science research in general and within social research methodology specifically. Methodological and epistemological reflections on method integration have been established as a distinctive niche in the international social research literature. To illustrate the idea of the existence of mixed methods as a sub-specialty with a specific identity, we can look to the text written by PLANO CLARK and IVANKOVA, entitled "Mixed Methods Research. A Guide to the Field" (2016). What these authors designate with the term "field of mixed methods research" is indeed "the corpus of literature and the community of scholars discussing and applying all the aspects of mixed methods research" (p.4). Generally speaking, the idea that mixed research can be considered a field in its own right seems to be a consolidated issue in the literature ever since the publication of the "Handbook of Mixed Methods in Social & Behavioral Research" (TASHAKKORI & TEDDLIE, 2003), in which the authors refer to a tradition of at least thirty years (GREENE, 2008). [12]

Nevertheless, the area of mixed methods can hardly be considered a discipline according to the criteria identified by KRISHNAN, namely, that "disciplines have a particular object of research (e.g., law, society, politics), though the object of research may be shared with another discipline" (2009, p.9). In contrast, the term mixed methods can be applied to diverse research objects; it is rather a methodological approach with which it is possible to gain knowledge of phenomena. Hence, on the one hand we can look at the perception of mixed methods as a sub-specialty with its own identity, while on the other hand it is important to recognize that integrative research designs are applied across several areas of social inquiry, each with its own specific tradition and, sometimes, part of separate disciplines. The Journal of Mixed Methods Research published an editorial about the uses of mixed methods within various disciplinary fields (TASHAKKORI & CRESWELL, 2008). Here, TASHAKKORI and CRESWELL emphasized the existence of a distinct research area, while also stressing that it is interconnected with several disciplines that each have their own independent development. They specifically mention program evaluation, international development studies, anthropological demography, educational psychology, health care, management, and architecture, all of which are described as playing a role in the development of mixed methods as a field. Nevertheless, I suggest here that rather than considering mixed methods as a field, it would be more helpful to adopt the concept of research community, to highlight the cultural aspect of a group of scholars and researchers actively producing a new specialty area. As I mentioned in Section 1, there are specific journals, associations, and courses at various universities dedicated to mixed methods. However, the idea of a research community, rather than a field, is closer to the perception of a developing research group whose members intentionally create their own space in the academic and scientific landscape. A research community is also a community of practice, therefore I focus here on citation practices, in order to acknowledge the agency of individual scholars participating in a research network and building knowledge through their actions (GHERARDI, 2008). [13]

Furthermore, in her article "Is Mixed Methods Social Inquiry a Distinctive Methodology?", GREENE (2008) showed the contribution of different research areas to mixed methods developments, each with its own particularities. Subjects of inquiry explicitly mentioned in the article include: social research, evaluation, demography, development economy, and anthropology. The author's aim was to present applications of mixed methods in those fields and she underlines the need to learn more deeply from conversations among scholars who engage diverse disciplines and areas of applied inquiry. Similarly, MAXWELL described three research communities—design-based research in education, process tracing in political science, and sociolinguistics—in which scholars are involved in "combining qualitative and quantitative approaches and methods without substantial interchange with other fields [including the self-defined mixed methods community] that are also doing this" (2018, p.318). Although my interest in this paper is the self-identified mixed methods community—i.e., scholars using the specific term mixed methods—it is necessary to note that researchers in different areas outside this community are also engaging similar methodological practices. [14]

In view of this premise, I will consider what are generally viewed in the literature to be application fields for mixed methods by investigating citation structures and in particular sub-communities in subject or disciplinary fields within the larger mixed methods community. However, I do not expect to directly find research communities comprised of scholars who do not self-identify with the label mixed methods, for instance sociolinguists, because the focus of the paper is on the self-identified community of mixed methods researchers. Specifically, I will focus on the following fields, considered as a synthesis of what I have mentioned so far and in particular of what has been explored by GREENE (2008), MAXWELL (2018), and TASHAKKORI and CRESWELL (2008): social research (including political science), education, health care, evaluation research, anthropology, demography, development economy (including international development studies), management, and architecture. [15]

I applied citation network analysis (CNA) for my study. This is an effective strategy for the exploration of processes of knowledge production and diffusion in a certain scientific field and also for mapping the structure of a research area (LEE & SOHN, 2015; ZHAO & STROTMANN, 2015). In CNA, traditional social network analysis is taken into the territory of bibliometrics (OTTE & ROUSSEAU, 2002). The data set I used for this study was extracted from Scopus, using Elsevier's APIs tool, including as many papers as possible from the social sciences in which mixed methods were mentioned (written in English), spanning the period between 2003 and the beginning of 2017. This period is of particular interest in the context of this paper, since it corresponds to a phase in which mixed methods research became more widely practiced and its institutionalization was intensified. In 2003 the "Handbook of Mixed Methods in Social & Behavioral Research" (TASHAKKORI & TEDDLIE, 2003) was published. Before this date, despite the already widespread application of mixed approaches as research practice, the use of the specific term mixed methods was not that common. The year 2017, on the other hand, would later mark the transition to a new phase—as FETTERS and MOLINA-AZORIN (2017) stated in their editorial to the tenth anniversary edition of the Journal of Mixed Methods Research—which can be considered as an institution for the field, as I mentioned earlier. [16]

To be included in the data set for this study, papers needed to meet the following criteria:

Applying these criteria, I was able to extract 4,785 records from the Scopus database, including publication information—such as title, journal, authors, DOI, volume, issue, page span, reference list, date, abstract and keywords—about papers in which mixed methods were mentioned in the title, abstract or keywords. Although I also retrieved index information, such as SJR (Scientific Journal Ranking) and CiteScore, I decided not to use this in my study because these metrics are related to a journal and not to individual researchers. Furthermore, given that the metrics are sensitive to subject area and that publication rates vary across these, if I had used them, I would have introduced a publication bias depending on the journal where the article is published, as well as a comparability bias. The distribution of the data set over time is presented in the following graph (Figure 1). The increasing number of mixed methods papers (self-identified with the label) that were published is visible, especially between 2012 and 2014, when growth was fastest, while for the years 2015 and 2016 we can still see a rise, though at a slower pace.

Figure 1: Annual frequency of mixed methods papers between 2003 and 2016¹⁾ [18]

In Table 1 I show the top six journals in the set of extracted records, listed by frequency. We can spot the Journal of Mixed Methods Research in fourth position, with 37 publications. This is the only interdisciplinary—and methodological—journal, while all other journals in the table are from the subject areas of medicine and health care studies. However, we need to consider this table with caution, since the frequencies are quite moderate, and the distribution is rather skewed and sparse. Moreover, within certain journals a number of articles might be published with different regularity.

Table 1: Top six journals by frequency count and percentage in the set of extracted records [19]

I later adopted reference lists as the basis for network construction and the data set was composed of authors of papers, considering first authors only, with three main columns: source and target of the act of citing, and a weight obtained by the count of citations in the set of extracted papers. To ensure data quality, pre-processing activities were necessary, specifically by making the names of the authors uniform using the surname and the first letter of the first name (in order to consider different authors with the same surname). In conducting the analysis, I mostly used the software Gephi, an open-source tool that allows the researcher to visualize networks and perform statistical analyses on them. Moreover, I relied on the software R for handling the data set. [20]

In CNA, nodes represent scholars who are either citing or being cited, while edges represent the action of citing; the weight of edges is given by the count of citations in the whole set of papers. The size of a node in a network is determined by what in CNA is called its degree. A higher degree implies that a node has a greater influence on the connections made: The higher the degree, the more connections a node has in a network. In citation networks where nodes represent authors, this means that authors are either publishing intensively in the area—and period—or that they frequently appear in reference lists, or a combination of the two, depending on the chosen measure. Three measures are common: Indegree is a measure of being cited by others in the network, outdegree is a measure of citing others in the network and average degree is the sum of the two. The weighted versions of indegree and outdegree are constructed by also considering citation counts (i.e., weights). Therefore, if authors are central nodes this indicates their key role in the network, and if they are scholars that all other members of the community refer to, they could be said to represent some sort of authority in terms of publications. The overall network of the mixed methods field investigated in this paper is composed of 79,646 nodes and 158,165 edges. [21]

It is necessary to emphasize here that a set of assumptions is inevitable when working with CNA, with some consequent limitations. The main assumptions concern the influence of the cited work and the similarity between the citing and cited works:

"It is generally assumed that a citation represents the citing author's use of the cited work, and indicates an influence of the cited work on the author's new work, and as such a flow of knowledge from the cited to the citing works' authors. Citations also indicate relatedness (e.g., similar subject matter or methodological approach) between these two works" (ZHAO & STROTMANN, 2015, p.1). [22]

As a limitation, we also need to consider the possibility that an author's work could be cited not to be acknowledged, but to be criticized, and it is not possible to discern which is the case in large networks. Nevertheless, the most relevant issue I had to deal with during the process of data set construction and cleaning was name ambiguity. More specifically, I was aware of the possibility of facing non-unique names, which is inevitable in a data set of 79,646 nodes. While it was possible to distinguish between different authors with the same surname, disambiguation could not always be realized, for example in cases of both surname and first name homonymity. Moreover, sometimes the surname was spelled slightly differently, or letters from the first and middle names were not reported in the same order, or the first name and surname were inverted. However, it is arduous to completely avoid these problems in citation network analysis. The only possible approach is to appeal to the law of large numbers, with eventual bias kept to a minimum (ZHAO & STROTMANN, 2015). [23]

Specifically, using CNA strategies and theoretical elements from NPSS presented above, I addressed the research questions of this study by focusing on the relevance of networks for processes of scientific knowledge circulation, paying attention to citation practices within a specific research community. Moreover, while conducting the analysis and interpreting the results, I inevitably adopted a feminist lens to investigate the research questions in the terms described in the introduction to this paper, paying particular attention to citation as a practice. In relation to the first set of questions—Which different fields are researchers involved in the mixed methods area connected to? How are researchers from different fields related in the mixed methods network?—I identified different sub-communities, which in network analysis refer to clusters in the network with closer relations, and I isolated them by using a measure of modularity, following the algorithm by BLONDEL, GUILLAUME, LAMBIOTTE and LEFEBVRE (2008). The higher this parameter is, the more defined the sub-communities in the network are. More specifically, a result of 0.4 or more is usually considered meaningful. [24]

Moreover, I identified the diverse sub-communities in visualizations, and by exploiting various colors and network graphs, additional information on communities' relations could be gained. To understand what these communities referred to in terms of publication areas or fields, I focused on the more relevant nodes in the graphs and queried the data set to identify particular publications by these specific scholars. By exploring the content of titles, journals, keywords and abstracts of the publications it is possible to understand if there is a common thread between authors in the community, eventually also further investigating the Scopus database. [25]

Regarding the second set of questions—What structure of citing practices do researchers in the specific community of mixed methods produce? Which specific sub-communities can be identified within the mixed methods citation network?—an analysis of density and centrality can be helpful. Density can be defined as an indicator of the general level of connectedness of a certain graph, in this case, how close individual researchers in a network are in terms of citations. For this purpose, I relied on the measure of average path length, which is the average graph distance between all pairs of nodes in a network—or, in other words, the average length of the shortest path between any two nodes— and is an indicator of the overall connectedness of a research area. The lower the measure is, the more connected the nodes in a network are, since there are fewer steps in the path from one node to another. [26]

Thus, I will compare different portions of the overall network to understand how connected sub-communities are. With regard to centrality, different measures can be distinguished: In addition to a measure of centrality for the overall network, degree centrality is the number of links that a node has; closeness centrality represents the total distance of one node from all other nodes; betweenness centrality is the number of shortest paths passing through an author since for every pair of nodes in a connected graph there exists at least one shortest path between the nodes such that the sum of the weights of the edges is minimized (OTTE & ROUSSEAU, 2002). I will specifically focus on the betweenness centrality of nodes, which in citation networks can be considered an indicator of the extent of an author's influence in the network. [27]

After this necessary methodological detour, in the following sections I will present some results of the CNA that I performed, showing structures of citations in the mixed methods area as networks. Starting from a description of the field(s) in which mixed methods are used in social inquiry as sub-communities, I then move on to the methodological sub-community of scholars writing about mixed methods and enduring discourses on this methodological approach. Within the set of papers selected for this study, there is, indeed, a specific sub-community within the network not only using mixed methods in research, but also writing about mixed methods. Moreover, sub-communities of scholars that apply mixed methods in different subject and disciplinary fields will be identified and described, allowing for an exploration of areas where mixed methods are particularly popular. All the graphs I present were constructed using HU's attraction-repulsion layout model (2006), an algorithm to visualize networks combining a multilevel approach. In the graphs, the node and label size are defined by the degree, and for each graph I will specify the measure used for degree (indegree, outdegree, average). Finally, thicker edges in graphs indicate higher citation counts between one author and another. [28]

In order to investigate the fields in which researchers engage in mixed methods and their relations in the mixed methods network, I isolated various sub-communities from the overall network, relying on the measure of modularity explained above. Therefore, in Figure 2 a portion of the total network considered in this study is presented. It was obtained by removing the less well-connected nodes (sparsity), setting the k-core parameter—the largest subgraph where nodes have at least k interconnections—to 3, i.e., eliminating nodes that are not as well connected to others, and setting edge weights to at least 2—i.e., ignoring those citations that only occurred once in the set of papers for the study. The size of the node is determined by the average degree, i.e., the number of times an author is cited or cites another. [29]

Four main sub-communities in the mixed methods discourse can be obtained in this way. Looking at the four main sub-communities in the graph, identified with different colors (violet, blue, green, red), we can clearly distinguish a particular one (here in violet), by identifying scholars in this group as the core of authors publishing about mixed methods. I refer to this sub-community as the methodological mixed methods sub-community, and names like ONWUEGBUZIE, GREENE, JOHNSON, TASHAKKORI, BRYMAN (mentioned by indegree)—all well-known authors in the mixed methods field—are visible. In the green sub-community, which is dominated by BORGATTI, other names emerge as well, like DALY, PUTNAM, GRANOVETTER, and CROSSLEY (mentioned by indegree)—all authors who seem to refer to the tradition of social network analysis, another methodological—rather than disciplinary—field. The red sub-community, the sparsest one, is dominated by AJZEN, whose node, however, appears less influential compared to the leaders of other communities. The blue sub-community, which overlaps with the others in the figure (and is illustrated here in the same focus window as the violet sub-community), is dominated by the name CRESWELL with the largest node, i.e., the largest average degree in the network. Hardly any other authors' names are at all prominent in this sub-community, which appears to be sparse and primarily composed of small marginal nodes. [30]

Nevertheless, the name CRESWELL is one that would emerge as prominent in any literature review on the topic of mixed methods and that appears as particularly influential in this network, as well as in the mixed methods community. Therefore, it was unexpected to see this author in a sub-community other than the one I identified as the methodological mixed methods sub-community. A possible interpretation could be that, since this author has a substantial influence on the whole set of papers in this study, his name has become relevant to different sub-communities and not only to a single one. This would mean that he is cited by a large number of other authors in the overall network, regardless of which sub-community they belong to. Moreover, this also shows how the different communities are related through scholars who act as a bridge between different fields. Further below, I present additional graphs with further sparsity removed, which means that only the more interconnected nodes are considered (Figures 3, 4, 5). In these graphs, CRESWELL belongs to the mixed methods sub-community. [31]

For this specific network in Figure 2, defined by removing sparsity (nodes with less than 3 connections and edges with only one citation are excluded), the mean average degree is 5.30, the mean average weighted degree is 6.40 and the average path length is 6.97. Without removing sparsity, the overall network of all citations in the data set has a mean average degree of 1.97, a mean average weighted degree of 2.30 and an average path length of 7.57. This means that authors in this particular portion of the network are considerably more influential than those in the overall network (according to the measures of average degree and average weighted degree). Moreover, as average path length is slightly lower here than in the overall network, it is possible to assume that this portion is made up of more interconnected authors, also as a result of the reduction in sparsity (excluding nodes with less than 3 connections from the graph).

Figure 2: Main sub-communities emerging in the network (nodes have at least 3 connections, edge weights at least 2). Please click here for an enlarged version of Figure 2. [32]

To further reduce complexity I constructed a second graph (Figure 3) with edge weights set to at least 3—i.e., excluding works that were cited only once or twice. The methodological sub-community—this time in light blue—is slightly different than in the first graph with CRESWELL now the dominant presence, although he was previously the largest node in a different sub-community. Sparsity was again removed by setting the k-core parameter to 3, but this time the node and label sizes reflect weighted average degrees—i.e., including citation counts between nodes—with some slight adjustments. For example, in the sub-community identified here with the color light green, previously dominated by BORGATTI, now the node referring to DALY looks larger. Other sub-communities are included, e.g., the yellow one, where the node BOURDIEU—a well-known author in the sociology of culture and economic sociology—now becomes visible. Other emerging nodes can be spotted, like the one related to SEIDER—an author from the education field—in the bottom-right corner of the red sub-community, and the one referring to BAIRD—an author on health care who uses an interdisciplinary approach that encompasses multiple methods—in the purple sub-community (top-left).

Figure 3: Main sub-communities emerging in the network (nodes have at least 3 connections, edge weights at least 3). Please click here for an enlarged version of Figure 3. [33]

Therefore, generally speaking, the subject or disciplinary fields that emerged in the analysis were: sociology of culture/economic sociology, the field of education, and health care. There is also at least one other methodological sub-community in addition to that of the mixed methods sub-community: that of social network analysis. Moreover, it can also be seen in the networks in both Figure 2 and Figure 3 how, despite their stronger internal links, the subject sub-communities identified still have nodes that represent a link to other sub-communities. These nodes are relatively small in terms of degree and they are marginal, but their edges have different colors, i.e., citations from different sub-communities. For instance, in Figure 2 we can spot the node referred to as EVANS, part of the purple sub-community but with links to both the blue and the green sub-communities. Similarly, the node CASTELLS, again in the purple sub-community, has connections to the green and the red sub-communities. In the following section, I will isolate what I called the mixed methods sub-community, putting aside for a moment the bigger picture and the connections between the various fields and sub-communities explored here. [34]

Focusing exclusively on what I called the methodological mixed methods sub-community—i.e., scholars writing about mixed methods—with a network built by setting parameters of k-core equal to 3 (to reduce sparsity, excluding nodes with less than 3 connections), and edge weights (citation counts) of at least 3, several observations can be made (Figure 4). In this specific graph the colors represent the measure of betweenness centrality, an indicator of the extent of an author's influence in the network, with darker colors being associated with a higher level of centrality. This means that we can not only discern the larger nodes with a higher indegree (i.e., authors who are frequently cited), but also darker nodes with an elevated betweenness centrality (i.e., a central position). The most cited authors might not also be the most central and best connected ones. Once again, the name CRESWELL is highly visible, both in terms of indegree and betweenness centrality. However, nodes representing authors that are only slightly smaller in terms of indegree (being cited), such as ONWUEGBUZIE, GREENE, or BRYMAN (mentioned by indegree), are less well-connected in the network in terms of betweenness centrality. In contrast, some nodes that appear rather small in terms of degree, i.e., cited less often, have a higher level of betweenness centrality. One possible interpretation is that even though these scholars are not being cited directly by the mixed methods sub-community, they have more citations across sub-communities, which implies that they have a bridging function. [35]

The measures for the portion of the network associated with the mixed methods sub-community—with 1,473 nodes and 4,754 edges—are: mean average degree: 3.23; mean average weighted degree: 4.27; average path length: 3.85. Compared to the other portion of the network seen before (referring to the main communities in the network, with mean average degree of 5.30, mean average weighted degree of 6.40 and average path length of 6.97), in this graph the authors—represented by nodes—are on average less influential within the network; however, looking at average path length it is possible to affirm that this particular graph is constituted of authors that are more interconnected. This aspect is even more accentuated by comparing it to the overall network, which has an average path length of 7.57.

Figure 4: The mixed methods community, sparsity reduced (excluding nodes with less than 3 connections and edges with citation counts of at least 3)—betweenness centrality identified by color (with darker colors being associated with a higher level of centrality). Please click here for an enlarged version of Figure 4. [36]

However, if a different degree measure is used, a different network is produced. As introduced in Section 3, indegree is the count of citations received by an author, while outdegree allows us to inspect the extent of a scholar's contribution to the network, by considering how many times a single author has cited another in the network. If we now examine the measure of outdegree (Figure 5), the most important role in the network, occupied so far by the node CRESWELL in every graph, is now that related to ONWUEGBUZIE. In this graph, node size represents the outdegree measure, while the color intensity represents the indegree measure, so we can notice the discrepancy between the number of citations made by the author (outdegree) and how often the author was cited (indegree). Thus, we can see that CRESWELL, despite being the most cited author (darkest node), is not the author making the most citations in the network (largest node). ONWUEGBUZIE, in contrast, is the author with the largest outdegree, i.e., he is the one to most often cite other authors (also see Table 2).

Figure 5: The mixed methods community, sparsity reduced (k-core of 3, edges with weight equal to at least to 2)—outdegree. Please click here for an enlarged version of Figure 5. [37]

Finally, to summarize, in Table 2 I show the main measures (indegree, outdegree, average degree, weighted indegree, weighted outdegree, weighted average degree, closeness centrality and betweenness centrality) for selected nodes of the mixed methods community. These were determined after considering graphs related to the methodological sub-community, as well as the literature review in my PhD dissertation and interviews with community members who participated in another phase of the larger study. We can also see from this table how CRESWELL, a name that emerged predominantly in all graphs, has the highest average degree and weighted average degree, with a particularly large gap separating this node from all the others. However, when it comes to weighted indegree—how many times an author has been cited—the node referring to TASHAKKORI is larger than that for CRESWELL. Furthermore, the highest measure for outdegree and weighted outdegree—indicating how many other authors in the network are cited by a node—is for ONWUEGBUZIE, followed by GREENE and WEISNER. These latter authors also show a rather high value for closeness centrality and betweenness centrality, together with BRYMAN. Authors who are only in the network because they are cited by others but whose publications are not part of the data set necessarily have a value of zero for outdegree, weighted outdegree, closeness centrality and betweenness centrality.

Table 2: Main measures (indegree, outdegree, average degree, weighted indegree, weighted outdegree, weighted average degree, closeness centrality and betweenness centrality) for some of the most recognizable authors in the mixed methods community, ordered by the highest indegree. Click here to download the PDF file. [38]

6. Consolidating a Research Community: Citation Practices Among Mixed Methods Scholars in its Emerging Phase

The investigation of the citation networks in mixed methods papers presented so far was helpful to better understand how knowledge is circulated within an emerging research community, such as the self-identified mixed methods community in the period 2003 to 2017. First of all, it is fundamental to note that the overall network appears to be rather interconnected, with scholars recognizing each other's work, thus building upon previous knowledge in the community and mostly validating what other authors have written. This aspect seems relevant in understanding what strengthens a research community in its growth phase. Moreover, differentiating between various sub-communities within the overall network of citations was important for discerning in which distinct research fields a mixed methods approach has been adopted in scientific publications. The most frequent subject or disciplinary fields in the analysis were: sociology of culture/economic sociology, the field of education, and health care. [39]

By the same token, a distinct sub-community was developed with regard to social network analysis (SNA). Similar to the case of mixed methods, this sub-community is defined by its methodological approach rather than a disciplinary tradition or a subject area. Furthermore, it is useful to note that SNA is often described as an inherently hybrid approach (BAZELEY, 2017), in which aspects typical of quantitative research—using numerical measures to provide information on networks—are combined with elements that might be considered more qualitative—such as descriptions of the visual aspects of a network.

"The foundation of social network analysis (SNA) lies in mathematical graph theory on the one hand, and ethnographic studies of kinship and interpersonal relations on the other [...]. SNA allows complexity of extended social relations to be reduced and summarised in a way that both facilitates comprehension and offers a new vantage point on the social world" (p.245). [40]

As I presented above (Section 2), the fields that I identified from the analysis of sub-communities in citation networks are only partially concurrent with the representations in the literature on mixed methods. Considering that the focus here was on the self-identified mixed methods community, I did not expect to find those research communities that are not necessarily in dialogue with the mixed methods community in the citation networks identified in this study, even though their members also combine different methods in their research (GREENE, 2008; MAXWELL, 2018). However, in most cases there was a convergence between fields explicitly associated with mixed methods in the literature, and the research areas I identified from the citation networks. [41]

Regarding the relations between various sub-communities in the network, it is necessary to underline the role of specific nodes referring to authors who act as links across different areas where mixed methods have been adopted. These authors might become part of one sub-community in the network although they cited or were cited by scholars from other sub-communities. When focusing on the mixed methods sub-community, we could see that certain nodes which appeared to be rather marginal in terms of degree referred to scholars who, even though they are not being cited directly by the mixed methods sub-community, turn out to be bridges to others. Therefore, when investigating the elements in the consolidation of a research community, starting from citation structures, it is crucial to notice the role of bridging authors, such as CASTELLS or EVANS (see Section 5): Although they make and receive fewer citations, their contribution to the network is their ability to connect different research areas to the backbone of citations more typical of the mixed methods community. Thus, they allow for specific knowledge created in the context of the methodological sub-community to circulate across diverse sub-communities. As explained above, I refer to the methodological mixed methods sub-community to identify the portion of the citation network composed of those scholars who write about mixed methods. In terms of citation structures, here more than in other sub-communities, some nodes representing authors like ONWUEGBUZIE, GREENE, TASHAKKORI, JOHNSON, WEISNER and BRYMAN (mentioned by indegree) are considerably larger, which means that these authors play a significant role, influencing the whole network. Specifically, it can be seen that only a few nodes predominate—and one node is particularly noticeable when measuring indegree, making CRESWELL by far the most cited author. As mentioned in Section 1, citations can be considered ways for researchers to position their work within a community, as well as a system for recognizing other authors' work (CALERO-MEDINA & NOYONS, 2008). [42]

In the emergence phase of mixed methods research, scholars participating in the network adopted strategies for consolidating the group of mixed methods researchers, such as instituting a dedicated journal, founding associations and research networks, and publishing textbooks and handbooks, as described in Section 1. This is also visible when looking at citation practices and structures of citation networks, primarily by noticing how interconnected the mixed methods methodological sub-community is. Having a specific group of authors who recognize the work done by each other might indeed be beneficial to solidifying the research network. It is also possible to identify a backbone of authors (such as CRESWELL, ONWUEGBUZIE, GREENE, TASHAKKORI, JOHNSON, WEISNER and BRYMAN—mentioned by indegree) in the methodological sub-community who have played an influential role in the whole network. Moreover, with this study I contribute to building on the idea that the circulation of knowledge in social inquiry has to do with the concept of authority—i.e., recognizing the power of specific authors in a research community—within a community of knowledge production. [43]

Drawing on the theoretical framework of the new political sociology of science (NPSS) presented in Section 1, the influence of specific authors on a network can be regarded as a form of power, considered in the WEBERian sense as:

"[...] the ability to influence others directly or indirectly, subtly or overtly, legitimately or illegitimately. Power is a dynamic and social condition whose character can be described empirically by the forms it takes, its distribution across sciences, the mechanisms through which it is expressed, and the scope and intensity of its effects" (FRICKEL & MOORE, 2006, p.8). [44]

Moreover, even within citation structures, power dynamics are not detached from socially structured forms of power (HARDING, 1986) because a community of researchers is consolidated by their (citation) practices, often building on the idea that certain authors are considered to be authorities. Therefore, it appears important that the members of the mixed methods research community reflect on the processes of knowledge circulation. To this extent, GHERARDI (2000) wrote:

"[...] knowledge-producing practices may either be annulled to sustain disembodied and disembedded scientific authority, or they may become the object of reflexive knowledge. [...] That is, the meaning of knowing is given in a community of listeners and speakers, and every new occasion for the use of this topos recast in a more densely textured form. In short, the theories we create and the ways we talk about them are not separate" (p.221). [45]

Nevertheless, in order to fully understand processes of knowledge production and circulation based on citation structures relating to mixed methods in social science, some further reflections would be needed. This paper is a starting point, but the focus here is on first authors only and the information given about those scholars is limited. In developing this study further, it would be advisable to take additional information into account, such as co-authorships and affiliations in terms of academic or research institutions. The analysis of citation networks should be combined with a deeper investigation of other aspects of research communities and individual researchers. For example, BRANDHORST and KRZYZOWSKI (2022) presented an approach called biographical reconstructive network analysis (BRNA), which could be beneficially applied to CNA, assuming that biographical aspects related to a researcher's career—necessarily situated in contexts of dominance based on gender, race, geographical origin, ability/disability, social class of origin—are considered relevant in order to better describe a research community. [46]

Another limitation of this study is related to the context of the papers selected. The decision to focus on articles indexed in Scopus and written in English necessarily had an effect on the network of citations extracted. The geographical context of North America—and mostly the United States—is indeed dominant (in terms of the authors' affiliations) in the set of extracted papers. Nevertheless, following NPSS principles (FRICKEL & MOORE, 2006) as well as those of feminist theories when looking at processes of knowledge production and circulation in publications, it would be crucial to understand what is happening in those areas of the world regarded as less privileged, both in terms of being considered marginal in relation to research and academia and in terms of having less immediate access to resources. Therefore, when conducting further studies on the topic of this paper, these aspects should also be taken into account. [47]

1) The year 2017 should not be considered, because I only extracted papers for January when the editorial in JMMR mentioned earlier was published. <back>

Bazeley, Pat (2017). Integrating analyses in mixed methods research. Newbury Park, CA: Sage.

Blondel, Vincent D.; Guillaume, Jean-Loup; Lambiotte, Renaud & Lefebvre, Etienne (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 10, P10008.

Brandhorst, Rosa & Krzyzowski, Lucas (2022). Biographical reconstructive network analysis (BRNA): A life historical approach in social network analysis of older migrants in Australia. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 23(1), Art. 1, https://doi.org/10.17169/fqs-23.1.3715 [Accessed: July 7, 2022].

Bryman, Alan (2006). Integrating quantitative and qualitative research: How is it done?. Qualitative Research, 6(1), 97-113.

Calero-Medina, Clara & Noyons, Ed C.M. (2008). Combining mapping and citation network analysis for a better understanding of the scientific development: The case of the absorptive capacity field. Journal of Informetrics, 2(4), 272-279.

Fetters, Michael D. & Molina-Azorin, José F. (2017). The Journal of Mixed Methods Research starts a new decade: Principles for bringing in the new and divesting of the old language of the field. Journal of Mixed Methods Research, 11(1), 3-10, https://doi.org/10.1177/1558689816682092 [Accessed: April 29, 2019].

Frickel, Scott & Moore, Kelly (2006). Prospects and challenges for a new political sociology of science In Scott Frickel & Kelly Moore (Eds.), The new political sociology of science: Institutions, networks, and power (pp.3-34). Madison, WI: University of Wisconsin Press.

Gherardi, Silvia (2000). Practice-based theorizing on learning and knowing in organizations. Organization, 7(2), 211-223.

Gherardi, Silvia (2008). Situated knowledge and situated action: What do practice-based studies promise?. In Daved Barry & Hans Hansen (Eds.), The Sage handbook of new approaches in management and organization (pp.516-525). Newbury Park, CA: Sage.

Greene, Jennifer C. (2008). Is mixed methods social inquiry a distinctive methodology?. Journal of Mixed Methods Research, 2(1), 7-22.

Haraway, Donna (1988). Situated knowledges: The science question in feminism and the privilege of partial perspective. Feminist Studies, 14(3), 575-599.

Harding, Sandra G. (1986). The science question in feminism. Ithaca, NY: Cornell University Press.

Hu, Yifan (2006). Efficient, high-quality force-directed graph drawing. The Mathematica Journal, 10(1), 37-71.

Krishnan, Armin (2009). What are academic disciplines? Some observations on the disciplinarity vs. interdisciplinarity debate. Working paper Disciplines and Interdisciplinarity, NCRM Working Paper Series, 03/09, https://eprints.ncrm.ac.uk/id/eprint/783/1/what_are_academic_disciplines.pdf [Accessed: October 17, 2020].

Lee, Chul-joo & Sohn, Dongyoung (2015). Mapping the social capital research in communication: A bibliometric analysis. Journalism & Mass Communication Quarterly, 93(4), 1-22.

Maxwell, Joseph A. (2016). Expanding the history and range of mixed methods research. Journal of Mixed Methods Research, 10(1), 12-27.

Maxwell, Joseph A. (2018). The "silo problem" in mixed methods research. International Journal of Multiple Research Approaches, 10(1), 317-327.

Otte, Evelien & Rousseau, Ronald (2002). Social network analysis: A powerful strategy, also for the information sciences. Journal of Information Science, 28(6), 441-453.

Pearce, Lisa D. (2012). Mixed methods inquiry in sociology. American Behavioral Scientist, 56(6), 829-848.

Plano Clark, Vicki L. & Creswell, John W. (Eds.) (2008). The mixed methods reader. Newbury Park, CA: Sage.

Plano Clark, Vicki L. & Ivankova, Nataliya V. (2016). Mixed methods research. A guide to the field. Newbury Park, CA: Sage.

Tashakkori, Abbas & Creswell, John W. (2008). Editorial: Mixed methodology across disciplines. Journal of Mixed Methods Research, 2(1), 3-6, https://doi.org/10.1177/1558689807309913 [Accessed: October 6, 2020].

Tashakkori, Abbas & Teddlie, Charles (Eds.) (2003). Handbook of mixed methods in social & behavioral research. Newbury Park, CA: Sage.

Tashakkori, Abbas & Teddlie, Charles (Eds.) (2010). Handbook of mixed methods in social & behavioral research (2nd ed.). Newbury Park, CA: Sage.

Timans, Rob; Wouters, Paul & Heilbron, Johan (2019). Mixed methods research: What it is and what it could be. Theory and Society, 48(2), 193-216, https://doi.org/10.1007/s11186-019-09345-5 [Accessed: October 4, 2022].

Westmarland, Nicole (2001). The quantitative/qualitative debate and feminist research: A subjective view of objectivity. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 2(1), Art. 1, https://doi.org/10.17169/fqs-2.1.974 [Accessed: February 2, 2022].

Zhao, Dangzhi & Strotmann, Andreas (2015). Analysis and visualization of citation networks. Kentfield, CA: Morgan and Claypool Publishers.

Noemi NOVELLO is a postdoctoral researcher at the University of Milano-Bicocca. In their research, they focus on scientific knowledge and its backstage, also including methodology for social research. Currently they are involved in a project about preterm parents' experiences (ParWelB) implementing, among other things, forms of digital support for parents.

University of Milano-Bicocca
Department of Sociology and Social Research
Via Bicocca degli Arcimboldi, 8
20126, Milano, Italy

Novello, Noemi (2023). Communities of scholars and mixed methods research: Relationships among fields and researchers [47 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 24(1), Art. 19, http://dx.doi.org/10.17169/fqs-24.1.4008.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License

Journal	Frequency	Percentage
Social Science and Medicine	68	1,4%
BMC Public Health	59	1,2%
PLoS ONE	53	1,1%
Journal of Mixed Methods Research	37	0,8%
BMC Health Services Research	31	0,6%
BMJ Open	31	0,6%