Abstract: In both academic and practitioner literature, the term "impact" is conceptualized broadly. Yet the application of impact is construed much more narrowly, in association with (uni)-directional relationships between variables and methodological frameworks oriented towards a positivist approach. Such a conceptualization is problematic, particularly in the context of initiatives that have a goal of internal, individual transformation. Thus, I suggest reconceptualizing impact to acknowledge human agency and explore change more holistically. I argue for a reclaiming of impact by the post-positivist qualitative research community, given the potential of qualitative methodologies to elucidate dialogic understandings of impact and the intersubjective context through which transformation emerges.

Key words: impact; qualitative research; transformation; dialogue; intersubjectivity

In this article, I argue for a reclaiming of the term "impact"¹⁾ in the context of post-positivist qualitative research. In both academic literature and working papers in the field, impact has been conceptualized with acknowledgment of its wide ranging, long-term, and both intended and unintended nature. Yet the methodological conceptualization of impact has been construed much more narrowly, in association with positivist epistemologies, causal, (uni)-directional relationships between variables, experimental (or quasi-experimental) research, and, therefore, primarily with quantitative techniques for data analysis. Indeed, measurement of impact continues to rely mostly on counter-factual approaches to causation (e.g., COOK, SCRIVEN, CORYN & EVERGREEN, 2010; MOHR, 1999) even as alternative conceptualizations of impact and a call for the use of qualitative approaches have been brought forth in methodological literature (e.g., CUNLIFFE & SCARATTI, 2017; DONMOYER, 2012; NORGBEV, 2016; WHITE, 2009). In this sense, there is a significant disconnect between impact as it is conceptualized practically, and the approaches used in impact-related research. [1]

This disconnect is especially problematic when we consider research that focuses on programs or interventions with goals of enabling change that cannot be easily understood or reconstructed through standardized measures. For instance, in the educational sphere, areas such as multicultural education, civic education, and social emotional learning aim to enable transformation among students in how they relate to themselves, their classmates, and more broadly to those different from them. In conflict regions, peace education and peacebuilding programs that bring together participants from different identity groups create opportunities to shift how participants in these programs view members of the adversarial group(s) and conflict narratives (SALOMON, 2002). In these areas and many others, even as research on the impact of programs abounds, it often relies on measures that are questionable in terms of being meaningful to participants, or that fail to take into account how program outcomes are shaped by external factors (e.g., ALLEN & SHARP, 2017; FIRCHOW, 2018; ROSS, 2017). More broadly, the very nature of impact is mostly conceptualized as occurring unidirectionally, where programs are viewed as active causes while individuals are assumed to be passive recipients of the effects of these programs. [2]

This article, therefore, serves as a call to qualitative researchers to reclaim impact as a term that encompasses what can be known through non-positivist approaches to researching change, and that moves beyond the narrow epistemological and methodological frameworks within which it is currently perceived as holding legitimacy. Broadening the frame of reference for this term is important in order to bring the methodological approaches used for researching impact in line with how the term is used in social inquiry, particularly given the emphasis placed in policy and practice contexts on utilizing impact as a central factor in making choices about funding and program support. Moreover, it presents an opportunity to address ethical and justice-oriented implications of defining impact in ways that account for the active role individuals take in their own transformation. [3]

To that end, I argue in this article that impact should acknowledge human agency as well as include a more holistic focus on what individuals take away from their experiences, how they perceive the influence of their experiences, and why these experiences are important. This points to the potential benefit of a dialogic conceptualization of impact, which engages participants' perspectives on their own active engagement with transformation, rather than a view that frames impact as something done to them—and thus to the role that qualitative methodologies can—and should—play in our use of the term. In the following, I define impact as it has been discussed both conceptually and methodologically in the literature prior to discussing the relationship between impact and conceptualizations of causality more specifically. I then utilize a series of empirical examples from my own research to illustrate the limitations of existing approaches, and draw from these examples to propose a holistic, dialogic reconceptualization of impact. [4]

Definitions of impact in much of the literature, as well as among development agencies that engage in transformative initiatives, are wide ranging. For instance, ROGERS (2012) noted that in multi-lateral agencies, impact is viewed as, "the positive and negative, intended and unintended, direct and indirect, primary and secondary effects produced by an intervention" (p.2). Similarly, the WK KELLOGG FOUNDATION (2004), in its distinction between outcomes and impacts, framed the latter as intended or unintended changes "in organizations, communities, or systems as a result of program activities" (p.2) as opposed to specific changes in participant behaviors, knowledge, skills, or status, which are defined as program outcomes. The UNITED KINGDOM RESEARCH EXCELLENCE FRAMEWORK (2019) defined impact, "as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life" (p.68); moreover, the framework explicitly indicated that reductions in negative effects are constitutive of impact, just as are increases in positive effects. When taken together, these suggest that impact is both broadly conceptualized and that it might be explored through a range of possible methodological frameworks. [5]

BHOLA's (2000) distinction between three types of impact indicates a similarly broad understanding of the term, with a focus on temporal rather than conceptual distinctions. He defined impact as one of the following: 1. impact by design, where "an impact can be seen to have resulted from an immediately preceding intervention" (p.163); 2. impact by interaction, the "outcomes of an original intervention interacting with other concurrent interventions made by other agents and agencies, and thereby enhancing or inhibiting effects of the original intervention" (p.164); and 3. impact by emergence: "those outcomes that emerge from the original intervention, its interactions with other concurrent interventions, and historical and cultural processes" (ibid.). BHOLA's distinction between these types of impact is useful in illustrating that, with the exception of impact by design, which focuses on immediate results, impact may only become known over longer timeframes. Even when impact is viewed more narrowly, e.g., as the end of the causal chain (WHITE, 2010), it is clear that longer temporal periods are necessary for understanding what the impact of an initiative actually is. [6]

Finally, BHOLA's (2000) distinction between types of impact also adds to our understanding of what the breadth of impact may potentially entail: not only outcomes directly linked to an intervention, but also changes that result during different temporal periods and/or through a broad range of processes and experiences. In other words, BHOLA pointed to impact as defined not only in terms of outcomes but also in terms of the underlying processes that lead from interventions to outcomes. Moreover, BHOLA noted that impact by emergence includes both unanticipated and unexpected elements, further illustrating that impact cannot be explained only in relation to pre-determined indicators of success. [7]

Based on this, I suggest that we should define impact as a concept that is characterized as open, broad, and reflective of a broad swath of changes that might be attributed to some program or intervention, over a significant length of time, and in multiple areas. As BELZER (2003) argued, the breadth of this definition is necessary in order "to deliver an enriched understanding of what [interventions] can accomplish" (p.46), and to move beyond a focus on individual outcomes. [8]

The wide-ranging understanding of impact narrows considerably when discussing methodological engagement with the term. This is especially true in the evaluation literature; however, a narrow methodological framing of impact occurs more broadly in social inquiry as well. The United States Agency for International Development (USAID), for example, defined impact assessment thus:

"Impact evaluations measure the change in a development outcome that is attributable to a defined intervention; impact evaluations are based on models of cause and effect and require a credible and rigorously defined counterfactual to control for factors other than the intervention that might account for the observed change" (quoted in ROGERS, 2012, p.2). [9]

Likewise, while WHITE (2010), as referenced above, acknowledged that impact often refers to long-term effects, in discussing its methodological framing he focused on assessment of attribution, in a way that draws on counterfactuals to demonstrate how much the intervention contributed to the overall change in the outcome of interest (p.159). By emphasizing causal models that require counterfactuals, as I argue further below, definitions such as those articulated by WHITE and by USAID inevitably create strict limitations on what can be understood within the broad concept of impact. [10]

Of course, not all scholars or practitioners share this view. For example, YU and McLAUGHLIN (2013) discussed measurement of impact as broadly as impact itself is conceptualized: for them, the process is one of "analyzing significant changes, whether positive or negative, intended or unintended, as a result of a particular planned activity, program, intervention, or project, on people's lives" (p.25). BECKER (2001) framed the concept similarly, with a future-oriented emphasis: he defined impact's measurement as "the process of identifying the future consequences of a current or proposed action" (p.312). Likewise, researchers' own "impact case studies" are used by the UNITED KINGDOM RESEARCH EXCELLENCE FRAMEWORK (2019) to evaluate the significance and reach of research in non-academic contexts; the same is true in the Australian Excellence in Innovation program (see also GALÁN-DÍAZ, EDWARDS, NELSON & VAN DER WAL, 2015; MORGAN JONES, MANVILLE & CHATAWAY, 2017; MORGAN JONES, CASTLE-CLARKE, MANVILLE, GUNASHEKAR & GRANT, 2013). [11]

Defining processes of measuring impact in these broader ways allow for methodological orientations encompassing a range of qualitative approaches to inquiry. However, narrower conceptualizations of impact as something to be measured through traditional notions of causality, and, in many cases, counterfactuals, suggest that impact is a concept that exists primarily in a positivist epistemological framework and thus, is a legitimate concept to discuss only in the context of research that relies on experimental designs and/or randomized control trials (RCTs). For example, SONDERGELD and KOSKEU (2011) argued that according to federal government standards, experimental methods are necessary to infer causal relationships (p.95). Similarly, the World Bank and other agencies primarily recognize experimental design and counterfactual-based approaches to assessing impact (JONES, JONES, STEER & DATTA, 2009). Although others have suggested that causality can be understood by identifying underlying processes or attributes that do not require experimental designs or even, necessarily, collection of numerical/quantitative data (e.g., GARBARINO & HOLLAND, 2009), quantifiable data remain the priority in the majority of studies where impact is an area of focus. YU and McLAUGHLIN's (2013) discussion of evaluation in non-profit organizations exemplifies this, highlighting how impact assessment often focuses on quantifiable results even as this misses assessment of many of the ways in which organizations' work can be transformative. [12]

Central to the predominant approach of measuring impact is a unidirectional understanding of causality and causal inference, which persists even as impact as a concept opens itself up to methodological pluralism, and thus reinforces a positivist "claiming" of the term. As MAXWELL (2004a) noted, there are two main schools regarding the nature of causal inference and how it can be ascertained: variable-based approaches where scholars focus on causal connections (Variable A influences Variable B); and mechanism-based approaches via which researchers investigate the processes that lead from one variable to another (see also MAXWELL, 2004b). To this, SMALL (2013) and other sociologists added the qualitative comparative analysis (QCA) approach to causal inference, which focuses on reverse causal questions and identifies combinations of conditions that can be considered necessary and sufficient for an outcome to occur. [13]

Within the variable-based approach, randomized control trials (RCTs) are the "gold standard" in measuring impact. Indeed, central to traditional approaches for researching impact is an emphasis on comparison, specifically through the use of randomized control trials within which treatment groups are compared to non-treatment (control) groups. ROSSI, LIPSEY and FREEMAN (2004) stated that "[a]ll impact assessments are inherently comparative" (p.236) and require considering what the condition would have been in the absence of the program. In discussing how this comparison is made, they encouraged use of either experimental or quasi-experimental research designs involving a randomized control group and an intervention group (in experimental designs), or a nonrandomized participant group and a nonparticipant group in which members are similar to participants (p.237). [14]

Clearly, there are inherent challenges in designing and implementing randomized control trials and experimental designs, particularly when research is constrained by limited resources. However, assumptions of a unidirectional relationship between cause and effect are not limited to RCTs, but are present in social inquiry drawing on divergent methodological frameworks. For example, in ethnographic research, strong causal arguments in much of the literature are based on researchers' use of comparisons in the style of experimental studies (ABEND, PETRE & SAUDER, 2013), and scholars seeking to understand causal mechanisms (an approach that ABELL & ENGEL [2021] referred to as "ethnographic causality") also hold assumptions about unidirectional causal frameworks (e.g., BLEE, 2013; JOBLING, 2014). Researchers focused on the persuasive power of language also tend to look at the effects of certain forms of speaking or writing in ways that assume unidirectionality (e.g., BASAVE & HE, 2016; BULL & WADDLE, 2019; MENEGATTI & RUBINI, 2013). Moreover, investigators proposing suggestions for overcoming the challenges of implementing RCTs still often assume the need for counterfactuals: AZZAM and CHRISTIE (2007) suggested using public or semi-public databases as a source for creating comparison groups when it isn't possible to compare participant and nonparticipant groups. Similarly, BJURULF, VENDUNG and LARSSON (2013) argued for an approach they call measuring cluster effects through triangulation (MCET), which involves "methodological triangulation" (p.57) by using a combination of "shadow controls" (p.59) (estimates of which elements of an outcome are generated from an intervention, as provided by individuals who have special insights, through a process of comparing actual outcomes with estimates of what would have happened without the intervention) and "generic controls" (ibid.) (comparing target group outcomes with outcomes of one or more control groups to which the target group could potentially belong). Thus, even given the legitimacy of process-based approaches that open up possibilities for the use of qualitative techniques for understanding impact, causality remains overwhelmingly associated with a positivist epistemological framework in which causality as unidirectional is assumed (MAXWELL, 2004a; see also MANZANO, 2016; MATTHIAS, DOERING-WHITE, SMITH & HARDESTY, 2021; MOHR, 1999). [15]

Two particular challenges with unidirectional, counter-factual-focused approaches to measuring impact are first, that scholars using them miss the possibility of change being something unknown, relying instead on pre-determined indicators; and second, that in the context of interventions that target change at an individual level, researchers seem to suggest that individuals who are changed are unable to articulate what that change is. This means that the researcher must compare that person's experience to what "might have happened if" in order to determine whether change has occurred. In this sense, in the counter-factual approach, the researcher's predetermination of what impact might be takes on more significance than an individual's conceptualization of self or awareness about one's own internal transformation. This aligns with the onto-epistemological understandings of a positivist approach to social inquiry; however, it does not align with the broad conceptualization of impact which scholars note should encompass intended and unintended, short- and long-term dimensions of change. [16]

To this end, in the remainder of this article, I ask: what might a reimagined framework for researching impact entail? Below, I draw on three empirical examples from my own research to illustrate a possible approach. Through these, I show the nature of impact as first, more than just the effect of some-thing on some-one, and second, as constructed dialogically through reflection with self or with others, during or following transformative experiences. I expand on these points in the discussion. [17]

The first example comes from an interview I conducted as part of a participatory research study about restorative justice initiatives implemented within a correctional institution in New England. In this project, I focused on trying to understand the nature of individuals' experiences within a specific restorative justice program, as well as participants' perspectives about whether and how the program changed them. The following (condensed) excerpt comes from a part of an interview where one participant (P) described his program experiences and his perceptions about how he changed as a result:

"P: I think like letting go and understanding what shame is, and embracing it, not avoiding it, not running from it, so I think ... where we are dealing with shaming, kind of helped me ...²⁾

P: [S]ocial identity, the reason why I say this is because, for the longest, I never knew I felt some type of way about my heritage, where it was like, growing up, we was all together and then at a certain age we split apart. So, after that... it was just my mother, sister and brother, whatever the case is and for the longest, I mean the streets, it was predominantly black, I mean, the neighborhood I grew in so for years I didn't know, but I became ashamed of being Puerto Rican and Dominican, because, I felt like, [my Puerto Rican father] left. So [long pause] yeah, so [voice trembling] boy oh boy. Alright. So, so one day we was doing the exercise [a social identity exercise that is part of the curriculum of this program], and the whole thing started of like oh, are you black, or you—whatever the case is and somebody was like Puerto Rican, you know, [and I was like] no! And ... I actually had to think about it, like, I wasn't always proud to be Latino ... I felt, you know, growing up as a kid, whatever, as I came to prison, whatever, and I, I basically grown up in prison, and being around people of my heritage, and not understanding, from, not a child's mind but a mature man, that I was actually ashamed of it because in my head it was like that man's leaving ... I see that it was just a perspective but I never knew that there was something I was ashamed of, until going through the exercises, and it was like, somebody asked that question and I was going to answer, oh yeah, I am proud of being [Puerto Rican] but it wasn't always that way, and it was something hard to admit, like why don't you identify with your own race? You know, so there was like a, a lot of shame in there ...

P: It was something that I never really thought about, it was just something that happened and it was in discussion like oh shit! It kind of hit me.

P: And that's why we kind of, put it into the curriculum [this participant was part of a group that was involved in revising the initial curriculum for the restorative justice program]. Cuz it's something people don't talk about ... It was a good thing to bring in but it's something as simple as that, that kind of, [laughs], you know blows your mind. So like I said on many levels, it helped me out ... it was just, I was able to work through some issues and now being able to listen and be able to admit that, listen, maybe this, I was ashamed of this and I can address it." [18]

Among the conversations I had with restorative justice participants, this one stood out in two ways. First, it stood out in terms of the participant's statements about how he came to realize his shame through an intersubjective experience. As he noted, "I never knew that there was something I was ashamed of, until going through the exercises, and it was like, somebody asked that question." In other words, it was from engaging in dialogue with other participants in the restorative justice program that he came to better understand himself and thus "be able to admit that, listen, maybe this, I was ashamed of this and I can address it": that is, be in a position to change. His statements to this effect illustrate that the impact of this program, for this participant, was emergent, and more specifically, emergent through the experience of engaging with others in the restorative justice program. [19]

The second key piece of this excerpt that stands out is the way that it was articulated. Having not been at the session where this particular conversation took place, how could I understand the significance of what had been experienced, and how would I know what emotions this participant felt on this day? My understanding of the impact of the restorative justice initiative on this individual came from the fact that in the context of his dialogic encounter with me, this participant experienced (re-experienced?) his transformation in the context of that dialogue. The emotions attached to discussing his identity and shame about that identity were explicit in the conversation, and beyond his verbal utterances, were expressed in a particularly significant way: through tears that rose up in his eyes as he spoke about shame, through long pauses in his speech and the difficulty that he clearly experienced in articulating himself. That is, it was through our dialogue about his experience that he was able to communicate the change that the program enabled. As a result of how he communicated this change, I had the sense of experiencing it alongside him—I almost viscerally could feel the significance he attributed to his transformation. To this end, the impact of the program on this participant not only emerged intersubjectively, but also became explicit through the dialogue between the two of us. This suggests the importance of considering dialogic approaches to researching impact, which allow for participants to articulate—and at times to understand through that articulation—how they were transformed by the program or initiative being assessed. [20]

A second example comes from a follow up conversation held with a woman I had initially interviewed two years prior as part of a different study. My focus in this project was on the perceived impact of participating in programs implemented by Jewish-Palestinian encounter organizations in Israel and how these experiences fit into individuals' life histories. V, a Jewish woman, had just finished her military service in the Israel Defense Forces when I first interviewed her; two years later she contacted me to let me know she wanted to meet because she had come to some important realizations about the significance of her encounter program participation. [21]

During our second conversation, V explained to me that she was part of a program for young Israeli leaders and, as part of that program, had recently organized a set of activities for her group to get to know Palestinian citizens. After discussing the specifics of these activities, V told me,³⁾

"I felt that the messages I wanted to convey, were conveyed to the rest of my group … but I think that, like, the kind of getting to know one another that we had in the encounter program—it's almost impossible in other situations. I mean ... [these activities with the leadership group] just brought into focus for me how much the experience [in the encounter program] actually changed my life path. I think I am a different person as a result of [this encounter program experience]." [22]

V's comment about feeling she was a different person speaks to her understanding of the impact of participating in the encounter program. What is particularly important about her statement, however, is not just what that impact was, but that she realized it retroactively, in the context of other activities she had experienced. In other words, the impact V refers to is not something she could have articulated in the immediate context of her participation in the organization's activities, or even shortly after. For V, rather, impact was something that emerged over time, in the context of participating in other initiatives. Specifically, it emerged through active engagement and reflection (dialogue with self) about her encounter participation in the context of participating in these other initiatives. Moreover, V's self-reflection during the period between our two meetings enabled her to articulate this impact in the context of her conversation with me. Thus, as in Example 1, impact in this context both emerged and was articulated in the context of dialogic reflection: with herself, initially, and then with me. [23]

Later in our conversation, V said something that illustrates in a different way how impact can emerge through dialogue. Discussing her experiences in the encounter program, V talked about how it was only while she was a soldier in the Israel Defense Forces that she fully realized the significance of what another encounter program participant had said in a conversation they had several years prior. V reminded me that when interacting with Palestinian citizens in the encounter program, one of them had indicated to her that what each of the two saw and absorbed about the "realities" of the Israel-Palestinian conflict was quite different. She told me,

"[in the army] I looked at the Arab media and saw pictures ... in Jewish society we don't, like, the second someone is dead, it's like you feel like they're innocent, like they're holy, I mean, that's how we speak about the dead among us, and in Gaza, at least, after Oferet Yetzuka,⁴⁾ it was like simply, [the Palestinian media] didn't, they didn't hold back. There were pictures of, like, pieces of people. After a terrorist attack [in Israel] you don't see pictures like that, you see blood on the street, but you don't see ... like, let's say, when they killed Yassin [a Palestinian Hamas leader assassinated by the Israel Defense Forces], Arab media showed pictures of his brain, scattered ... and I guess that's what he was talking about, when he said that I don't see the whole picture.

V: [Name of encounter program participant], when he said that he sees other things that I am not exposed to." [24]

In explaining her realization of what was meant, "[h]e said that he sees other things that I am not exposed to," V articulated a new sense of understanding about what she experienced several years previously. Here, it is important to note that she gained that understanding in the context of her dialogue with me, as suggested by her statement, "I guess that's what he was talking about." In other words, V's language indicated that it was in the moment of discussing that incident that her realization occurred. Again, this points to the nature of impact as something that is emergent through dialogue, rather than measured unidirectionally through pre-determined variables. In this particular case, V's transformation was reflective of what can happen in the context of interviews aimed towards fostering participant change (e.g., WOLGEMUTH & DONOHUE, 2006), although this particular interview was oriented more towards understanding impact substantively rather than fostering it methodologically. It also reflects further the nature of impact as an active phenomenon that often happens retroactively: V's comments illustrated very clearly that she came to these realizations, not because the encounter program directly changed her, but because through the course of her encounter program experiences, along with other experiences, V was able to better understand the complexity of socio-political phenomena and thus realize how she herself changed in the wake of her interactions with Palestinian citizens. [25]

The third example comes from a set of interviews conducted in a follow up study to the one discussed in Example 2, which focused on the processes motivating participants in Jewish-Palestinian educational initiatives to engage in social activism. In this interview excerpt, U, a Palestinian woman and alumna of the organization's programs, was telling me about her decision to remove her hijab. U talked about removing her hijab as something that happened as a result of her participation in organizational activities, but not directly: instead, it was a decision she made as a result of dialogic experiences throughout her time there. U told me,

"[I wore the hijab for] three years. And in the last 2 years [of wearing it], I always wanted to [take it off], but I didn't have the courage, I wasn't comfortable [wearing the hijab] but I knew that in my society, it wouldn't be accepted, and it was really hard socially. But at the moment that I changed my mind, I told myself, I changed my way of thinking about it and simply decided, I will do it and whoever accepts it will accept it and whoever doesn't, won't. I took that from [the encounter program].

Researcher: What was that connection? I mean, when you say that you changed your perspective, how is that connected to what you got from the program?

U: Because we did all of these activities, I mean, it didn't connect directly but it did, because when your awareness comes then everything comes together and everything changes together. It's one of the things that really influenced me and changed in me ... it's not like it was directly like, something we learned in a specific activities, it's not something like that, but it was the change that came all together.

Researcher: So what was actually the change, that is, when you talk about the change in the specific sense of taking off your hijab? You said earlier that you didn't have the courage to do that, and then, you said that you changed and you took things in a different ways ... so are you saying that you got up the courage to do it while you were [a participant in the organization's activities]?

U: During that year [when I was a participant], my self-confidence increased, and also, the way I think about my society and the way I want to act within it and how I want to interact with it and speak to it ... I always thought that as someone with a hijab I would better represent my society and would be from it more and would be able to defend it and defend myself, but now that I've changed, I told myself, I can do this also without [the hijab]. Like, I always felt that the hijab was something that was within me and within something central that if I let go [of the hijab] I'll be letting go of part of my identity. But I changed my perspective in terms of how I think of myself and how, in terms of my identity, [how I think] about what my identity is made up of, and I tell myself, it's a piece of cloth, so I can take it off and I can still represent and I can still be part of this society." [26]

In the interview, U said that her self-understanding emerged from conversations with others in the organization, as well as from conversations that happened in the context of leaving the group each week and going back to her family and her community. In that sense, for U, impact emerged iteratively, through multiple dialogical processes, illustrating the nature of impact as a phenomenon that is realized often indirectly, while reflecting upon or comparing experiences from different contexts. In this case, it was not directly in the context of experiences within the organization, but more broadly through conversations with others, both within and outside of the organization, that U came to realize that the hijab [for her] was "just a piece of cloth," and that her identity as a member of her community was not connected to the hijab. U's discussion about removing her hijab also points, again, to the nature of impact as emergent over time: she clarified that the decision to remove it developed as her awareness shifted during the entire period when she was participating in the organization's programs, rather than as the immediate, direct result of some activity or set of activities. [27]

The examples above help us see that impact, far from being unidirectional or passive, is claimed through active, iterative reflection on the part of the individual experiencing that impact, as well as during and through intersubjective engagement among community members and between researchers and research participants. Intersubjective engagement in this context refers to the process through which people come together to jointly create understanding (GILLESPIE & CORNISH, 2009; RUSSELL & KELLY, 2002; ZURN, 2008). As SHOTTER (2010) stated,

"[i]nstead of taking it for granted that we understand another person's speech simply by grasping the inner ideas they have supposedly put into their words, we should recognize that it is from within the dynamically sustained context of ... actively constructed relations that what is uniquely being talked about gets its meaning" (p.278). [28]

In other words, rather than being experienced in a purely internal or private manner, impact emerges in the context of relational interactions that enable iterative reflection on the part of the individual claiming that transformation has occurred. The three examples discussed above allow me to illustrate in multiple ways how change occurs and is understood in ways that challenge traditional approaches to impact assessment, and in doing so, lend legitimacy to the call for qualitative researchers to reclaim this term. First, through these examples, I show that change happens through dialogue with others in the context of interventions (or through dialogue with oneself when reflecting on previous experiences), thus pointing to the significance of conceptualizing impact as an active endeavor—this challenges the passive role accorded to "those impacted" in traditional cause-and-effect approaches to measuring impact. Second, I demonstrate that impact occurs over time—in many cases, over months and years rather than over the days or weeks of a program meant to set the stage for transformation. To this end, it is also important to note that transformation often happens indirectly rather than in the moment of interventions: as both the second and third examples indicate, it occurs as a result of reflecting upon or re-experiencing what was experienced, through dialogue with others as well as in dialogue with oneself about other life experiences (LI & ROSS, 2020; see also PRODINGER & STAMM, 2010). CUNLIFFE and SCARATTI's (2017) concept of opacity, that is, the "notion that much of what we do is not immediately transparent to us and can be perplexing if we are asked to articulate why and what we are doing in the moment of doing it" (p.31), elucidates the importance of retrospective reflection as a basis for understanding impact. This suggests that a conceptualization of impact that relies on predetermined indicators of change (usually short-term) is insufficient, and that a more open approach is necessary. [29]

Finally, through all three examples, I point to how impact emerges through reflection that occurs in the context of research itself, that is, through dialogic interview experiences that enable what WAY, KANAK ZWIER and TRACY (2015) referred to as "flickers of transformation": new understandings, including understandings among participants about how they have changed (for an example of how researchers can facilitate this through interview research, see: LEVITAN, MAHFOUZ & SCHUSSLER, 2018). Of particular importance here is the idea that the research participation enables not only reflection about changes that may have occurred previously, but also is transformative in its own right. That is, through dialogic interviews, research participants can experience spaces where transformation occurs and where impact emerges. [30]

The significance of reflection for the articulation of impact, and the emergent nature of change, point to the need for a reconceptualization of impact that shifts in two main ways from how it is primarily understood in most methodological literature, as discussed above. First, a shift away from dominant epistemological frameworks suggests that rather than impact existing as something that can be operationalized and measured through pre-determined indicators and use of comparison, we must understand impact within a constructivist framework where data is co-constructed in the context of research—just as realities are co-constructed within the context of social interactions (GUBA & LINCOLN, 1989; LINCOLN & GUBA, 1985; STUFFLEBEAM, 2008). More concretely, as I illustrate through these examples, we should reconceptualize the notion of impact, not as something that happens to a person but as something that an individual is actively engaged in constituting through the process of self-reflection (dialogue with self) or dialogic engagement with others. To this end, we might learn from STERN et al.'s (2012) argument that participatory approaches have a positive role to play in measuring impact, precisely because researchers utilizing these approaches "do not see recipients of aid as passive recipients but rather as active 'agents'. Within this understanding, beneficiaries have 'agency' and can help 'cause' successful outcomes by their own actions and decisions" (p.29, see also GATES & DYSON, 2017). [31]

Second, through the examples above, I illustrate that a reconceptualization of impact must foreground mutually constituted relationships rather than a unidirectional understanding of cause and effect. In particular, I emphasize contexts in which impact as change is expressed and/or experienced through social interactions and dialogic processes, pointing to intersubjective communicative relationships (HABERMAS, 1984 [1981], 1987 [1981]) as the spaces within which impact emerges and is understood. As internal understandings of how one has changed are expressed, they are no longer private but enter the intersubjective domain (FREEMAN, 2011; LI & ROSS, 2020). And it is through this intersubjective domain, either during transformative experiences (as in Examples 1 and 3), in dialogue with oneself/self-reflection about those experiences (as in Example 2), or in the context of research about those transformative experiences (as in Examples 1 and 2), that impact is realized and made explicit. Moreover, if impact is not passive but rather an active endeavor, where the some-ones who "are impacted" actively engage in their own transformation, understanding impact requires a methodological approach that takes into account how it is expressed and recognized dialogically, in the context of inter-subjective engagement (MacINTOSH, BEECH, ANTONACOPOULOU & SIMS, 2012). This approach aligns with REICHERTZ's (2018) discussion about "communicative power," specifically that communicative action is oriented towards having an impact, that is, "making someone do what we expect him to do" (p.292). REICHERTZ argued that impact is emergent from the communicative power of the social situation within which it arises, rather than defined through classic understandings of causal relationship; similarly, I suggest that impact in the form of transformation is emergent through social situations and intersubjective communication. In the context of researcher-participant interactions, understanding impact aligns with the approach used in interventionist evaluations where the role of the researcher is to enable a process of meaning making among stakeholders through the evaluation itself, as per MÄÄTTÄ and RANTALA (2007). [32]

BARAD's (2003) concept of agential intra-action, drawing from the field of quantum physics, provides a possible framework for re-imagining impact in a way that takes into account its active, reflexive, and dialogic nature, and serves as an alternative to dominant approaches to impact research. BARAD argued that there is a causal relationship between "the apparatuses of bodily production and the phenomena produced" (p.814); in other words, "a causal relationship between specific exclusionary practices embodied as specific material configurations of the world (i.e., discursive practices/(con)figurations rather than 'words') and specific material phenomena (i.e., relations rather than 'things')" (ibid.). She stated that the primary unit of focus is the phenomenon made up of inter-acting components, rather than the components themselves or a set of independently existing components interacting. Intra-action, further, includes what BARAD (p.815) termed an "agential cut" (set in contrast with a Cartesian "cut" that sets the binary subject-object distinction). The "agential cut", in BARAD's conceptualization, refers to the boundaries of a phenomenon (made up of interacting components) as it occurs in specific interactions; the meaning attributed to phenomena are likewise contingent upon the dynamics of these interactions. [33]

How does agential intra-action apply to thinking about impact? For one, it opens up a new way of thinking about causality: rather than some thing impacting some other thing (or person), it is a specific intra-action among components that creates a phenomenon of interest. Thus, we might think about the intra-action of individuals occurring in some program or initiative as enabling change and leading to the emergence of impact (and thereafter articulation of what this impact is), rather than the initiative impacting the (passive) individuals involved with it. Within BARAD's framework, in other words, the meaning of impact is created through dialogic interactions (within the experiences themselves, with oneself in the context of other life experiences, and in the context of research about the experiences), rather than being measured from an external, third-person point of view. Agential intra-action can therefore be understood as a way of conceptualizing impact as an active phenomenon that requires engaged interaction rather than passive receipt of some new understanding or cognitive awareness that becomes an operationalized indicator of transformation. Moreover, the fluid and dynamic nature of phenomena within BARAD's framework reaffirms the significance of a contextually constituted process of meaning making, wherein the relationship among individuals—as researcher/participants as much as among participants in some initiative aimed at transformation—shapes the significance of the phenomenon itself, that is, how we understand impact in that context. Indeed, as MARN and WOLGEMUTH (2017) pointed out, within the framework of agentic realism, "the interview process itself [is] complicit in the production of identity performances through its material presence/effects" (p.372). In other words, transformation and impact are emergent within the context where they are articulated. [34]

There are, of course, challenges inherent in conceptualizing impact this way, especially in terms of applying BARAD's (2003) concept of agential intra-action as a framework for understanding and measuring impact. For one, this approach necessitates a dual-level view on the part of the researcher, who must consider both the dialogue of research participants with others, during and in the wake of an intervention, and their own dialogue with participants, as shaping change and thus enabling the articulation of impact. In other words, researchers must tease apart impact claimed in the interaction between themselves and their participant, from impact claimed due to the interaction between the participant, an intervention, and other experiences enabling reflection on the experiences of that intervention. This suggests the importance of analytic strategies that allow researchers to focus not only on the substance of conversation, where evidence of impact emerging from an intervention might be articulated, but also on narrative form and on language use that can provide insight into how the interview itself creates a space for reflection, and through reflection, for transformation. Approaches to interviewing that are oriented towards building in time for reflection (e.g., NARDON, HARI & AARMA, 2021), may also be helpful in this respect, as may the Bakhtinian approach to narrative analysis that SKINNER, VALSINER and HOLLAND (2001) proposed. [35]

A second, related issue to address is BARAD's (2003) conceptualization of a "phenomenon" as the unit of focus rather than the independent components interacting to create/enable that phenomenon. BARAD placed primary emphasis on interpersonal construction and creation of impact through the processes of active, iterative reflection; this means that the role of environmental and contextual forces may be backgrounded in the process of making meaning of transformation and impact. However, given the lack of attention to the interactions of individuals with their environments that characterizes much of the scholarship on impact and individual transformation (ROSS, 2017), finding ways to foreground this is an imperative for a reconceptualization of impact. Indeed, the interactions of individuals with their environments, as seen especially in Examples 2 and 3 above, are central to enabling the kind of self-reflection that allows for impact to emerge. This further reinforces the need for analytical strategies that can explore what NEALE, HENWOOD and HOLLAND (2012, p.5) called "timescapes," or "flows of time"—the ways that individuals actively make sense of their past and navigate their futures (see also HENDERSON, HOLLAND, McGRELLIS, SHARPE & THOMSON, 2012; HOLLAND, 2011; NEALE & FLOWERDEW, 2003). For instance, DENNIS' (2020) analysis of change using time analytics provides a schematic model through which qualitative researchers can reconstruct change over time that manifests implicitly. Though DENNIS focused on understanding culture, her schematic model can be used and adapted at an individual level to make explicit the emergence of impact through self-reflection. [36]

Despite these challenges, the concept of agential intra-action, and constructivist frameworks more broadly, help reconceptualize impact in a way that transcends many of the limitations of its use in existing methodological literature and returns agency to individuals who are the focus of inquiry into and about impact. In particular, these frameworks enable us to shift away from a static and unidirectional orientation towards impact, to a reflective and contextually-driven understanding, particularly in the context of interventions aimed at individual change. They allow us to more deeply understand what change is and how it happens, and in ways that provide insight into the meaning accorded to that change by those who experience it. Such an understanding is necessary if we are to engage in measuring impact in a way that takes into account the complexity inherent in social life, and thus furthers the imperative for qualitative researchers to reclaim the term in the work with which we engage. [37]

1) I use quotation marks here around the word impact to note the significance of the term; however, in the remainder of the article the term is used without quotation marks. <back>

2) All excerpts are drawn from interviews I transcribed word for word. The excerpts in Examples 2 and 3 I translated from the original language (Hebrew) in which the interviews were conducted. <back>

3) Citizens of Israel who identify as part of the Palestinian nation but who hold Israeli citizenship are referred to officially as "Israeli Arabs" or "Arab Israelis." I use the term Palestinian citizens (or Palestinians) to reflect the way that the overwhelming majority of my research participants define their identity. <back>

4) Oferet Yetzuka is the Hebrew term for "Operation Cast Lead," an Israel Defense Forces military assault on the Gaza Strip that was launched in December 2008. The assault lasted for twenty-two days. <back>

Abell, Peter & Engel, Ofer (2021). Subjective causality and counterfactuals in the social sciences: Toward an ethnographic causality?. Sociological Methods & Research, 50(4), 1842-1862.

Abend, Gabriel; Petre, Catlynm & Sauder, Michael (2013). Styles of causal thought: An empirical investigation. American Journal of Sociology, 119(3), 602-654.

Allen, Nathaniel & Sharp, Travis (2017). Process peace: A new evaluation framework for track II diplomacy. International Negotiation, 22(1), 92-122.

Azzam, Tarek & Christie, Christina (2007). Using public databases to study relative program impact. The Canadian Journal of Program Evaluation, 22(2), 57-68.

Barad, Karen (2003). Posthumanist performativity: Toward an understanding of how matter comes to matter. Signs: Journal of Women in Culture and Society, 28(3), 801-831.

Basave, Amparo Elizabeth Cano & He, Yulan (2016). A study of the impact of persuasive argumentation in political debates. In Kevin Knight, Ani Nenkova & Owen Rambow (Eds.) Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp.1405-1413). San Diego, CA: Association for Computational Linguistics.

Becker, Henk A. (2001). Social impact assessment. European Journal of Operational Research, 128, 311-321.

Belzer, Alisa (2003). Toward broadening the definition of impact in professional development for ABE practitioners. Adult Basic Education, 13(1), 44-59.

Bhola, Harbans S. (2000). A discourse on impact evaluation: A model and its application to a literacy intervention in Ghana. Evaluation, 6(2), 161-17.

Bjurulf, Staffan; Vedung, Evert & Larsson, Carl G. (2013). A triangulation approach to impact evaluation. Evaluation, 19(1), 56-73.

Blee, Katherine (2013). How options disappear: Causality and emergence in grassroots activist groups. American Journal of Sociology, 119(3), 655-681.

Bull, Peter & Waddle, Maurice (2019). "Let me now answer, very directly, Marie's question": The impact of quoting members of the public in Prime Minister's Questions. Journal of Language Aggression and Conflict, 7(1), 56-78.

Cook, Thomas D.; Scriven, Michael; Coryn, Chris L.S. & Evergreen, Stephanie D.H. (2010). Contemporary thinking about causation in evaluation: A dialogue With Tom Cook and Michael Scriven. American Journal of Evaluation, 31(1), 105-117.

Cunliffe, Ann L. & Scaratti, Giuseppe (2017). Embedding impact in engaged research: Developing socially useful knowledge through dialogical sensemaking. British Journal of Management, 28(1), 29-44.

Dennis, Barbara (2020). Walking with strangers: Critical ethnography and educational promise. New York, NY: Peter Lang.

Donmoyer, Robert (2012). Can qualitative researchers answer policymakers' what-works question?. Qualitative Inquiry, 18(8), 662-673.

Firchow, Pamela )2018(. Reclaiming everyday peace: Local voices in measurement and evaluation after war. Cambridge: Cambridge University Press.

Freeman, Melissa (2011). Validity in dialogic encounters with hermeneutic truths. Qualitative Inquiry, 17(6), 543-551.

Galán-Díaz, Carlos; Edwards, Peter; Nelson, John D. & van der Wal, René (2015). Digital innovation through partnership between nature conservation organisations and academia: A qualitative impact assessment. Ambio, 44(4), 538-549.

Garbarino, Sabine & Holland, Jeremy (2009). Quantitative and qualitative methods in impact evaluation and measuring results. Birmingham: University of Birmingham Press.

Gates, Emily & Dyson, Lisa (2017). Implications of the changing conversation about causality for evaluators. American Journal of Evaluation, 38(1), 29-46.

Gillespie, Alex & Cornish, Flora (2009). Intersubjectivity: Towards a dialogical analysis. Journal for the Theory of Social Behaviour, 40(1), 19-46.

Guba, Egon G. & Lincoln, Yvonna S. (1989). Fourth generation evaluation. Thousand Oaks, CA: Sage.

Habermas, Jürgen (1984 [1981]). The theory of communicative action (Vol. 1). Boston, MA: Beacon Press.

Habermas, Jürgen (1987 [1981]). The theory of communicative action (Vol. 2). Boston, MA: Beacon Press.

Henderson, Sheila; Holland, Janet; McGrellis, Sheena; Sharpe, Sue & Thomson, Rachel (2012). Storying qualitative longitudinal research: Sequence, voice and motif. Qualitative Research, 12(1), 16-34.

Holland, Janet (2011). Timescapes: Living a qualitative longitudinal study. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 12(3), Art. 9, https://doi.org/10.17169/fqs-12.3.1729 [Accessed: March 2, 2022].

Jobling, Hannah (2014). Using ethnography to explore causality in mental health policy and practice. Qualitative Social Work, 13(1), 49-68.

Jones, Nicola; Jones, Harry; Steer, Lisbet & Datta, Ajoy (2009). Improving impact evaluation production and use. Working Paper 300, Overseas Development Institute, London, UK, https://cdn.odi.org/media/documents/4158.pdf [Accessed: March 1, 2022].

Levitan, Joseph; Mahfouz, Julia & Schussler, Deborah L. (2018). Pragmatic identity analysis as a qualitative interview technique. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 19(3), Art. 18, https://doi.org/10.17169/fqs-19.3.3032 [Accessed: March 4, 2022].

Li, Peiwei & Ross, Karen (2020). Validity of transformative experiences: An unfolding. Qualitative Inquiry, 27(3-4), 385-396.

Lincoln, Yvonna S. & Guba, Egon G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

Määttä, Mirja & Rantala, Kati (2007). The evaluator as a critical interpreter. Evaluation, 13(4), 457-476.

MacIntosh, Robert; Beech, Nic; Antonacopoulou, Elena & Sims, David (2012). Practising and knowing management: A dialogic perspective. Management Learning, 43(4), 373-383.

Manzano, Ana (2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342-360.

Marn, Travis M. & Wolgemuth, Jennifer R. (2017). Purposeful entanglements: A new materialist analysis of transformative interviews. Qualitative Inquiry, 23(5), 365-374.

Mathias, John; Doering-White, John; Smith, Yvonne & Hardesty, Melissa (2021). Situated causality: What ethnography can contribute to causal inquiry in social work. Social Work Research, 45(1), 7-18.

Maxwell, Joseph A. (2004a). Using qualitative methods for causal explanation. Field Methods, 16(3), 243-264.

Maxwell, Joseph A. (2004b). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3-11.

Menegatti, Michela & Rubini, Monica (2013). Convincing similar and dissimilar others: The power of language abstraction in political communication. Personality and Social Psychology Bulletin, 39(5), 596-607.

Mohr, Lawrence B. (1999). The qualitative method of impact analysis. The American Journal of Evaluation, 20(1), 69-84.

Morgan Jones, Molly; Manville, Catriona & Chataway, Joanna (2017). Learning from the UK's research impact assessment exercise: A case study of a retrospective impact assessment exercise and questions for the future. The Journal of Technology Transfer, https://doi.org/10.1007/s10961-017-9608-6 [Accessed: March 2, 2022].

Morgan Jones, Molly, Castle-Clarke, Sophie; Manville, Catriona; Gunashekar, Salil & Grant, Jonathan (2013). Assessing research impact: An international review of the excellence in innovation for Australia trial. Santa Monica, CA: RAND Corporation.

Nardon, Luciara; Hari, Amrita & Aarma, Katlin (2021). Reflective interviewing: Increasing social impact through research. International Journal of Qualitative Methods, 20, 1-12, https://doi.org/10.1177/16094069211065233 [Accessed: January 21, 2022].

Neale, Bren & Flowerdew, Jennifer (2003). Time, texture and childhood: the contours of qualitative longitudinal research. International Journal of Social Research Methodology, 6(3), 189-199.

Neale, Bren; Henwood, Karen & Holland, Janet (2012). Researching lives through time: An introduction to the Timescapes approach. Qualitative Research, 12(1), 4-15.

Norgbey, Enyonam B. (2016). Debate on the appropriate methods for conducting impact evaluation of programs within the development context. Journal of Multidisciplinary Evaluation, 12(27), 58-66.

Prodinger, Birgit & Stamm, Tanja Alexandra (2010). Self-reflection as a means for personal transformation: An analysis of women’s life stories living with a chronic disease. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 11(3), Art. 6, https://doi.org/10.17169/fqs-11.3.1379 [Accessed: March 3, 2022].

Reichertz, Jo (2018). From understanding to impact: communicative power. In Michaela Pfadenhauer & Hubert Knoblauch (Eds.), Social constructivism as paradigm?: The legacy of the social construction of reality (pp.292-309). London: Routledge.

Ross, Karen (2017). Youth encounter programs in Israel: Pedagogy, identity, and social change. Syracuse, NY: Syracuse University Press.

Rossi, Peter H.; Lipsey, Mark W. & Freeman, Howard E. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: Sage.

Russell, Glenda M. & Kelly, Nancy H. (2002). Research as interacting dialogic processes: Implications for reflexivity. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(3), Art. 18, https://doi.org/10.17169/fqs-3.3.831 [Accessed: March 2, 2022].

Salomon, Gavriel (2002). The nature of peace education: Not all programs are created equal. In Gavriel Salomon & Baruch Nevo (Eds.), Peace education: The concept, principles, and practices around the world (pp.3-14). Mahwah, NJ: Lawrence Erlbaum Associates.

Shotter, John (2010). Situated dialogic action research. Organizational Research Methods, 13(2), 268-285.

Skinner, Debra; Valsiner, Jaan & Holland, Dorothy (2001). Discerning the dialogical self: A theoretical and methodological examination of a Nepali adolescent’s narrative. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 2(3), Art. 18, https://doi.org/10.17169/fqs-2.3.913 [Accessed: March 3, 2022].

Small, Mario L. (2013). Causal thinking and ethnographic research. American Journal of Sociology, 119(3), 597-601.

Sondergeld, Toni A. & Koskey, Kristin L.K. (2011). Evaluating the impact of an urban comprehensive school reform: An illustration of the need for mixed methods. Studies in Educational Evaluation, 37(2-3), 94-107.

Stern, Elliot; Stame, Nicoletta; Mayne, John; Forss, Kim; Davies, Rick & Befani, Barbara (2012). Broadening the range of designs and methods for impact evaluations. Working Paper 38, Department for International Development, London, UK, https://www.oecd.org/derec/50399683.pdf [Accessed: March 1, 2022].

Stufflebeam, Daniel L. (2008). Egon Guba's conceptual journey to constructivist evaluation. Qualitative Inquiry, 14(8), 1386-1400.

Way, Amy K.; Kanak Zwier, Robin & Tracy, Sarah J. (2015). Dialogic interviewing and flickers of transformation: An examination and delineation of interactional strategies that promote participant self-reflexivity. Qualitative Inquiry, 21(8), 720-731.

White, Howard (2010). A contribution to current debates in impact evaluation. Evaluation, 16(2), 153-164.

Wolgemuth, Jennifer R. & Donohue, Richard (2006). Toward an inquiry of discomfort: Guiding transformation in "emancipatory" narrative research. Qualitative Inquiry, 12(5), 1012-1021.

Yu, Stan & McLaughlin, Darrell Arthur (2013). Program evaluation and impact assessment in international non-governmental organizations (INGOs): Exploring roles, benefits, and challenges. Canadian Journal of Nonprofit and Social Economy Research, 4(2), 23-36.

Zurn, Christopher F. (2008) Intersubjectivity. In William A. Darity (Ed.), International encyclopedia of the social sciences (Vol. 4, pp.116-117). Detroit, MI: Macmillan Reference USA.

Karen ROSS is an associate professor in the Conflict Resolution Programs at the University of Massachusetts-Boston. In her research, she focuses on questions at the intersection of grassroots peace-building, education, and social activism. She is particularly interested in methodological and conceptual issues related to assessing the impact of endeavors aimed at nonviolent social change.

Department of Conflict Resolution, Human Security & Global Governance
University of Massachusetts Boston Wheatley 04-128a, 100 Morrissey Blvd
Boston, MA 02125, USA

Ross, Karen (2022). Reclaiming impact in qualitative research [37 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 23(2), Art. 5, https://doi.org/10.17169/fqs-22.2.3826.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License