Abstract: Any debate about the quality of research may be wise to include how the knowledge claims that result from that research are generalised. This paper is about the different conceptions of making knowledge claims general, making them applicable to more than one situation. The more general a knowledge claim, the more significant it becomes. A quality of qualitative research debate needs to identify and compare the different priorities each epistemology has regarding generalisation. After outlining these priorities for four overlapping epistemologies, scientific, systems thinking, argument, and interpretive, this paper will use the ironic view to argue that each epistemology might learn from the others so as to enrich their own priorities. Identification of difference may not only improve the quality of qualitative knowledge but may also provide the opportunity to creatively define what is meant by the quality of qualitative research.

To make a contribution to knowledge, to be a knowledge claim, most epistemologies (communities of practice) argue that research results need more than having been gathered under some explicit methodology. These epistemologies require that significant (K)nowledge claims be justified to a community and, relevant to this paper, they have some generic properties such as being applicable to more than one situation. For example if I claim to "know" that my small work group had been creative once last week, can I call this knowledge? Some epistemologies would say not, unless my knowledge claim was justified to others and that it was also shown to be true in other situations, it should not really be called a (K)nowledge claim. A preferable term may be "description"; I have described my small work group. If however I could justify to a sceptical audience that my work group was creative across a range of tasks then it would be more appropriate to call this a knowledge claim, especially if I could explain why the group was creative; we would (K)now something. [1]

For scientists (POPPER, 1963), knowledge claims tend to be supported by physical measurements or theories that have to be shown to be true across space and time, that is, universal. For interpretivist (KLEIN & MYERS, 1999), it is more the interpretation of phenomena that needs to be generic, to apply to more than one situation. If the interpretation is to be critical, then the aim seems to be to improve the lives of more than one person. For systems thinkers, knowledge is bounded (RESCHER, 1979) so generalisation suggests seeking new boundaries. For the argumentative epistemology (CROSSWHITE, 1996) a rational argument explicitly declares its level of generality which in turn suggests what is appropriate supporting evidence. In summary, generalising has a different set of priorities within each of these four epistemologies, yet it can be seen that these priorities have some relevance within all the epistemologies. For example, improving the world, new boundaries and argument are all relevant to science. By making these different priorities for generalising knowledge explicit, it is believed that the debate on the quality of qualitative research might be both improved and better appreciated relative to other epistemologies. Moreover, this paper might also expand the debate on defining knowledge a little by suggesting that these differences be used to further expose a contrast between the one-world view of scientific knowledge and the ironic or multiple perspective view, where differing viewpoints are not to be reconciled but rather to be seen as a creative source of new knowledge. Identifying differing generalisation priorities may assist all epistemologies improve and clarify their own conceptions of research findings. [2]

Science usually declares its primary purpose to be the pursuit of general truths, universal laws, which are true across time and space (MATT, 2001). The laws of gravity should be true wherever you are in the cosmos and as true one billion years ago as in one billion years. Therefore any research findings, that are knowledge claims, should aim to be universally true. Science claims to only recognise "universal" truths, one boundary, which is the physical universe through all time. Knowledge claims justified from one experiment need to be generalised, to justify that they are universally true. This is similar to saying they need to be "repeatable". When a scientist can repeat an experiment anywhere and anytime and get the same result this helps define scientific knowledge. Repeatability and generalising appear to be linked. The same is true of prediction. A knowledge claim that withstands counter-claims over time suggests it can be the basis for prediction. Generalising, repeating and/or predicting seem inter-related in some complex manner. Using the example of the research finding that appropriately cohesive small groups are more creative than individuals or large groups (ARMSTRONG, 2000), this finding has been repeated for a range of tasks over three decades. It allows for a prediction of some confidence that if a small group of the right cohesion forms, then it will outperform alternative arrangements over time given a well defined task. [3]

Scientific research methods use simplified representations of the real world, such as samples. Pollsters for political elections are a good example. A sample of voters is a simplified representation of the population. Pollsters put a lot of effort into selecting a sample which can be used to know something about the whole population (MATT, 2001). With very careful sub-division of a sample of 1000 voters they claim to be able to accurately generalise to a population of tens of millions. Psychologists design experiments on a very small number of people, usually students, with the intent of generalising across homo sapiens. Forecasters ask a small sample of buyers their opinion on something in order to generalise across the buying population. A controlled laboratory experiment uses purified chemical samples to remove some of the confounding variables in order to simplify the real world. If this form of simplification is used to try and discover something about the universal world, then it immediately raises questions about whether the results from such simplifications are valid universally; i.e. whether they are indeed "generalisable" to the real cosmos. Complications arise because generalising often means first recombining the simplified elements back together. Doing so can create emergent properties. The classic example is that of water. Experiments on the separate (simplified) elements of water, oxygen and hydrogen reveal very explosive gases. A simple generalisation of how they would react when integrated together suggests rocket fuel. However, in the right circumstances an unexpected property emerges, water, the base of all known life forms. The same may be true of research into human behaviour as with the findings about small group creativity. When the group is placed in a complex socio-political environment with an ill defined task, unexpected emergent properties like the Abilene paradox and Groupthink start to occur. [4]

For research into complex social problems these combining issues have lead led to dissatisfaction with experimental research. The call is rather for social research to be undertaken in real settings, not to be reduced to using simplified representations. However, a problem still exists. If you study a company or an entire country going about its daily work then the issue of whether the findings are also true for other companies or countries still exists. KENNEDY (1979) suggests careful sampling of attributes within the "special case" (company or country), however this still sounds like advice to make sure your chemical sample is pure. [5]

Other problems with generalising within the scientific epistemology seem to persist. POPPER's (1963) work on falsification ended a century's long debate about whether scientific knowledge was best created using induction or deduction. Induction, such as predicting the sun will rise tomorrow because it has on numerous occasions done so in the past, was criticised because it was unclear how many observations it took to be certain. How many experiments with small groups on creativity would it take to convince you that the finding was convincing? Deduction, the searching of evidence to support a prior theory, was criticised because finding confirming evidence could be influenced by a priori biases. POPPER's solution was to call for a justified upfront conjectures (argument) for which a universal audience is encouraged to find disconfirming evidence. This would suggest in the example of the small group creativity research that the research effort would be in finding when small groups do not perform well. Generalisability sounds very similar to induction and falsification. How many observations (experiments) does it take to be certain that something is true in a range of situations? POPPER's advice almost suggests that rather than look for generalisations; the researcher should look for evidence that their findings are not generalisable. [6]

MATT (2001)] is careful to distinguish the generalisabily that has been discussed so far as "empirical generalisability"; that is, are the empirical findings generalisable? Ever since KANT's a priori argument, empirics have come to be understood as being "theory laden", from within a paradigm (CHALMERS, 1982). Researchers' prior concerns will tend to only let them see what they are looking for. They interpret what is seen in line with their paradigm or worldview. KUHN (1970) provides the example of scientists moving from a clockwork language of planetary motion to an attraction one. This must have some effect on the researcher's interpretation of the generalisability of their empirics. If their view of the world does not change then they will see the same thing everywhere. Scientific generalising would therefore appear to mean from within a paradigm. [7]

On the other hand, scientific evidence can involve reasoning evidence as well as empirics. Reasoning in science often appears in the form of mathematical proof or predicate logic. However, TOULMIN's (1964) informal logic suggests this can be extended to the other essential part of human inquiry, language and argumentation. The syllogistic reasoning, All men are mortal, Socrates is a man, Therefore Socrates is mortal, is basically universal, and therefore already generalised. This will be revisited under the argument epistemology. [8]

Scientists also talk a lot about theory. Generalising "theory" may need specific explanation. Some knowledge claims are not explanations, while some are. Theory is seen as being a generalisable explanation why a phenomenon occurs. Evolution theory is a very grand explanation why particular complex biological system designs exist. Grand because it is generalisable to all species, and has been generalised outside species development to situations like explaining antibodies responses, small business survival, community development (structuration) and our brain's neural connections. A grand explanation needs to be generic, if it is to be useful for making empirical predictions and repetitions. However, it is the explanation that is being generalised not the empirics. Using the small groups creativity example, a theory or explanation why small groups are creative is required. One theory is that it is because members can argue directly with each other. Generalisation may focus on this explanation why (theory) rather than the empirics of their creative output. Where else is this argument theory relevant? Perhaps small groups will be more equitable also because of the structure of their communication system. [9]

System thinking, as developed from LAMBERT (RESCHER, 1979) in contrast to KANT's scientific thinking, sees knowledge as bounded. Lambert seemed particularly impressed by the power of metaphors for "discovering" the shape of the heavens. The cleverness of LAMBERT's system concept is that the boundaries themselves were not defined in universal (non systems terms) terms but rather could be defined by whoever wanted to think about a particular subset of the universe. Systems are human constructs, including an airline ticketing system for travel agents, the solar system, the respiration system of a dragonfly, or a system for solving mathematical puzzles. Under a system view, generalisation is different as it is not intended that what was found to be true of a dragon fly's breathing system was also true of the system of calculus. This does not make the generalisation problem go away but it does put a different boundary on the problem. BERTALANFFY (1976) tried to address the issue of generalising across systems. However the CHURCHMAN (1971) stream of systems thinking re-introduces the idea of systems research as being a process of seeing differently. To see something (e.g. an organisation) as process or connectivity rather than as static object may be argued to be one form of "generalising". Perspectives are being generalised. [10]

Systems research can be defined as seeking new perspectives (paradigms?) on a problem; a new perspective means new knowledge if well justified to a knowledgeable audience. Kuhn uses the example of Newton's famous research on heavenly bodies for how a shift in perspective means a shift in knowledge from a COPERNICan "clockwork" view to NEWTON's "attraction" one. More down to earth examples come from MORGAN's (1986) metaphors and LINSTONE's (1999) suggestions to inquire into complex social situations using a combination of technical, organisational and personal perspectives. This approach to research challenges the scientific view of generalisability. A new perspective may be generalised onto many problem domains, or the same problem may be generalised using numerous perspectives. The example of small group creativity may be generalised in a systems sense by looking at using a communications, equity, social pressure and/or inclusion needs perspectives. [11]

However, there are numerous streams of systems thinking and each places a different interpretation on generalising. Critical systems thinking argues that social research must suggest actions to improve the lives of people, which means more than pointing out inequality, lost opportunity or inefficiencies. Knowledge claims are about what should be not how universal things are. ULRICH (2000) calls for the creation of critical heuristics (rule of thumb critique methods) to explore the boundary of things like social research findings. He advocates systems thinking, as seeking concerns (perspectives, viewpoints) and argumentation. A simple systems thinking approach to a research finding would be to ask about boundary, purpose, connectivity, interlocutors' concerns and underlying tensions. So sticking with the small group creativity example, if someone ran further experiments and found groups were more creative the more the financial rewards, the boundary questions could seek to generalise the results by asking, when will offering increased rewards still continue to work and when won't it? The purpose questions may include, what purpose do the workers give to the financial rewards? The connectivity questions may include, what else would they work hard for? The interlocutors' concerns may include how would a wider society see this experiment ethically, or how much reward gets how much creativity for how long? The underlying tension question may include, why do they need the rewards, and what are the motivational forces involved? In this epistemology the critical heuristics are being generalised, as is a concern for fellow humans. [12]

CHECKLAND (2000) claims that his LUMAS model (soft systems methodology) was specifically developed not to generate generalisable solutions. Given his assumptions of a complex ill-structured problem domain, generalisability of solutions could be more of a problem than a solution. He was concerned with consulting techniques being applied in an ad hoc manner across a number of very different "situations". He wanted rather to develop a way of setting out to get people to think about and question their situation. Ironically, the LUMAS model itself seems generalisable as a design for thinking across many different complex social change situations. [13]

The epistemology that sees knowledge as best justified through reasoned public debate or argument (thesis, conclusion) contains its own interpretation of generalisability (CROSSWHITE, 1996). The argument that: this small group is more creative, has a different degree of generalisability than the argument that: all small groups are more creative. The evidence to support either argument may be generic or specific. For example, that one small group was creative could be partially supported by the generic explanation of why groups are creative at all. Also conversely, the more generic argument that: all small groups are creative, could be partially supported by evidence of a particular group's experiences. So the outcome of argumentative inquiry might not be generalisable empirics but rather a series of well supported "if then" arguments (knowledge claims) with an explicit degree of generalisable supporting evidence. Therefore, argumentation theory (VAN EEMEREN & HOUTLOSSER, 2001) considers the argument to be as general as it claims to be and the supporting evidence to be as general or specific as the interlocutors choose as relevant. [14]

Argumentative Inquiry (CROSSWHITE 1996; METCALFE 2002; ULRICH, 2000) has a slightly different agenda, one aligned with some of the systems thinking ideas about generalising perspectives. It emphasises the usefulness of an explicit separation of the object (phenomenon) under research from the perspective being taken of that object in someone's argument. In the running example, the argument that small groups inherently are creative states that the phenomenon of small groups is being studied from the perspective of creativity, rather than efficiency or equity. Making explicit the phenomenon and the perspective explicit creates knowledge by appreciating that the argument can be generalised by studying the phenomenon from different perspectives (using different boundaries). The same can be said of what evidence is brought forward. If the argument is supported by evidence of small groups doing word games, then this reveals a perspective on creativity. This evidence would be open to counter evidence from a perspective of creativity as being about making an idea work. Generalisation may be seen as an anticipation of novel perspectives. [15]

ARGYRIS (1996) recommends that social research should result in a knowledge claim that is a heuristic, a simple decision rule, for future action. ARMSTRONG (2000) seems to call these heuristics, "principles". The heuristic should not be so specific as to have no regard for differing contexts. An example of a heuristic is that if you want promotion in a human system then seek gate-keeping roles and perform them to the satisfaction of those who can promote you. The heuristic from the small group creativity research example is that if you want a creative solution then carefully form a small group. Generalisation also means knowing the limits of a heuristic. Research aims to emerge and appreciate these generic heuristics of managerial action, not just generalise specific empirical findings. Heuristics (principles) all tend to come in the form of arguments (claims) supported by appropriate evidence. [16]

The argument epistemology is but one stream of the many operating under the umbrella of interpretive research. WINDRUM and DE JONG (2000) argue that claims about the findings from one complex social situation being true in alternative situations needs to be explicitly researched (argued). Rather than research ending with a claim that the findings are generalisable to other situations, the research that would demonstrate this needs to be outlined. In his later life, CRONBACH (1982), famous for the intra-judge reliability correlation test (Cronbach Alpha), supports this concern about abuse of the concept of generalisability. He couches these concerns in critical social theory in terms of races and social groups being branded as all having generic similarities based on a sample result. An example is HOFSTEDE's (1984) psychological test on IBM employees being used as a simple measure of culture for the country the IBM factory was based in. CRONBACH calls for an explicit identification of the situations in which a research result is and is not applicable. To do this he calls for research to be more appreciative of its context; the historical and cultural conditions or setting of those to whom research is being addressed. This sounds like JAMES' (1907/1910) pragmatic definition of truth. [17]

The interpretive researcher's assumption is typically that the social world is much too complex to generalise specific empirical findings from one situation to another. This is also what might be called the history view of generalisation. The reasons for the outbreak of the First World War are not really generalisable to anything in the strictly empirical sense as the conditions are not likely to repeat themselves. However, some general principles/heuristics/theories may be, which is why policymakers study history. For example, adopting a brinkmanship policy may spin out of control due to a small, unexpected, event. These heuristics/principles should provide some sort of guidance for future action. However, the interpretive view is also that the researcher cannot have a "God's eye" view of a situation and that research is only one interpretation of a situation. It is a complex world, thus it is not possible for any one researcher to fully appreciate when their interpretation will be useful to others. The term "transferability" of research findings, rather than generalising, is used to emphasise the limitations of the research. Perhaps it is not up to the researcher to even claim he or she knows where their findings can be generalised but for others to identify if the research can be transferred to their situation. [18]

WALSHAM (1995) focuses on an issue similar to that already touched upon when discussing theory above when he asks what can be generalised from an interpretive case study. For example, imagine you are seeking evidence to confirm or deny the argument that small groups are more creative. To do this you include a case study of creativity in ABC organisation. This organisation may have agreed to form a whole series of groups of various sizes and allow you to seek to show how this has impacted on the day-to-day creativity of operational decisions in their organisation. Your research would need to consider both "group composition" as the perspective, and on the creative decisions as the phenomenon under study. You might conclude that altering group composition had little impact on creativity in this complex environment, however you might have thought of some suggestions for improving the conceptualisation of group composition. Moreover, you may conclude that while group composition or creativity as understood in that particular organisation was not very successful, it does not mean it would not be so for all organisations. The way WALSHAM words this is to say the research may be generalisable in the sense that the perspective/principle/heuristics/concepts used could be found to be useful or that there was a richer understanding of the problem domain. This knowledge could be useful for improving future actions. That is, interpretive research can improve generalisability in the areas of: (a) development of heuristics/principles/perspectives, (b) generation of perspectives etc., (c) drawing of specific implications, and (d) contribution of rich insight, as opposed to merely wanting to generalise the empirical findings. [19]

The overlapping nature of some of these epistemologies and the reoccurrence of concepts, suggests that there may be some opportunity for them to clarify their own conceptualisations of generalisability. It is certainly not being suggested that some universal concept for generalisability developed from stirring the epistemologies together into some universal multi-method (MINGERS & GILL, 1997). This would sound like a power play by the one-world epistemology of science. Rather an ironic view would see the dialectic tension between the different epistemologies, like co-evolving but separate communities of practice, as having the potential to be creative, provided a tolerant rather than aggressive stance be taken by researchers embedded in any one epistemology. This may well improve the quality of qualitative and quantitative research. [20]

Holding that different conceptions of generalisation are capable of existing at the same time is itself an epistemology, one that has been called ironic (MUECKE, 1982). RORTY (1989) calls those who are comfortable with this view, "ironists", they accept the presence of apparently contradictory opposites as being capable of existing in reasonable harmony, at the same time, each with its own language. HATCH (1997) and other social inquirers see the existence of contradictions as a creative opportunity rather than as a problem that needs to be explained and erased. The leads to the suggestion that the four different epistemologies mentioned in this paper may want to consider the language of the other epistemologies to re-view, see afresh, their own set of priorities for generalisation, not to align anything but rather to improve their own conceptualisation. [21]

Figure 1: "Generalisation Over Four Epistemologies" is an attempt to visually connect some of the concepts that may be used from one epistemology to clarify its own concepts of generalisation. So, link NO.1 in the figure suggests for example that the science view may use the systems concept of boundary to ask not only whether a knowledge claim is falsifiable but also when and where it is falsifiable. The laws of physics may not apply in black holes, different "big bangs" may have resulted in different physical constants. No.2 in the figure suggests the science concept of "ruthful knowledge claims" may make the interpretivist more sceptical of some interpretations and so think more about how they are to be distinguished from relativism. No 3. reflects ULRICH (1983) work that uses concepts from argument theory to critique the concept of boundary in systems thinking so as to clarify the boundary of arguments. No 4 reflects RORTY's (1989) point that science already uses logical argument to persuade that certain knowledge claims are generalisable. RORTY contrasts logical argument with investigative or creative argument that he labels "dialectic argument". Science may think more about how to use this creative aspect of argument to learn from their own assumptions about generalisation. The systems epistemologies priority of metaphors for re-visualisation is reflected in No.5. Re-visualisation may also provide learning to argumentative inquiry, interpretivism and science.

In this paper, it has been argued that (a) it is useful to explicit identify the priorities for generalisation of differing epistemologies, and (b) the ironic view is that each epistemology might improve its quality by learning from the others how to better conceptualise their on understanding of the generalisation of knowledge claims. This is not to suggest that each needs to change or align, but rather research quality might be improved if researchers consider clarifying and extending their thinking on the generalisation of their findings. Systems thinking, interpretive-critical research and argumentative inquiry have different priorities, as against traditional scientific (objectivist) research. Systems thinking bounds it findings to the system under consideration, and places considerable emphasis on revisualisation, re-seeing, as providing a new generality. Interpretive researchers distinguish generalising the interpretive frame (lens, perspective, principles) from the empirical findings from one context, while critical social theory requires that any generalisations empower the weak. Argumentative inquiry, takes yet a different approach by explicitly addressing the issue of generalisation in the knowledge claim. [23]

Generalising was presented as central to the definition and creation of valid public knowledge. This paper adopts the ironic view of believing differing epistemologies not only can, and should, co-exist. This suggests researchers use the different priorities over generalisation to think creatively about their own research design. For example, the generalisation of empirical results needs to be distinguished from the generalisation of other aspects of research including theory, perspective and heuristics. It is hoped that a creative dialectic between each epistemology can be tolerated so as to improve the quality of everyone's definition of generalisation and thus of knowledge. [24]

Argyris, Chris, (1996). Actionable Knowledge. Journal of Applied Behavioural Science, 32(4), 390-406.

Armstrong, Scott J. (Ed.) (2000). Principles of Forecasting. Norwell, MA: Kluwer.

Bertalanffy, Ludwig von (1976). General System Theory. New York: George Braziller Publisher.

Chalmers, Alan F. (1982). What is this Thing Called Science? Brisbane: University of Queensland Press.

Checkland, Peter (2000). Soft Systems Methodology: A Thirty Year Retrospective. Systems Research and Behavioural Science, 17(1), 11-58.

Crosswhite, James (1996). The Rhetoric of Reason. Madison: University of Winsconsin Press.

Habermas, Jürgen (1984). The Theory of Communicative Action: Reason and the Rationalisation of Society, Vol.1. Boston: Beacon Press.

Hatch, Mary Jo (1997). Irony and the Social Construction of Contradiction. Organization Science, 8(3), 275-288.

Kennedy, Mary M. (1979). Generalizing From Single Case Studies. Evaluation Quarterly, 3(4), 661-678.

Klein, Heinz. K. & Myers, Michael D. (1999). A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems. MIS quarterly, 23(1), 67-94.

Kuhn, Thomas S. (1970). The Structure of Scientific Revolutions. Chicago: University of Chicago Press.

Linstone, Harold A. (1999). Decision Making for Technology Executives: Using Multiple Perspectives. Boston: Artech House.

Metcalfe, Mike (2002). Argumentative Systems For Systems Design. IT and People, 15(1), 60-72.

Mingers, John & Gill, Anthony (Eds.) (1997). Multimethodology: Theory and Practice of Combining Management Science Methodologies. Chichester: Wiley.

Popper, Karl R. (1963). Conjectures and Refutations: The Growth of Scientific Knowledge. London: Routledge & K. Paul.

Rescher, Nicholas (1979). Cognitive Systematization: A Systems-Theoretic Approach to a Coherentist Theory of Knowledge. Oxford: Basil Blackwell.

Rorty, Robert (1989). Contingency, Irony and Solidarity. Cambridge: Cambridge University Press.

Toulmin Stephen (1964). The Uses of Argument. Cambridge: Cambridge University Press.

Ulrich, Werner (1983). Critical Heuristics of Social Planning: A New Approach to Practical Philosophy. Chichester: Wiley.

Ulrich, Werner (2000). Some Guidelines for Boundary Critique. Reflective Practice, 1(2), 247-268.

Van Eemeren, Fran. H. & Houtlosser, Peter (2001). Managing Disagreement: Rhetorical Analysis Within a Dialectical Framework. Argumentation and Advocacy, 37(Winter), 150-157.

Walsham, Graham (1995). Interpretive Case Studies in IS Research. European Journal of Information Systems, 4, 74-81.

Windrum, Paul & de Jong, Simone (2000). Foundations of Evaluation Theory: A Review. The SEIN-Project, Project Paper No. 9, Maastricht.

Mike METCALFE is an associate research professor at the University of South Australia, where his main duties are as PhD thesis adviser. His own PhD is from Adelaide University, on group problem solving. He has published five books and over 50 lead-author refereed academic articles on different aspects of perspectival systems thinking and argumentative inquiry in journals that include Systems Research and Behavioral Science, IT & People, FQS and the European Journal of Information Systems. He has lived in over six countries including 17 years in Australasia, and has worked in the merchant navy, the army reserves, system design, university lecturing and consulting, including a term as adviser to the Deputy Premier and Treasurer of South Australia.

School Of Management, University of South Australia (City West)
Adelaide, South Australia, 5000

Metcalfe, Mike (2004). Generalisation: Learning Across Epistemologies [24 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(1), Art. 17, http://nbn-resolving.de/urn:nbn:de:0114-fqs0501175.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License