Abstract: In this article, I examine the possibility of using TURING's concept of "imitation games" to analyze political discourse. This poses the theoretical question of identity matching. It also poses a methodological question: Is it possible to distinguish, using only internal criteria, the political discourse of political actors that belong to two distinct categories? The effort to answer these theoretical and methodological questions highlights important common motives in quantitative and qualitative research.

Key words: Turing Test, political discourse, classification systems, thought experiment

In this article, I want to discuss the possibility of using a criterion—the Turing Test (TT)—for the analysis of political discourse as an "imitation game". I will start presenting the case in favor of the use of TT in social science and qualitative studies. Then I will try to show the advantages of TT-type experiments in the specific domain of political discourse. Finally, drawing on the previous discussion, I will highlight some basic links with quantitative and formal studies. [1]

Fifty years ago, in a seminal article, Alan TURING (1950) crafted an "imitation game" that proposed a criterion to know if a machine could "think" (also FRENCH 1996; HARNAD 1992; SHIEBER 1994). In the game, an interrogator would speak through a screen with two completely different respondents, a human and a machine, and through successive questioning try to identify who is who. The interrogator would not have any physical cue (would not see the embodiment nor hear the voice) of his interlocutors. Though the game is apparently played by three subjects, actually it resembles a typical 2-player setting, each hit being equivalent to a victory for the interrogator while a miss is tantamount to a victory for the machine. If, after a session of questions, the interrogator can't tell the human from the machine, i.e. if the machine wins, it is impossible to deny, according to TURING, that it can think. In his mental experiment, TURING allowed the interrogator to dip into any subject whatsoever, but imposed upon him clear time limitations (he had no more than five minutes for his endeavor). [2]

TURING's article had a telluric effect over the young artificial intelligence research community, and soon it also became a matter of debate among philosophers concerned with the issues of thought, language and meaning. In 1980, SEARLE came up with his "Chinese room" thought experiment, in which he took TURING to task as "unabashedly behavioristic" (SEARLE 1980; 1982). The moral of SEARLE's own little story is that a being can produce an adequate output in a conversation, without understanding a word of it. Correct symbol manipulation, SEARLE argued, is not necessarily a symptom of the existence of subjective states that depend on meaning and, so, on semantics. Although I think the importance of the Chinese Room argument has been exaggerated, it does stress adequately the irreducibility of meaning to syntax (a good discussion of the Chinese room argument can be found in HARNAD 1989; an apt but extremely unsympathetic criticism is HAUSER 1997). [3]

But a major point that might be behind TURING's mise en scene, and that remains untouched by SEARLE's critical remarks, is that classificatory systems are a main arena of interaction between mind, language and social action. In fact, typologies help to create the social world (LAKOFF & JOHNSON 1980; LAKOFF 1987; SOLOMON, MEDIN & LYNCH 1999). Typologies are anchored in our bodily and socio-cultural experience, and so a sufficient dialogue should allow us to unmask a speaker (in the sense of: mapping him or her into some basic typology). [4]

What I mean by sufficient is simply: as long and variegated as to color discourse with the basic characteristics (i.e., gender: TANNEN 1996) that spread over the totality of vital experience²⁾. Discourse is so marked by cognitive and subcognitive cues (FRENCH 1996) that there should be an inverse function³⁾ that returns us to pigeonholes of a classificatory system from strings of utterances. If this is true, Turing's Test is only a particular case of the question: How to distinguish, according to a previously set and extremely simple (dualistic) typology, player A from player B using only their language games? [5]

Dualism can be linked to our cognitive activity in at least two manners. First, a predicate in a pretty standard way defines a set S, the objects that possess the predicate, so any predicate is a potential base for a dualistic characterization. If my universe are the tables, S might be the set of red tables, and its complement the tables that aren't red. This is the classical notion of categories, which is only a particular case of a variegated toolkit of category building (see for example LAKOFF 1987). Second, in language proper the distinction between being A/not being A may be a basic operator. JAKOBSONian dualism gives origin to the very important concept of "marked" and "unmarked".

The concept of marking comes from the idea that the marked term of an opposition is more complex than the unmarked term ... In grammar, marking involves contrasting values in meaning. The marked term is "characterized by the conveyance of more precise, specific and additional information than the unmarked term" (HAGE 1999, p.425). [6]

1. The presence of the marked term implies the presence of the unmarked term, but not necessarily conversely ... 2. the marked term in lexicon is overtly indicated, e.g. parent (unmarked) versus grandparent (marked) ... 3. Par excellence expression. The unmarked term may represent the entire category or the opposite of the marked term ... 4. Certain categories present in the unmarked are absent in the marked (HAGE 1999, p.426).⁴⁾ [7]

For example, in the TT, human is clearly unmarked while alien-non human is marked. [8]

Dichotomies built upon marking are not necessarily classic, in the sense of crisp and well differentiated. But, as JAKOBSON's concept of markedness suggests, they play a very important role in categorization. Sometimes, when finer distinctions are called for, dichotomies can be viewed as high level operators that subsume them. As other high level operants, they are central tools in inference and identity matching (BARNES-HOLMES & BARNES-HOLMES 2000), the core notion behind the TT. Now, it might happen that the proposed typology serves neither purpose adequately, because it outputs too many ambiguous answers. In this sense, the TT is a criterion to evaluate contestants but also to evaluate the typology itself: how much input does it need to produce a characterization that will be correct and stable under any further supply of information? It may provide a benchmark for building categories of categories according to their performance. [9]

My claim is that, thus, the TT can be fruitfully applied to qualitative research. But doesn't this mean incurring in gross behavioristic misunderstandings, or simply using a mechanical analogy from a completely different domain of knowledge? I believe not. First of all, the Turing Test was born as an effort of human classification: in the game from which TURING got his idea the interrogator would probe a male and a female and guess which is which. The male tried to fool the interrogator into believing that he was also a woman. [10]

So the Turing Test is an analogy taken from human affairs to artificial intelligence, not in the other direction. This is much more than an anecdote. Presently, when the Internet provides the interactive screen TURING perhaps had in mind for his test (he himself spoke about a teletype), Turing-like experiments and questions are being actively developed. For example, at http://www.cc.gatech.edu/elc/turing/ (which I'll call T page from now on) you can play the TT with male-female and other human dichotomies, and of course also with the canonical machine-human one (CARLSON 1999). The notion behind these engrossing exercises is that discourse is marked vis-à-vis some basic dualistic typologies, so even a sophisticated mimicking will be, sooner rather than latter, unmasked. At the T page, in the man-woman game the question: how much toilet paper do you waste monthly? was raised. Women answered correctly "one or two", but men offered wildly hyperbolic answers (10, 15 toilet paper rolls). This gave the women a cue to classify adequately their interlocutors. Men, instead, thought that the 10-15 hunches came from women, who they imagined to be "those fancy wasters", so they got it backwards (CARLSON 1999).⁵⁾ [11]

Second, qualitative researchers get involved in these classificatory practices anyway. For example, a study about the fascist political discourse (SCHMITT 2000) found that it used a specific set of metaphors. Specific in relation with whom? Here we have a nearly perfect set theoretical predicate, which divides the universe of political parties and movements between Nazis and non-Nazis, the former being the marked pole of the dichotomy. [12]

I believe there would be a near-consensus among qualitative researchers in the sense that decisive experiential traits tinge discourse, and the Turing Test is offering us a tool to evaluate and discuss systematically such conjecture. This tool is not behaviorist in the sense that SEARLE used the concept against TURING. For SEARLE, linguistic behaviorism consisted in judging a being by its observed output (perhaps devoid of meaning). But in the {public} human domain, we know that discourse is meaningful, so from the very beginning the test accounts for much more than "only symbols". Furthermore, classificatory testing of discourse is also more than "only words", in the sense that it systematically searches for cues in the "speakers-tokens" at both levels, cognitive (what they know) and subcognitive (how are they placed in the world, what are their implicit skills; FRENCH 1996). [13]

Triangulation between hermeneutics and more formal methods can be used with advantage. If hermeneutic interpretation is an art more than a method (MAYRING 2000), its results can be profitably compared with those that come from the use of more explicit patterns of evaluation. Furthermore, this might help prevent indulging in overinterpretation, an easy and very common pitfall of which I would bring out two particular cases:

The fallacy of expansion is related to a poor notion of causality, and the fallacy of restriction to a poor notion of discrimination. Of course, there is a generous supply of extreme instantiations of overinterpretation, some of which would do well as textbook examples of where uncontrolled hermeneutics can lead. The problem with overinterpretation is that it opens the door (or the door was opened just before it?) to circular reasoning. [15]

An additional and seldom spoken of problem is overfitting. Suppose you know that party A is racist, and you are studying its political discourse. The result will probably be to show how A's discourse, even when it is related to areas far from the subject, promotes its racist practices and values (unless you are an A militant, but I will not reckon with that case). It may happen, however, that there are serious discontinuities and maladjustments between A's practice and discourse, and these should also be uncovered. Actually, they play a very important role in the outcomes of political conflict. Isn't it possible that in some interpretative efforts there is an internalized functionalism, which highlights only the mutual reinforcement of practices and discourse while losing their discontinuities and tensions? [16]

If so, discriminatory (semi) formal tests⁷⁾, do not "unground" social theory—in the sense of separating discourse and other social practices—but rather provide for a "veil of ignorance" to prevent circularity, overinterpretation and overfitting. New, possibly important, phenomena-differential ways of advancing goals discursively and in social action, polysemy of discursive devices, etc.—can be brought to the fore. And the quality of typologies—how many false positives, false negatives and "don't knows" they output—might be systematically assessed. [17]

All this seems pretty crucial in domains in which imitation is an important issue. Take political discourse. It is widely-understood that in today's politics, dictators try to appear as democrats, and mass killers as humanitarian reformers. Why is this effort possible, imaginable? Why should it be so difficult to tell a dangerous and homicidal yahoo from a virtuous houyhnhnm? [18]

Mass politics implies argumentation and deliberation in front of distant audiences that must take cues from fragments of information to form opinions and take decisions (i.e., vote for A or for B). Discrimination is at the very heart of it. That's why the political in the 20th century can be read also as the uninterrupted weaving and unweaving of imitatory practices. On the side of contentious politics, perhaps the most serious issue was to know who was really a defender of the working class and popular interests. On the side of liberal politics, who is or is not a true democrat may have been an equivalently important question. The future, with the formation of a world-wide public opinion that discusses and deliberates about remote events "disembodiedly" through an interactive screen in a very turinguian fashion, will only make the problem more salient and difficult. My contention is that the TT appears here as a valuable benchmark because it is an abstract distillation of quotidian processes embedded in the very (contemporary) notion of the political. To think politically is to engage not only in strategic games, but also in imitation games. [19]

On the one hand, the present institutionalization of politics demands that politicians and parties speak to heterogeneous constituencies addressing their often contradictory interests. On the other hand, public discourse, even in its most official and formal varieties, is heavily marked vis-a-vis many fundamental categories. Politicians and bureaucrats speak to diverse constituencies and, to gather their support, must refer if only in a tenuous and indirect manner to past and present experiences, interests, alliances and expectations. In this sense their statements are always a blend of candor and mimesis. For any association that is more than marginal, to mobilize and to dissimulate are necessary but partially contradictory efforts. The political, as common sense correctly guessed a long time ago, is a domain replete with imitation games and so an ideal arena for TTs. [20]

Imagine a researcher wants to apply a TT to distinguish party A from party B. The researcher has at hand a corpora of texts and a knowledge about the deeds of A and B. She doesn't have direct access to the minds of the participants⁸⁾. If she had, she could raise the "is the deliverer sincere or not?" issue. But she doesn't. Even in the cases of the (for me or you) most outrageous contradiction between fact and words, some kind of discursive mediation is necessary to expose it. It is typical of the political that the contradictions between the facts and the words can be bridged by new words. To try to solve the identity matching problem only through a comparison of strings of actions and signs is a doomed effort, internal and not external to direct unmediated political practice itself. Less than ever, facts here do not speak for themselves. [21]

Our researcher would still have plenty of cues, however. She can try to distinguish between A's texts and B's, for example by their styles. Perhaps A uses idiosyncratic metaphors, so even if A tries mimesis it can be uncovered. But humans learn to learn. Style can be adopted, and in a market economy even purchased (for example, by buying sophisticated intellectual output). So the analysis of imitation becomes increasingly complicated. How can this problem be addressed? I'll mention here two dimensions of analysis that I deem important. First, the intention should leave a trace. The problem with political discourses is that they are much more than words (and in this common sense goes wrong). Imitation can express ambiguous intentions (I want to imitate B for purely instrumental reasons, but at the same time I really aspire to resemble B), or have unintended consequences (I start imitating B, and end up being a little bit like it). The PASCALian advice has more than a grain of truth to it: mutter your prayers, and perhaps you will start to believe. Gesture may precede belief. Or is it that purely instrumental imitation has internal signals that highlight the "non-PASCALian" parts of the speech? (the parts not to be taken seriously, the parts you can repeat an indeterminate number of times without changing the way you act). Second, overcorrectness (once more a stylistic issue, but far more subtle than making open errors). Overcorrectness can appear because there are states that are "subproducts of themselves" (ELSTER 1989; ELSTER 1983); they can't be reached by a direct quest. Overcorrectness can be identified by (stylistic) superarticulateness. If the imitator tries to conceal his superarticulateness, intentions of doing so will also leave a trace⁹⁾. [22]

So the TT would go on in the following way. In the first step, the researcher chooses a corpora of texts of A and B, for example those produced around a common theme in a public debate in a relatively long span of time. In the second, she develops some working hypothesis about the differences between A and B (as we have a working hypothesis about the difference between human and machine, say, that the latter has no bodily experience). In the third, she suspends belief about her contextual knowledge and, covered with this version of the veil of ignorance, probes each text—for example, from the point of view of style, imitatory devices and overcorrectness—to ascribe it to A or B: text 1 was uttered by A, text 2 by B. In the fourth step, she compares the result of the third step with the actual authorship. In the fifth, she engages in identity matching. Finally, she looks if necessary for new ways of categorization. [23]

TT's of this type would allow for a structured confrontation of contexts and discourses. Suppose our researcher believes A and B belong to two distinct, dramatically different categories. She has a lot of historical context to back her statement. Now she can choose a coherent corpora of texts produced by A and B and subject it to systematic analysis from the point of view of style, imitative devices and overcorrectness. She may find that she has a brand new category, with some texts of A and B on one side of her new dichotomy, and other texts of A and B on the other side. Or she might simply put into question her patterns of reading the historical context by discovering that A and B have long engaged in cross-fertilization, each one imitating a different aspect of its adversary's discourse. The most obvious forms of circular reasoning are thus avoided. Furthermore, this type of exercise may show us how categories are socially built and how diverse political classificatory systems coexist, a very important problem in its own right (KAGAN 1998; LIEN & CHENG 2000; SMITH & MARK 1999; WORTHEN, HUTCHENS & BRET 1998). It may also help evaluate the validity of the initial conjecture that A and B were qualitatively different: how many good matches does this conjecture produce? [24]

If the TT has some use in the analysis of political discourse, this provides more evidence in favor of the notion that there is not a Chinese Wall between qualitative and quantitative social science, and that this particular dichotomy shouldn't be pressed too far. [25]

At least three arguments appear to be relevant in the perspective of relaxing the qualitative-quantitative cleavage. First, frequently both poles are confronting the same type of methodological problem, and none can provide for a "perfect" solution. One of the main issues addressed in this article, discriminating, is indeed central to the statistical literature, and each possible solution has to be qualified with several technical niceties. Overfitting is also a known nuisance not only for statisticians but for researchers that use neural networks. I don't believe it is desirable that one type of analyst imitate the other—people who work with texts and contexts follow the statistician—because they are probing different aspects and moments of social life. Even worse would be to imitate or reject a caricature of the other (statistical technique as provider of some absolute positivistic truth independent of theory and values). Instead, comparing methodically different ways of attacking the same problem might offer interesting possibilities of cross-fertilization. [26]

Second, common styles of thought pop out here and there and sometimes run across wide scientific and intellectual fields. Mind experiments make possible theoretical physics, illuminate artificial intelligence (actually, founded it) and, as LEVI-STRAUSS showed in his comments about ROUSSEAU, also played a foundational role in anthropology and other social sciences. [27]

Third, and perhaps most importantly, from time to time a theoretical earthquake provides unifying concepts and sets of preferred questions to disparate intellectual endeavors. Then, what is qualitative and what is quantitative, what is soft and what is hard, is neither possible nor interesting to tell apart: a good example of a distinction which does not pertain. Those concepts go back and forth between one discipline and another, changing and being changed in the process. During his voyages in the Beagle DARWIN read and was inspired by MALTHUS, and a few years later economists were reading DARWIN to determine what biology had to teach economy. Perhaps deplorable, in any case ironical, but there are many other more encouraging examples. To have a deep understanding of semiology and linguistics one had better grasp some basic concepts of geometry, as JAKOBSON once stressed.

Peirce belonged to the great generation that broadly developed one of the most salient concepts and terms for geometry, physics, linguistics, psychology and many other sciences. This is the seminal idea of INVARIANCE. The rational necessity of discovering the invariant behind the numerous variables, the question of the assignment of all these variants to relational constants unaffected by transformations ... Invariance was the main topic of Felix Klein's Erlanger Program in 1872 ... Thus, convergent ideas destined to transform our science, and sciences in general, emerged almost simultaneously. No matter where the model came from, those were timely pursuits for a wide field of research and they are still able to engender new, fruitful interactions between different disciplines (JAKOBSON 1977, p.1030). [28]

One of the contemporary "timely pursuits for a wide field of research" is the quest for human subjectivity and conscience. It is at the base of the artificial intelligence effort, that not by chance gathers social scientists, mathematicians, engineers and philosophers. It is the objective of at least some fundamental rational choice subfields, for example the area of common knowledge, and their many relations with abstract logic. Doesn't it define in some sense hermeneutics? Not by chance historical narrative and strategic calculus are being brought together (BATES 1998). I believe one of the seminal ideas behind this quest is the concept of reflexivity. Imitation is a typically reflexive action, no wonder it has inspired a powerful literature in artificial intelligence, philosophy, semiotics, linguistics, rational choice (FRANK 1996), anthropology and political science. [29]

1) I want to thank the editors of this number for their valuable critical remarks. <back>

2) This reminds me of a joke raised against TURING by JEFFERSON in a BBC 1951 radio series: "that he would not believe a computing machine could think until he saw it touch the leg of a lady computing machine" (HODGES 1983, p.452). The lack of gender is, of course, a very gendered experience. <back>

3) A perhaps idle technical finesse: to speak about functions and inverses, I am supposing the domain and the range are equivalence classes (males and females, on the one hand, and male and female discourse, on the other). To say that there is an inverse arrow is equivalent to assert that they "behave well" , that is, that an imitation game under TURING's conditions (possibly dropping the time limitations) would discriminate properly between males and females. <back>

4) Once again, gender is a good example for the man-machine imitation game. <back>

6) Many Latin American qualitative studies (based for example in the notion of social capital) are based in such kinds of reasoning, and end in a very biased and ideological "inverted ethnocentrism" (KNIPPERS BLACK 1998). <back>

7) Of course, there are several methods and techniques that have been used in a highly profitable way. I believe that the TT would have some specific advantages: it would be able to address very explicitly the problem of discrimination and imitation. <back>

8) The "other minds" problem is a thorny one but it is at the heart of the TT. How to know about some thing other subjectivity only with indirect evidence? Of course, it addresses an old philosophical problem, but in more or less contemporary terms. See ECO 1999. <back>

9) "Part of my impeccable performance is trying to be as imperfect as everybody". But this is the most extreme form of overcorrection. So you can be caught after all. <back>

Barnes-Holmes, Dermot & Barnes-Holmes, Yvonne (2000). Explaining complex behavior: Two perspectives on the concept of generalized operant classes. Psychological Record, 50(2), 251-265.

Bates, Robert (Ed.) (1998). Analytic narratives. Princeton, NJ: Princeton University Press.

Brass, Paul (1997). Theft of an idol. Text and context in the representation of collective violence. Princeton, NJ: Princeton University Press.

Carlson, Scott (1999). An on-line "quiz show" uses the anonymity of the internet to reveal biases. Chronicle of Higher Education, 46(6), A36-A37.

Elster, Jon (1983). Sour grapes: studies in the subversion of rationality. Cambridge, NY: Cambridge University Press.

Elster, Jon (1989). Salomonic judgements: studies in the limits of rationality. Cambridge, NY: Cambridge University Press.

Frank, Robert (1996). The political economy of preference falsification. Timor Kuran's private truths, public lies. Journal of Economic Literature, 34(1), 115-123.

French, Robert (1996a). Subcognition and the limits of the Turing Test. In Peter Millican & Andy Clark (Eds.), Machines and thought (pp.11-26). Clarendon Press: Oxford.

Hage, Per (1999). Marking universals and the structure and evolution of kinship terminologies. Evidence from Salish. Journal of the Royal Anthropological Institute, 5(3), 423-441.

Harnad, Stevan (1989). Minds, machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence, 1, 5-25.

Harnad, Stevan (1992). The Turing Test is not a trick: Turing indistinguishability is a scientific criterion. SIGART Bulletin, 3(4), 7-9.

Hauser, Larry (1997). Searle's Chinese Box: debunking the Chinese Room argument. Minds and Machines, 7, 199-226.

Hodges, Andrew (1983). Alan Turing: The enigma of intelligence. New York: Touch-stone/Simon and Schuster.

Jakobson, Roman (1977). A few remarks on Peirce, pathfinder in the sciences of language. Comparative Literature, 92(5), 1026-1032.

Kagan, Jerome (1998). The quiet return of categories. Social Research, 65(2), 449-461.

Knippers Black, Jan (Ed.) (1998). Latin America, its problem and its promise. A multidisciplinary introduction. Boulder: Westview Press.

Lakoff, George (1987). Women, fire and dangerous things. Chicago: Chicago University Press.

Lakoff, George & Johnson, Mark (1980). Metaphors we live by. Chicago and London: University of Chicago Press.

Lien, Yunwen & Chen, Patricia (2000). Distinguishing genuine from spurious cases: a coherence hypothesis. Cognitive Psychology, 40(2), 87-137.

Searle, John (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3, 417-424.

Searle, John (1982). The Chinese Room revisited. Behavioral and Brain Sciences, 5, 345-348.

Shieber, Stuart (1994). Lessons from a restricted Turing Test. Communications of the ACM, 37(6), 70-78.

Smith, Barry & David, Mark (1999). Ontology with human subjects testing: An empirical investigation on geographic categories. American Journal of Economics and Sociology, 58(2), 245-272.

Solomon, Karen; Medin, Douglass & Lynch, Elizabeth (1999). Concepts do more than categorize. Trends in Cognitive Sciences, 3, 99-105.

Stevens, Tyler (1996). "Sinister fruitness": Neuromancer, internet sexuality and the Turing Test. Studies in the Novel, 28(3), 414-433.

Worthen, James; Hutchens, Scott R. & Bret Gutiérrez, Guadalupe (1998). The internal structure of social categories. Components and predictors. American Journal of Psychology, 111(4), 581-605.

Francisco GUTIERREZ, presently researcher at the Instituto de Estudios Politicos y Relaciones Internacionales of the Universidad Nacional de Colombia

Main research fields: Violence, Political discourse, Political system, Comparative politics, Research methods.

Gutierrez Sanin, Francisco (2001). Imitation Games and Political Discourse [29 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [On-line Journal], 2(1), Art. 10, http://nbn-resolving.de/urn:nbn:de:0114-fqs0101103.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License