Strong Interdisciplinarity and Explanatory Pluralism in Social Scientific Research
Dimitris Xygalatas engages the problems of the generalizability and comparability of research results and their “ecological validity.” Xygalatas argues for the “methodological interaction between forms of participant-observation and experimentation,” combining the insights of approaches often seen as at odds with each other, to produce a collaborative and strong version of interdisciplinary research. Drawing from his own work on extreme religious rituals such as fire-walking and body piercing, the author demonstrates the benefits of research designs that include perspectives from the “field” and the “lab.”
Over the last decade, various academic movements have begun to question current social scientific research practices—from the use of unrepresentative samples in psychological studies to the failure to replicate results across studies—and to make appeals for methodological reform. As a result of these realizations, there are now various coordinated efforts to address these problems through expanding subject pools and systematic replication projects. However, I worry that these efforts are often misguided, because they do not address the underlying problem.
At its core, the problem is one of ecological validity. It pertains to the relationship between the local and the global, between specifics and universals, and the ability to draw comparisons and generalizations from limited amounts of data. Unfortunately, however, this relationship rarely comes to the fore in social scientific work.
The problem of ecological validity
I recently attended one of the largest annual gatherings of social psychologists, where a plenary panel was specifically dedicated to problematizing current research practices and discussing the way forward. For about two hours, a group of prominent figures in the field brainstormed how to address the lack of generalizability beyond the narrow boundaries of Western educated urbanites—in other words, how to increase relevance and ecological validity. I expected words like culture, context, fieldwork, ethnography, and interdisciplinarity to come up early in the discussion. But much to my surprise, at no point were these words—or concepts—mentioned. The proposed solutions exclusively focused on collaboration between laboratories and various technological and social innovations that can facilitate larger and more diverse samples, involving scholars from around the world sharing or exchanging research protocols to be implemented in different contexts.
This approach is tempting, as it produces large amounts of data with relatively little effort. But you often get what you pay for. Replicating the same protocol in different contexts means that any design errors will also be replicated. When that happens, the problem is not that you won’t get data; it is that you won’t really know what those data mean.1
Such problems are inherent to the very structure of most academic institutions. Whether by necessity, politics, or inertia, academia is radically compartmentalized, and, as a result, our knowledge of human nature is fragmentary. Each of the various disciplines dedicated to studying this nature tends to focus on either the local or the global, with little (if any) interaction between them. For example, cultural anthropologists are primarily interested in describing diversity, while cognitive psychologists are more mostly interested in describing human universals.
Much of this discrepancy is due to philosophical or ideological differences: some disciplines emphasize the role of social forces in shaping our personality, thought, and behavior, while others see the main causal links as flowing the other way. But these presuppositions themselves are to some extent byproducts of the methodological tools they each have at their disposal. Those trained to work in the field typically use qualitative techniques such as participant-observation and unstructured interviews to collect mile-deep, inch-wide data from a handful of informants. Most of them never receive any training in statistics, and often intentionally refrain from using quantitative data to support their claims. On the other hand, those trained to work in the lab typically use controlled experiments to collect inch-deep, mile-wide quantitative data from larger samples without ever having any meaningful interaction with the people they study.
Those two modes of studying human nature result in radically different types of outputs. Field studies can be high in relevance and ecological validity while lacking control and precision. On the contrary, lab studies can have a high degree of control and precision while lacking relevance and ecological validity. There are also, of course, mixed modes, such as field experiments. Instead of taking participants out of context by moving them to the sterilized environment of a lab, the researcher moves the lab into context, by incorporating experimental methods in real-life settings.2 Critically, field experiments (when done well) require deep, contextual knowledge that goes beyond the “ethnographic setting” section of the paper. In this type of research, preliminary work, such as participant observation and detailed interviewing, is crucial, as it informs the design from the bottom-up.3
There are several advantages to this mixed-method approach. For one thing, some phenomena simply cannot be studied in the lab (much of my own research, focusing on painful collective rituals, falls under that category). But alas, there are always trade-offs when moving along the lab/field axis. Indeed, most quantitative field studies are not true experiments, but rather pseudo-experiments—random assignment is often not possible in the field. Inevitably, then, this approach consists in satisficing between as much “naturalness” as can be maintained (depending on how intrusive one’s methods are) and as much control as can be had in a natural setting.
Moving toward a strong interdisciplinarity
Of course, it would be nice if we could have our cake and eat it too, wouldn’t it? Interdisciplinarity offers a way out of this conundrum. I don’t mean a weak version of interdisciplinarity, where an individual researcher reaches out to other fields for theoretical and methodological inspiration. I mean a strong version that involves teamwork, collaboration, and continuous interaction between and within groups of specialists rather than uncoordinated efforts by isolated generalists.4
This deep interdisciplinarity essentially requires a shift in the way we understand the research process. A single study, using a single method, can only get us so far by providing a necessarily narrow and limited perspective. But ultimately, we need a cumulative body of convergent knowledge, produced by tackling a problem from multiple angles. This way, we cover more of the puzzle, one piece at a time. And to do that, we need to move back and forth, in coordinated and systematic ways, between the dominant paradigms—between the messy but real place that is the field and the artificial but precise space of the laboratory.
An example of this kind of synergy is what philosopher Shaun Gallagher calls “front-loading phenomenology,” whereby phenomenological insights can be incorporated into experimental design as testable hypotheses derived from the practitioners themselves.5This presupposes a close relation between emic (insider) and etic (outsider) perspectives, as well as a methodological interaction between forms of participant-observation and experimentation. Although this approach cannot substitute laboratory experiments or ethnographic observation, it can in certain cases answer questions that neither one approach can answer alone. Importantly, this process can also often generate new relevant questions and help highlight the limitations of each method and the need for a continuous dialectical relationship between the field and the laboratory.
An interdisciplinary study of painful rituals
I started applying this model of deep interdisciplinarity in my own research when I realized that neither ethnography nor experimentation alone could help me answer the questions I was asking—only the synergy between the two could. Much of my work has focused on the effects of high-intensity rituals. As a doctoral student, I spent two years in the field studying such rituals in Southern Europe, asking hundreds of people why they participated in painful and stressful activities such as fire-walking. Although most of them could provide no specific answer other than that it was their tradition, longer discussions revealed plenty of phenomenological insights. My informants commonly described a sense of identity and belonging produced by participation, and a powerful feeling of oneness with the group. This sounded similar to earlier anthropological observations, such as those of Emile Durkheim, who argued that through coordinated emotional exaltation, collective rituals result in group bonding by generating a feeling of togetherness that he called “collective effervescence.”
To measure this purported emotional alignment, I collaborated with an interdisciplinary team of anthropologists, cognitive scientists, biostatisticians, and religious studies scholars to design a study that could be run in one of my field sites: a small Spanish village where a fire-walking ritual is held annually. We used biometric monitors to quantify emotional alignment, and obtained sociometric measures to quantify social proximity. We found that the ritual indeed created emotional alignment: people’s heart rate patterns exhibited a high degree of synchrony. In fact, this was not only limited to the fire-walkers themselves, but extended to the entire community (even those who were merely watching), but not to outsiders. And when we looked at the village’s social network, we found that emotional alignment got stronger as social proximity increased.6 This echoed Durkheim’s own views, which argue that these rituals do not merely generate emotions but bring those who share them in a deeper and more dynamic relationship.
Looking beyond how collective rituals serve to reinforce social bonds, it would be crucial to document their behavioral effects. To do that, I initiated another two-year project, aimed at studying some of the painful rituals performed by Tamil Hindus in Mauritius. At the end of that project, I put together another interdisciplinary team to examine the social effects of the kavadi ritual, where devotees of Murugan, the Hindu god of war, have their bodies pierced by needles, hooks, and skewers, and suffer many other hardships. Our aim was to get a measurable indicator of prosocial behavior. While various previous studies had done this using economic games, those games often involved unusual interactions removed from everyday experience. To avoid this, we used a task that was familiar to everyone: we measured people’s contributions to charity after participating in different types of events. We found that the more pain people experienced, the more money they donated to charity, and that overall more painful rituals led to higher donations not only for performers, but also for spectators.7
But what are the mechanisms underlying these effects? Going back to the field, and turning back again to phenomenology, we examined another painful ritual and found that those participants who had suffered the most eventually felt more euphoric and less tired.8 This suggested that these rituals might produce neurophysiological effects similar to what is known as the runner’s high: an endorphin-mediated effect of prolonged exhaustion that is ultimately experienced as pleasurable. But to examine these mechanisms more directly, we needed to turn to controlled studies. In a series of laboratory experiments, my colleagues and I looked at some of the specific components of such rituals and examined the effects of each component independently. We found, for example, that when used in the right context, pain and physiological arousal can independently increase prosociality. We also found that manipulating synchrony, a frequent ingredient of collective ritual, led to increased endorphin release, as well as to an increased sense of similarity and cooperation between interaction partners, which in turn led to increased prosocial feelings and actual behaviors.9
The next step is to bring these different levels of analysis together. For example, can we use our biological measures to predict people’s loyalty to their group or their position in its social network? And, can we design longitudinal field studies to examine those outcomes more dynamically?
Avoiding the mistaken certainty of a misleading answer
This research paradigm is not easy. It is costly, and the costs are immediate, while the yields are long-term. It is stressful, challenging, and fraught with institutional obstacles. Most universities pay lip service to interdisciplinarity but rarely create the right conditions for it to flourish. Different disciplines have different incentive structures, different ways of assessing outcomes, and radically different ways of training graduate students. Both as a student and later as faculty, I have been lucky enough to be part of several interdisciplinary institutions that broke down those barriers. Not by doing away with disciplinarity and specialization (after all, there can be no interdisciplinarity without disciplines), but by fostering interaction and collaboration; by connecting people not based on common methods (methods are tools, not goals) but on common questions.
This collaborative model of research can allow us to answer these questions in more holistic ways, and to draw stronger inferences by triangulating our findings. And when those findings do not converge, it can give us the opportunity to reevaluate them and generate new questions. For example, in a study of a fire-walking ritual, we found that people’s phenomenological experiences radically differed from what our biometric measures showed.10 While they assessed their state as being completely calm during the fire-walk (and in fact challenged me to bet money on the veracity of their claim), our measurements showed that their pulses were so high they risked having a cardiac event. Two different methodologies, two completely different insights. So, what is the phenomenological experience of someone who remembers feeling completely calm but has over 200 beats per minute? We don’t really know, but at least now we know that we don’t know. Had we relied on either one methodology alone, we would have gotten the mistaken certainty of a misleading answer.
Dimitris Xygalatas is an anthropologist and cognitive scientist at the University of Connecticut. His research interests include some of the things that make us human, including ritual, sports, music, cooperation, and the interaction between cognition and culture. His work on those topics involves the application of scientific methods and technologies in ethnographic field research. He has conducted several years of fieldwork in Southern Europe and Mauritius. After receiving his PhD in anthropology from Queens University Belfast, he held positions at the universities of Princeton, Aarhus, and Masaryk, where he served as director of the Laboratory for the Experimental Research of Religion (LEVYNA). At UConn, he directs the Experimental Anthropology Lab, which develops interdisciplinary methods and technologies for studying behavior in real-life settings. His research has been published in numerous journals across various fields, including anthropology, biology, psychology, religious studies, and general science. He is the author of the monograph The Burning Saints: Cognition and Culture in the Fire-Walking Rituals of the Anastenaria and coeditor of the volume Mental Culture: Classical Social Theory and the Cognitive Science of Religion.
This first appeared on items and is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License.
Image credit: Daria Shevtsova via stocksnap CC0
