When your method fails

Stumbling upon new questions

Carla Pezzia

15 Feb 2016
Photograph of 'Donde Lloran los Valientes' (Where the brave cry), an oil painting by Antonio Coche Mendoza. Used with permission of the artist.Photograph of 'Donde Lloran los Valientes' (Where the brave cry), an oil painting by Antonio Coche Mendoza. Used with permission of the artist.

I set out to study recovery from alcoholism in highland Guatemala using a mixed-methods approach that included social network analysis (SNA). I had years of experience in alcoholism research, and SNA, I imagined, would help me understand how peer support influences sobriety. I drew heavily from the literature on Alcoholics Anonymous (AA) to formulate my questions. Because of the cultural context and the study population, Stanley Brandes’s (2002) work in Mexico and Ruth Bunzel’s (1940) work in Guatemala were the most informative. Brandes found in Mexico that the camaraderie developed through AA meetings substituted a night out with drinking buddies. In the Guatemalan context, particularly for the highland Maya, Evangelical Church masses serve a similar purpose to AA meetings (Bunzel 1940). However, based on my time working in the region, it seemed like this was no longer as common a practice, though it is often still considered to be in the literature. Therefore, I wanted to explore how supportive peer networks develop and might differ between those who attended AA, those who went to church, and those who found sobriety on their own.

Upon starting my SNA interviews, I realized that what I had planned for was not going to happen. For those unfamiliar with SNA as a method, it includes structured interviews and intricate computational analyses that allow the researcher to measure the relationships between people. This can then be used to make inferences about the network as a whole and the role of particular individuals within the network, such as which network characteristics and  peers best support recovery from alcoholism. My structured interviews included a series of questions that asked participants to name the people they regularly interacted with and the purposes of such interactions. When asked to identify individuals who could be leveraged in the recovery process, most interviewees typically answered, ‘no one’. This answer stayed the same regardless of a person’s stage of recovery. This left me with very little data to do the in-depth analyses I had hoped to accomplish through utilizing SNA. I had expected the occasional answer of ‘I do not have any support’ as a way to demonstrate how strong one was in achieving and maintaining sobriety on one’s own. I had even expected to hear the answer ‘God’ as a way for respondents to demonstrate their newfound religious fervor. But the reality was different than what I anticipated. The people I talked to were suggesting a profound sense of isolation and weakened ability to trust in others.

Reflecting back on this methodological ‘failure’ I realized multiple things about research in general and about myself as a researcher. The two most valuable lessons I learned were to be willing to push methodological boundaries and to not take anything for granted. As I designed my project, I knew I wanted to take a mixed-methods approach. First, my dual training in public health and anthropology positions me to appreciate both quantitative and qualitative perspectives to form a more holistic understanding. Second, I assumed this would increase my potential for success on the job market by allowing me to speak to a wider range of audiences. Alongside a community-based psychiatric assessment survey and phenomenological life stories on alcoholism and recovery, I incorporated SNA as a layer of methodological complexity that went beyond the traditional interviewing styles. The whole project was ambitious and several reviewers at different funding agencies thought I was trying to do too much with SNA, which only emboldened me more to do it. In the end, it was my failed attempt at SNA that led me to a more nuanced and holistic understanding of the local cultural context.

After I shook off my initial frustration with the fact that the method did not work as I intended, I decided to use this ‘failure’ as an opportunity to do some unplanned exploring instead of taking responses for granted while continuing onto the next planned stage of research. After thinking through my previous research and what I knew about alcoholism, I considered the response of ‘no one’ to be not all surprising, and interpreted the data as participants’ feelings of familial and social abandonment prior to achieving sobriety. This is common for people in recovery according to the literature. Had I stopped here and not strayed from my initial research design I would have missed out on some key insights. I decided to also interview people who were not in recovery to demonstrate the difference in perceived connectivity. I selected five people I knew to be socially active in the community and who had a good family life. I expected them to have a strong support network. Imagine my surprise when they answered the social network questions in very similar ways as the alcoholic individuals in recovery! It was apparent that there were other factors that contributed to general feelings of disconnection (for example, a  break in the social cohesion as a result of civil war and the small-town lack of privacy, which translates into guardedness). These data are still preliminary and I plan to conduct further research with a systematic sampling plan. However, while preliminary, they have allowed me to understand the recovery process from a different perspective than what has previously been reported.

Ultimately, the significance of these interviews – and thus for my subsequent analysis – is that they led me to realize that I needed to be willing to take risks with my research plan. I now know not to become disheartened by an apparent ‘failure’ and instead embrace the situation when things do not go as anticipated. Had I taken the initial responses at face value and gone along with the consensus currently in the literature, I would have missed a very important understanding of local dynamics.

About the author

Carla Pezzia received her PhD in anthropology from the University of Texas at San Antonio in 2013. Her dissertation on recovery from alcoholism in Guatemala won the 2014 Kurt M. Landgraf Outstanding Dissertation Award, co-sponsored by the American Association of Hispanics in Higher Education and the Educational Testing Service. She has worked in Guatemala since 2006 as the co-director for the Ethnographic Field School of North Carolina State University. She is now an Assistant Professor at the University of Dallas and has research affiliations with the University of Texas Health Science Center at San Antonio and the South Texas Veterans Health Care System.

References

Brandes, Stanley H. 2002. Staying Sober in Mexico City. Austin: University of Texas Press. 
Bunzel, Ruth. 1940. “The Role of Alcoholism in Two Central American Cultures.” Psychiatry 3: 361–87.