A Homebrew Method of Collaborative Qualitative Research

post-its

During the prior spring semester, I took a graduate course in qualitative research methods that focused on “action research.” That’s nothing unusual as far as graduate programs go — but what made this course truly unusual was that the entire sixteen-member class worked on a single, monolithic research project examining factors contributing to successful completion of the comprehensive exam in our doctoral program. A single qualitative researcher exploring a relatively well-defined problem can generate quite a bit of data; sixteen researchers using a “grounded theory” approach to a (initially) loosely-defined problem generate absolute scads.  Luckily, living in the Internet age, we leveraged quite a bit of technology to help us track, organize, code, and make sense of the massive amount of data we were collecting.

Herding Cats with Basecamp

The first thing we realized, even before we knew exactly what we were going to do, was that we needed some way to organize ourselves. Sixteen is a large workgroup, and even organized into smaller teams, inter- and intra-group communication and coordination would still be a challenge. We used a Web-based project management tool called Basecamp to help us keep track of the project. Basecamp is one of those suites composed of several different tools that are very simple by themselves, but quite powerful when used in concert. It provides assignable to-do’s, simple text documents, file storage, events, a calendar, and will send you email to summarize what’s changed. Just about every item — individual files, tasks, etc. — has a comments section for additional communication or notations. Everything from “Who still needs to complete IRB training?” to documentary exhibits went into Basecamp.

Collaborative Writing and Sharing Files with Google Drive

We used Google Drive — as part of our student WMApps accounts — for general file storage, collaborative writing, and passing things off between teams. For example, one team might interview a subject, another small group would transcribe the interview, and yet another would code the transcription. All these files needed to be collected, organized, and passed from team to team.

We were making up filing systems as we went, but in hindsight I think it would have helped us to be more directive about designing folder schemes and determining, for example, what types of documents should live on Basecamp, which should go into Google Drive, and how to structure the folders; occasionally, it was difficult to figure out where a particular item was being stored. But in general, the system worked very well. Straight-up sharing everything does require a certain amount of trust and deliberativeness of use, since anyone could delete, rearrange, or modify anything in the project space.

We also used the integrated Google Docs functionality of our shared Drive space to do collaborative writing and editing — using features such as simultaneous work on documents and spreadsheets as well as using the commenting feature to make suggestions and offer up information to each other.

Coding Research Data with nVivo

Finally, we used a piece of qualitative data analysis software called nVivo to analyze our findings.  NVivo is standalone software that runs on PC and Mac computers — and since it wasn’t “in the cloud” we had to come up with a system for integrating parallel work into a single nVivo file. We nominated one teammate as the keeper of the master file. Teammates who coded documents — whether primary sources or interview transcripts–would create a single nVivo file per item. We would then share the files with the “keeper” who would merge those individual files into the master project. This was somewhat fraught with peril, and if I had to do it all again I’d recommend using Dedoose or another qualitative data analysis tool that natively supports collaboration.  However, we were able to make it work (due in no small part to the efforts to the teammate who shepherded the nVivo files).

A Meeting of the Minds Through an Analog Exercise

A real challenge of distributed qualitative coding is that different people come up with different codes — or even similar terms that mean different things to the individual coders. To resolve this problem and perform the necessary task of categorizing our codes into themes, we turned to a tried and true technique from the realm of strategic planning — a “snow card” exercise.  We split the entire group into four groups of four. Each group then wrote every code — around 100 or so — down on Post-It(R) notes and went through the process of arranging them all into categories. Each team then wrote their categories down on Post-Its, stuck them on a whiteboard in the classroom, and collaboratively agreed upon a common set.

With that done, the keeper of our master nVivo file could use nVivo’s powerful organizational capability to rename codes across the entire project and also put them into individual categories- – giving us the capability to, for example, show in a few clicks a report of every datum that corresponded to a particular code or theme.

Technology, Our Mediator

If our semester’s work proved anything, it’s that sixteen determined people can do an amazing amount of work over the course of the semester. We generated hundreds of files, interviewed several students and faculty, transcribed those interviews, coded interviews and artifacts, themed our codes, and finally (as the “action” piece of our action research) wrote a sixty-some-odd page manual for our fellow students delineating our findings as recommendations to our fellow future colleagues in the EPPL (Educational Policy, Planning, and Leadership) doctoral program.

I don’t think we could have done it without technology, especially since many of our class members live outside of Williamsburg and/or have full-time jobs. Future iterations of the course will certainly improve the process, but we were, after all, building the ship as we sailed along.

About John Drummond

John Drummond is the Academic Technology Manager at the College of William & Mary. Originally from Mathews County, VA, John graduated from James Madison University with a BA in English in 1996 and an MS in Technical and Scientific Communication in 2002, and is currently studying for an Ed.D. in Higher Education at the W&M School of Education. He has been with W&M since 2007. In addition to working in IT, John has taught occasionally at W&M and previously at Tidewater Community College, and in other roles has been an author, a musician, a Perl programmer, a UNIX systems engineer, and a network manager. He resides in Toano with his wife Andrea and daughter Rebekah.

Comments

  1. Gene ROCHE says:

    Thanks for sharing your thoughts on this experience, John. I’ve worked on lots of projects over the last 30 year in my higher ed career, and I don’t think any other has generated as much energy across such a wide range of participants. Much of that energy came from the quality of the participants, but it was multiplied many times by the combination of technologies you describe so nicely in your post. Future generations of the course will build on these new tools, but you and your colleagues set a very high bar!