Coding qualitative data as a team


#1

I’ve coded a lot of qualitative data, but always solo or with just one other person. For the first time in my short career, I’m helping lead a team (4-5 people) that will code data from nine one-hour interviews. I’ve found info on creating codes and a code book as a team, but very little on the actual coding process (how many coders per transcript, how much detail each coder goes into, etc.).

So far, for each transcript we’re thinking of having 1 primary (fully codes the transcript) and 1 secondary coder who reads through the primary coder’s work, taking note of anything they disagree with (non-codes, inaccuracies, etc.). Then, the two coders discuss disagreements and come to a consensus.

Have you led a team of coders before? What sort of process did you follow? And any resources or general tips?


#2

I’ve been a part of a three-person coding team. We had a process to create a baseline of inter-coder reliability and then made notes of problematic passages. All three of us would code the same interview. We would then get stats on inter-coder reliability. If the reliability was high, at least high 80’s, we’d say we had a good enough baseline to start coding separately. If there was not high reliability, then we would meet to discuss where the issues were stemming from (often the size of coding chunks). Then, we’d code another interview together. If we were using the same codebook over a long period of time for longitudinal interviews or made significant changes to it, we’d repeat the process at the beginning of each year or after the change to re-calibrate. I hope this helps!


#3

I’ve had the privilege of coding using both of the methods that are mentioned. My personal preference is using the method @considersour mentioned because it does lend itself to greater reliability. However, I think what you are trying to do with the interview coding is critical as to the method that will be most useful to you. Additionally, the resources that you have may play a factor as well. For example, if you’re using Atlas.ti and have intercoder mode enabled, using the method @considersour mentioned is a lot easier. But, if you are doing it by hand (an the interviews are very long), then using the method you mention @evan_poncelet might be easier considering the purpose and time constraints.


#4

Thanks both for the helpful responses. We’ll be using NVivo for coding. It looks like NVivo has a “Coding Comparison query” function that can calculate IRR, which I haven’t used before but may try out. It should be quicker than the last time I calculated IRR for a project (using Word for coding :grimacing:).


#5

If it’s a quick and dirty coding of short responses, I find Excel is fine and there are some easy tools to use online for determining inter-rater reliability. I’ve really enjoyed using MaxQDA in the past, but haven’t tried it with other coders. I’ve also done Dedoose many years ago with a colleague and found that really good for coding with others. That software has the added benefit of an online portal and monthly pricing as opposed to other qualitative software that is annual and downloaded software.


#6

Any one aware of Nvivo training? I tried to work on it but wasn’t easy.