Nvivo 15 Coding Interviews with AI (Does it produce quality)

In the past few years, the use of artificial intelligence to make different tasks easier has become quite popular.

This has been the case in the academic field, although the use of AI for different academic tasks remains quite debatable and quite opposed or discouraged in many instances, the use of AI has also been introduced in qualitative analysis through the use of tools such as N-Vivo and MAXQDA, which are very popular qualitative data analysis tools.

In this article, I will explore how we can code interviews with N-Vivo 15 using AI and whether we can produce quality codes with the use of AI in coding.

I’ll also explain in the article other issues that emerge when a person decides to use AI to do qualitative coding or conduct thematic analysis.

Before we move to coding, I want to clarify what is qualitative coding and give examples of qualitative codes.

 I want to read this excerpt from the book Qualitative Coding by Johnny Saldana.

Give you examples of some codes, then look at some examples of manual codes, and then I will generate some codes using the AI function in N-Vivo, and then try to see the difference in quality.

So let’s read this together.

Driving West along the highway access road and up main street to Wild Pass Road, there were abandoned warehouse buildings in disrepair spray-painted gang graffiti on walls of several occupied and unoccupied buildings. I passed a Salvation Army thrift store, napa auto parts store, a tire manufacturing plant, old houses in between industrial sites, an auto glass store market liquors, budget tire, a check caching service, more spray paint, a check caching service . More spray paint was on the walls.

If you read that paragraph, you’ll see that there is this sentence which is labeled number one that reads abandoned warehouse buildings in disrepair.

This person coded this as buildings.

Then there’s a second sentence spray-painted gang graffiti on walls of several occupied and unoccupied buildings.

That one is labeled or tagged as graffiti.

 And then passed a Salvation Army reef store, neighbor auto parts store, a tire manufacturing plant, old houses in between industrial sites, an out glass store market liquors and a budget tire.

that section is coded as businesses. 

and then more spray paint was on the walls. 

Graffiti.

Now when you look at the quotes, you can see that this person is trying to bring interpretation.

When we are doing thematic analysis and qualitative coding which is the first step of thematic analysis, we tag important sections of our interviewee responses.

In those important sections we tag or label them into what we call codes.

So a code is a label or an interpretive statement to any information that is important to our research questions.

So by simply tagging these sections, we are doing what we call coding.

 By coding we are providing interpretive statements.

Now let’s look at examples of manual coding.

This are two interviews talking about the importance of parental engagement in what are called elementary school.

 These are the kind of interviews that were conducted on the importance of parental engagement in such schools.

So I went and tagged important sections of the transcripts with codes.

So if you go to the code section in N-Vivo, click on the arrow pointing to the right.

You can see that I first generated initial codes. Let us look at some initial codes I generated.

• Making phone calls to parents. 
• Social and community-based events.
• Establishing good relationship with parents.
• Holistic partnership between school and parents.
• Having informal chats with parents.
• Establishing a community with school at the center of it.
• Cultivating a good relationship with parents.

 All those codes are in initial codes.

 Then I went and revised the codes to make them better.

• Making phone calls.
• Social and community-based events.

And remember these codes were related to the first question, In your experience, what does it mean in school when people talk about parental engagement.

 And if I close my transcript, you can see there are codes that are marked red.

Those are the codes that came from the first question.

And there are those that are blue.

They came from the second question.

For this document I went and coded it manually and tagged important sections.

For example we can see in my revised codes in your experience, what does it mean in school when people talk about parental engagement?

Involves establishing a good relationship with parents of course they are replying.

That sentence is coded as establishing a good relationship with parents.

 Then I went and combined the codes based on shared meaning, and I got two themes.

Subjective meaning of parental engagement is one of the themes created.

Let’s see the codes under that theme.

• Holistic partnership between school and parents.
• Establishing a community with the school at the center of it.
• Good relationship with parents

Those are the codes.

 If you double-click on one of the code, you can see the original quotes that these codes came from.

 Creating a community if you like, so school is at the heart of the community.

That’s basically the first theme.

The second theme that I had was types of parental engagement.

• We have informal chats.
• Making phone calls. 
• Social and community-based events. 
• Cultivating good relationship with parents.

So these are codes that I went and read the transcripts, tag them, revise them and then combine them together based on shared meaning to form themes.

So let’s use AI to try and code the same transcripts and try see whether we will get the same results, and whether this work is justified.

 Open N-Vivo.

 We are going to create a new project and call it “AI project”.

We will analyze the same transcripts and we try to see the quality of the course and whether AI is able to fetch quotes.

So let’s do this

 In the file section I want to go and import the two transcripts that we intend to code.

 How do we code using AI in N-Vivo?

 So in N-Vivo at most of the times if you try to highlight a whole transcript then go to AI assistant, you can only summarize the text; this is one of the problems of trying to use AI to code in N-Vivo.

Ideally, it’s like use of AI is forcing us to already know the themes that we want to come up with because AI assistant only suggests child codes.

That means we have to go to the code section, look at the first question.

For instance, what does it mean in school when people talk about parental engagement?

First, I have to create a theme. This is like we are leading up to deductive thematic analysis rather than inductive, and I don’t think AI is a good tool to do inductive thematic analysis.

Here we have, for example meaning of parental engagement.

 You can see with AI we have to create the themes ourselves, know the  meaning then go and try to use it.

We can create another theme for the second question.

 Theme B : Effective forms of parental engagement.

 Effective forms of parental engagement that is the second theme.  

In coding, we’re forced to adopt a different method of conducting thematic analysis.

 If you have read materials or you have come across the two types of thematic analysis normally we have inductive and deductive thematic analysis.

Inductive thematic analysis is where we go and generate codes , then combine the codes based on shared meaning to form themes.

In contrast, deductive thematic analysis is where we come with the codes a priori or before.

 It’s like already by using AI, we are being forced to only use the deductive approach because we have to know the themes first.

 I have to highlight any section then drag and drop it inside the theme container so that AI knows we only want to review that section.

 Let’s try the second section from the second theme.

Drag and drop that inside the container.

 Then suggest child codes then generate.

And another disadvantage of using AI is that you can see we are dragging and dropping the quotes inside the theme, this will mess up our structure and our numbers.

Here we have so many codes I want to click okay to all of them and try to identify these codes.

 So effective forms of parental engagement.

• Building relationships.
• Informal communication. 
• Overcoming barriers. 
• Positive feedback. 
• Social and community based engagement.
• Strong purpose.

I think the most effective ones are the ones that have strong purpose so parents know where they are coming in, and AI coded that section as strong purpose.

I think that’s a very legitimate code.

If you double click on the code of course you are gonna reveal the whole paragraph.

 So I want to un-code all the other sections so that I’m left with the important section alone.

Let us begin with the first code building relationships.

It’s all about building those relationships, I think, so you un-code the other section, you can predict where the AI got that code from.

Informal communication is a code.

 Let us read, as well as even things like phone calls and grabbing parents on the playground for informal chats about their children.

That is informal communication, then un-code the unwanted section

 You can see when you’re using AI to do coding, you have to do a lot of work by reading and checking where the quote came from.

AI can give you suggestions of codes but AI can not give you accurate information or show you where the quotes came from.

 AI just includes the whole paragraph that we try to code.

 Overcoming barriers.

 I don’t think that’s a legitimate code because we are looking for effective forms of parental engagement. 

So overcoming barriers cannot be, you can see AI has one wrong code. 

positive feedback this is also not an effective form of parental engagement because if you read this quote

I think the most effective are the ones that have a strong purpose so the parents know where they are coming from, obviously some of our parents school is a barrier so I think some of most social community-based events that get parents through the door just for coffee or chat are really important as well as things like phone calls and grabbing parents on the playground for informal chats and not just the bad stuff but good stuff too it is all about building those relationships, so positive feedback and overcoming barriers.

 I don’t like those two codes

 You can see AI gives you a lot of work in editing and reviewing the codes, that is how AI does the work.

 But when we compare the codes, let us first delete overcoming barriers, and positive feedback.

 Let us now compare manual codes with those generated using AI.

Remember, we had a code called

• Holistic Partnership between school and parents.
• Good relationship with parents.
• Establishing a community with the school at the center of it.

Those are manual codes

Compare with this

• Community involvement.
• Partnership between school and parents.

Remember I had a code called holistic partnership between school and parents.

So these codes are kind of the same.

And remember, the first set of codes I got them manually, I didn’t use AI. They are my own ideas.

And the second set of codes I used AI.

In providing suggestions, AI is number one, but in getting the right quotes, AI doesn’t help you.

The use of AI in thematic analysis will not save you time; sadly, it will actually give you more tasks as you have to review the work.

Personally, I never use AI in coding, and this is my opinion.

 Why?

 Because I can do this work faster if I do it myself and avoid so many mistakes, and get the quotes directly.

Another thing is that AI destroys the coding structure.

For instance we see 4 codes.

 If I right-click and aggregate codes, we will see five codes.

Why?

 Remember we put a quote here so that AI could help us code and I believe if you were to remove it all the codes will disappear.

You see, they disappear because that’s where AI is getting information from.

Another issue that makes me not use AI in coding is that AI does not help you save time, as you have to go and review the work, record, and clean up the codes, which you could have done by just coding directly.

 Another issue with AI and coding is that you are providing your data from your interviews to OpenAI.

Now this raises a lot of ethical issues including confidentiality of the participants and other ethical issues related to research.

Another issue that you may face when you keep giving your data to AI and OpenAI is that you can risk getting your data flagged by AI detectors in tools like Turnitin, because now you have given your data to a third party.

 So sometimes the output from AI may be flagged as AI content.

If your content which you went and conducted interviews, did analysis is flagged as AI, that will not be a very good look on you especially for a PhD.

That is why I don’t like risking giving my data to AI for all these suggestions.

I like doing it manually which save more time and also guarantees the credibility of my work.

It gives me pride knowing that I understood the context and nuance and provided an important interpretation.

AI may miss the nuance of the participants. What are they responding to? What’s the context? AI may not be able to understand in a deep human level what the participants are trying to say, but the researcher will be able to do this.

My verdict is we should always use AI if we only want suggestions on how to rename our codes, and we must be very careful not to give AI our participant information.

Personally, I don’t use AI at all in coding because I’ve practiced and I’ve done coding, which means for me it’s faster to just do it manually.

Also, it will guarantee credibility of my work rather than trusting my work with AI tools.

You can always bring a big problem with the credibility of your work, especially with AI detectors such as Turnitin, which can flag you.

These tools are meant to make our work easier not to increase our workload.

So basically, these are my suggestions on why we should try to do manual coding, compile codes, and get themes rather than trying to use AI.