Can AI do Qualitative Analysis of Interviews ethically? (Testing MAXQDA AI)

AI is revolutionizing research. However, the main concern is whether we can use AI to do qualitative analysis or coding ethically.

 In this article we’ll explore whether MAXQDA AI can help us conduct qualitative analysis in an ethical way.

My name is Bernard Mugo; I like to refer to myself simply as an academic.

In the past three years, I’ve helped more than 200 PhD students analyze qualitative data and finally complete their thesis or dissertations.

In the course of helping such a large number of students, I’ve gained invaluable experience of the best practices you can use to perform qualitative analysis of your interviews and to be able to retrieve quality findings.

What is qualitative coding?

Before we define qualitative coding let’s first define what a code is.

In qualitative data analysis, a code is a label or an interpretive statement to  particular information that’s important to our research questions or objectives.

Here is an example of a paragraph that has been coded.

We normally do qualitative coding when conducting thematic analysis.

Thematic analysis can be defined as a qualitative research method that aims at identifying patterns of meanings within data.

Before we get to themes we have to get codes first and then combine the codes based on shared meaning.

Let’s move to doing manual coding using MAXQDA and then we are going to do coding using AI and compare the quality of those two.

The example of a study that I want to use is Experiences of Patients with Heart Failure.

So I have two transcripts and we are going just to review one.

We are going to start in the middle section because in my previous article I coded the beginning sections of each of these two transcripts

The first thing that I’m going to do is to go and open MAXQDA.

I want a new project I want to call this manual coding

Now I want to do manual coding for only one transcript.

Two ways to import a transcript, drag and drop so I drag the transcript.

Then I drop it in the document section.

By the way, if you don’t understand MAXQDA, you’ve never used it, it’s good for you to watch my previous video.

Now let’s do some coding on this transcript that we’ve just imported by dragging and dropping.

This is a patient who is giving their experiences with heart failure we are going to generate codes and we are going to put the codes in the coding area.

So let us generate some manual codes.

When coding I would read the whole paragraph, then I come back and code again.

We familiarize with the data first and then we code.

I’ve found so many codes in that paragraph and we are going to gather those codes, let’s get those codes.

When did you get diagnosed of having heart failure?

I can copy this question and use it as a container to contain my codes.

Let’s go to codes click on the plus sign

Remember this is not a code, this is just a container to contain the codes, I just want to contain the codes under this question and call this Q1 for question one.

You see now we have formed a container, let’s do some coding.

When did you get diagnosed as having heart failure?

While doing medical for going to Gulf, was made un-fit and associate a cardiologist.

Highlight that statement right click and go to the second option which says Code with new code.

And then say when performing a medical for a job offer.

If I double click on that code and activate the transcript, you can see that’s the code we have, then I drag and drop that code under Q1.

 

Let’s keep going.  

They’re saying it’s been three years since I came back from gulf. The main reason for coming back is when in between I had swelling in my legs. I’m a driver. If I start today then two, three days I can come back so there were swelling in the legs and blisters I was not able to keep my leg down, that kind of pain I was having.

This is another symptom, it’s not under that question but let’s just code this.

This is a code that talks about their symptoms so let’s code this as experienced swelling in the legs and blisters as a result of heart failure.

That’s one symptom of heart failure for that code, I want to use color code red. All the codes that talk about symptoms, I want to mark them as red. It shows that those codes are related.

There’s something about manual coding.

When we are coding manually, we understand context and nuance and these are the two main aspects that make coding important or this are the foundations of coding.

You can see I consider context, which is something AI cannot do.

So, artificial intelligence might not know the nuance and the context.

We can be able to get very rich codes through the manual way.

Let’s keep coding, I was not able to keep my leg down, that kind of pain I was having. Another symptom is experiencing a lot of pain.

Let’s code that as experiencing a lot of pain in the legs.

And I want to mark that code red because it’s related to symptoms of heart failure.

Then I took a leave and came to kerala. When I came and checked, it had worsened because of the heart problem. I had swelling in my leg.

 Let’s copy this. I had swelling. So they said they had swelling in the legs.

This is another quote that supports the swelling in the leg.

 If we get another quote that supports a code that we had developed before, we drag and drop that inside.

I want you to check the numbers they come to two that means we have two quotes here that support the symptom or swelling in the leg.

Another thing we can see here is this person says that they took a leave so they were not able to continue with their job.

I can highlight all these from where they say they were working they had swelling, they drive a truck. So let’s start there the whole statements.

It’s been three years since I came back from gulf the main reason for coming back is because in between I had swelling in legs. I’m a driver. If I start today, then after two, three days, I only come back. So there was some swelling in the legs and blisters were there. I was not able to keep my leg down and that kind of pain I was having. Then I took a leave and came to Kerala.

This person had to take a break from their work. I want to call this as Heart failure symptoms contributed to not being able to perform in the workplace.

I want to color code this as purple ’cause that’s a code that talks about the impact of heart failure on this patient.

They say i took medicines and since I was taking medicines, it was okay,

Let’s code previously taken medication to try to manage heart failure symptoms.

That is another code, which talks about a completely different thing. I can use dark purple for that.

Then I’m an alcoholic, doesn’t drink so much, but I drink, I had no control over food, all sorts of food. Then I went to the Gulf but then one and a half years i came back i had some difficulties.

These are statements that support heart failure symptoms contributing to not being able to perform in the workplace.

So let’s drag and drop that.

Then when I work, I was having palpitation. I was not able to climb stairs.

I had palpitations. So that is another symptom of heart failure experienced palpitations when walking or climbing stairs.

That one I am going to right click remember that is red.

Therefore, if you were forming themes you would go and label them as experiences of heart failure.

 If you double click on my codes, you can see that I am extracting very clear statements, which I will then use when writing the findings report.

These are the participant quotes. That is how we do manual coding using MAXQDA.

You can see in manual coding we can be able to read the statements and we can be able to understand the context, the nuance and use our interpretation, which will enrich our study.

Remember in qualitative analysis researcher bias is not seen as a disadvantage.

It’s seen as an advantage because your interpretive angle is important to making sense of the data.

So you understand the context.

You understand nuance, and you understand what someone is saying that makes you have accurate codes faster.

Now let’s go into doing coding using AI from MAXQDA, and let’s compare the codes from my manual coding and from AI coding.

I want to go and open another MAXQDA.

This is a new MAXQDA project. And call this AI coding.

Let’s save my file then do some AI coding still on transcript number two on the study about experiences of patients with heart failure.

 For this document I want to import it using the import function go to transcript without timestamps and open.

This is the patient. Let’s do some AI coding.

I want to go and press control A.

 Right click.

 Go to AI Assist.

Go to the options that we have.

 Suggest new codes from selected text.

Let us see now MAXQDA is analyzing our data.

So here are some thematic codes.

We get some major themes and codes.

So diagnosis of heart failure, that’s a very good code.

 But let’s see where do they get themes/codes from?

 Let’s just click on all this codes to select all of them. There are alternatives.

Let’s see interpretive codes:

When you click, okay, all those codes are populated in the codes area.

Remember, these are AI codes.

Let’s double click here and let’s see what they quote. So this code is called Acceptance and Adaptation.

 You can see what AI does and this is what I don’t like about AI and I would not advise you to use it to code.

AI doesn’t know nuance or doesn’t bring an exact quotes.

You can see AI cited a whole paragraph just to give us a major theme which is not a code of acceptance and adaptation.

Let’s click on here and see, another whole paragraph

Look at all that statement.

Where did it get the code of disruption of quality of life? You can’t really know.

Reliance on medical expertise where did AI get that? It’s a whole paragraph.

So AI has an idea of what these codes talk about, but you can see AI is including the codes in the whole block.

You would have to still go in there and look for the exact sentences that AI got those codes from.

I want to give my verdict early, I don’t like using AI in coding and qualitative analysis.

 Why?

Because a tool is supposed to make the work easier but we can see AI is making the work more difficult.

 We have to figure out where did AI get that sentence?

The codes sound good “lifestyle factors”

So AI is doing a very good description which I like by the way

 Very good memos if you click on memo.

The significant impact on the condition on individual daily activity sleep and overall wellbeing, disruption of quality of life ideas  

You would still have to go and find the statements to support these codes.

So AI is good for getting ideas of codes that are in a given paragraph.

 However, AI will not do clean work for you.

AI can make your work more difficult.

 Let’s compare these codes with my manual codes.

If you remember my manual codes they were kind of more interpretive.

They were very specific and my codes are a bit long, normally I would go back and edit them.

So experiencing a lot of pain in the legs. This one you can see it’s a symptom of Heart failure.

 

Experience swelling in the legs and blisters as a result of heart failure is another symptom.

 So my codes are very interpretive and more specific compared to AI.

I can confidently say AI has better sounding codes.

AI gives you an idea of the main themes when you combine codes like emotional burden.

 Let’s look at that the psychological impact of living with a chronic and potentially life threatening condition.

So AI is good for ideation but it is not good for coding, we still have some issues.

If I was to extract all these codes and let me extract them.

 Let me go to reports.

Export code system I want to export all the codes from MAXQDA AI coding.

 I want to save that and include memo by the way.

I want to go in my manual coding.

 Go to reports.

Export code system and Word document let us save that

We have exported two codebooks or code systems.

Let’s double click and see.

This is the code system for AI this is fully AI generated.

I’ve not added descriptions for my work because I would go and add descriptions later after I get themes.

I will just generate initial codes then I’ll get themes.

That’s my work these two documents I want to test them using something we call GPT zero to see if they are mentioned or they are flagged as being AI.

I go to GPT zero.

I click on GPT zero which is an AI detector, not as effective as Turnitin.

 It’s better to use Turnitin.

By the way, probably for most of you students your university uses Turnitin.

That is why you should not use AI. But I will get to that later on the reasons I would never use AI in my coding.

 There are so many reasons and most of them are ethical reasons they’re not quality reasons which we see AI doesn’t have.

AI has good themes but you can’t know where it got those ideas from.

So let’s start with manual coding.

 I want just to copy this.

And I want to paste this in this box. Let’s paste and scan for AI that’s my manual coding remember.

You see?

 We are highly confident that this text is entirely human.

 So that’s my manual coding.

My manual coding has less than 1% AI percentage.

What about the AI coding?

 Let us go and check the AI coding.

Let’s test it with GPT zero which is a tool for checking AI content.

Now we are on GPT zero.

Now let’s go and copy my AI coding.

Let’s paste it here and let’s check.

What do we have?

We are highly confident this text was AI generated.

If you are doing coding using MAXQDA AI, a hundred percent AI generated that’s so high.

If you do your coding using AI you are risking.

Remember you are doing a PhD or a Master’s.

Your academic integrity is very important.

You are risking your academic integrity because you can be flagged by AI detectors, which include:

or any AI detector you use.

You can see they are very confident that this text was AI generated.

I would say you avoid AI.

 But there are still some ethical reasons on why we should not use AI for qualitative coding and thematic analysis.

Now let’s look at some ethical issues related to the use of AI in conducting thematic analysis and coding.

1.The loss of human interpretation.

Although AI is at an advanced stage, we still find that AI doesn’t understand context and meaning.

And because when we are doing interviews we deal with context and meaning AI can really affect the quality of your codes

2. Bias in AI algorithm. 

Most of these AI models are trained with data that may be biased data from online sources.

So these biases might be introduced in your research when you use this AI in coding.

3. Transparency and trust.

Research is about being transparent and having an open way of saying how you conducted your research.

Most researchers, when they use AI to analyze their data, they do not reveal that they use AI to analyze their data or to do coding.

So you are not being transparent, that means you are breaking one of basic ethical principles of research which is transparency.

4. Data privacy issues.

Before you interview your participants, you make them sign a form that says that you will not give their information to any third parties.

 If you then go and use AI, probably these tools such as MAXQDA AI are uploading your data into AI models such as chat-GPT.

That means the privacy of the participants in our research is getting interfered with.

When we use AI to code, we are providing sensitive personal information to AI and we already assured our participants that we are not going to reveal this information to anyone else.

So we are breaking the privacy confidentiality rule in research.

Now what is my personal verdict on the use of AI in coding?

 If you consider the ethical issues and my previous sentiments, you can see I would never use AI in coding.

Why?

1. Because of the privacy and confidentiality of the participants.
2. Because AI doesn’t understand nuance and context, only a real human being can understand that.
3. Because AI is a tool which is supposed to make you faster not slower.

When you use AI you need to proofread your work trying to find codes.

Trying to support the codes.

At the end of the day you spend more time using AI to do your coding than you would have done if you did it manually yourself.

So I prefer doing manual coding because as a researcher, I have the background information, the nuance, the context, my mind is free and I can be able to interpret and find more meaning compared to AI because AI sometimes doesn’t understand context and nuance.

So I would stay away from these tools or these AI add-ons.

 I would just use MAXQDA and do manual coding.

I would use MAXQDA to just track my information.

Just do manual coding using MAXQDA.

So far, so good the qualitative coding using AI is not up to par, and it just adds more work into the already tedious process of coding.