ChatGPT vs. Doctors: The Battle of Bloating Notes and Hilarious Hallucinations
Written by Dr Nick on August 7, 2023
The Tale of Prompt Engineering Gone Wild!
This month’s episode of “News you can Use” on HealthcareNOWRadio features news from the month of July 2023
As I did last month I am talking to Craig Joseph, MD (@CraigJoseph) Chief Medical Officer at Nordic Consulting Partners.
This month we open with a detailed discussion of the recently published paper “Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine Residents” which dives into the use of ChatGPT in healthcare generating a concise history of present illness (HPI) based on patient interviews. They explore a study where an AI was given a prompt to create an HPI after analyzing a patient interview. Initially, the AI struggled with hallucinations, making up patient details that were not provided in the interview. However, with iterative modifications to the prompt, the AI’s performance improved significantly.
We discuss the potential approaches to using Large Language Models (LLMs) and the potential of AI to aid physicians by generating efficient and clinically relevant content, leading to increased efficiency in healthcare practices. We view these technologies as tools to complement and support doctors rather than replace them entirely. And we go further into the potential to enhance medical decision-making and bring specialized expertise into some of the more general medical domains, making physicians more efficient and effective in their practice. We both believe that the key asset and skill in the future will be that of prompt engineering in AI applications and the potential for generative AI to support and enhance the efficiency of medical professionals.
Digital Health Tech for Seniors: Where Wearables and Zebras Collide
We discuss the paper in Nature: “Digital Health for aging populations” and how these digital health technologies offer a real opportunity to promote independence and reduce hospital stays and drive positive outcomes for elderly individuals
Listen in to hear our review of the Note Bloat research in the Epic Research Tools: “Two Years After Coding Changes Sought to Decrease Documentation, Notes Remain ‘Bloated’” that offers some interesting if somewhat less than stellar news that our notes have not gotten less bloaty as notes have become longer despite efforts to make them more succinct. While doctors are spending less time writing notes, the overall quality of the documentation remains a concern.
You can read more about the series here and the concept of keeping up with innovation in healthcare. Please send me your suggestions on topics you’d like to see covered. You can reach out direct via the contact form on my website, send me a message on LinkedIn or on my Facebook page (DrNickvT), or on Twitter by tagging me (@DrNic1) and #TheIncrementalist or you can click this link to generate a ready-made tweet to fill in:
Listen live at 4:00 AM, 12:00 Noon or 8:00 PM ET, Monday through Friday for the next week at HealthcareNOW Radio. After that, you can listen on demand (See podcast information below.) Join the conversation on Twitter at #TheIncrementalist.
Listen along on HealthcareNowRadio or on SoundCloud
Raw Transcript
Nick van Terheyden
This month as I am each and every month, I’m delighted to be joined by Dr. Craig Joseph. He’s the chief medical officer at Nordic consulting partners. Great. Thanks for joining me today.
Craig Joseph
Thank you for having me.
Nick van Terheyden
So this is July’s News You Can Use July 2023. It’s the middle of summer, we’re under a massive Heatwave, or at least some people are, it seems spreading all over the place. And, you know, there’s certainly been some things going on, I think we’ve kicked off a lot of the recent episodes talking about AI and large language models and chat GPT hit the news, just recently, we saw that with a JAMA Internal Medicine report that talks about the comparison of HPI history of present illness summaries generated by chatbots. And then generated by senior internal medicine residents who of course, have lots of time to be able to do that, right?
Craig Joseph
They’ve got nothing better to do. And they’re really excited
Nick van Terheyden
about it, of course. So how did it do?
Craig Joseph
You know, I’ll summarize it, as I think we’ve summarized many things with, with generative AI, it’s all about the prompt, baby.
Nick van Terheyden
Oh, yeah. Yeah. Engineering.
Craig Joseph
They? Yeah, it’s a very interesting study, basically, they made up an interview of a patient. And so hey, what brings you in, or I’ve got this pain here. And then they asked some questions, and the patient answers those questions. And then from that you’re supposed to generate, if you were a human, you would generate a history of present illness HPI. And they wanted to see if Chet GPT could do a good job with that. And so to attempt to, you know, get that they gave it a prompt and said, Hey, read this interview and write a short, concise history of present illness. And what they got back was pretty good, except for some of the hallucinations. Nick, if it weren’t for the hallucinations, it would be it would be super awesome. And so the hallucinations were the the, the interviewer didn’t say, if this was a, I think it didn’t say the gender of the patient. It certainly didn’t say the age. And so most of the histories of present ailments that you and I read, always do start off with, you know, this is a 72 year old male with the previous history of whatever. And so it says, Oh, yes, let’s tell you that. So it started off with the age and gender of the patient, which again, was never presented. So that was a no, no. And they actually got it a little bit better. So they they went back to the prompt and added something about, you know, do not reference any information that is not in the in the transcription don’t make up stuff. And, and then they made another little tweak. So they tweaked it three times. And by the third time, they teach it twice to get to the third version. And by the third version, it was, it was pretty good. It did a pretty good job. Many doctors, you know, some of the attendings were asked to say, to read these histories and say, Hey, was this a, was this a human that created this? Or was this a chatbot? And they could still generally tell, not 100%, but I think it was like 65%,
Nick van Terheyden
I’m just gonna say they can probably tell because they know that some tired senior resident is so Junior resident is, is going to produce something that is way more succinct.
Craig Joseph
Well, that. Yeah. That might be true.
Nick van Terheyden
That’s an easy ask. I think that was a fair question. But anyway, Sorry, I interrupted
Craig Joseph
under any circumstances is a is quite interesting. This thing was not, you know, this is not a a specific, you know, kind of healthcare, AI, just kind of off the shelf, the same stuff that you and I have access to. And so it’s, you know, it’s pretty good that it got that far with very little training. I, again, I do not think that this is on the hype cycle. I think that that the generative AI like this text based AI is really there. Again, as we’ve said, probably for a year now. Yeah, it makes stuff up. Yeah, it gets its facts wrong. So yes, it all should be reviewed before it’s trusted. But But still, if it can do the first pass and get it 80% of the way there. That’s a huge help.
Nick van Terheyden
Yeah, so I’m gonna completely step outside of my lane here and suggest that what’s really interesting about this, is that it gives a little bit of insight into one of the two challenges that I think we have with these MLMs and AI, content generation in understanding how they do it. Right. And you know, everybody says, well, it and you you even use it, we both do. But I try not to hallucinate, anthropomorphize these devices that are essentially word generators. And what I find interesting about the fact that they pulled out, you know, this is a male, age 72, or whatever it was, is, it gives me a little bit of a peek underneath the covers, and I could be entirely wrong that says, you know, what it’s doing, it’s saying, I found all of these clinical things and phrases that were linked in some other documents somewhere else in the billions of data points, and so forth. And I see that, oh, and as I look further up the tree, I see it started with this, and I’m just gonna pull that up. So it helps me understand why it comes up with data. That’s not really there, because that’s the purpose. The other thing that strikes me as you describe this is their reaction, because my reaction to that would have been Hold on a second, why did you do that? And I think that tends to dive off into this deep end of AI interactions where the system gets angry, or, you know, stuff stances feed based on some of the things that I’ve read. And I don’t think I would be interested in because if you could actually query and say, Well, hold on a second, where did you see that? But I suspect it would just make up more stuff. Because it’s, it’s a little bit like a three year old being asked, Did you take the last cookie from the jar? And no, no, it wasn’t me, you know?
Craig Joseph
Yeah, it’s, it’s, um, it, you know, you can point out when it makes errors and say things, if you’re following the chat, say, you know, the, the gender or the age was never presented. And it generally replies with oops, let me I see that. Let me try that again.
Nick van Terheyden
And then comes up with another gender and another age.
Craig Joseph
Again, you know, I’m not gonna say you’re wrong. It’s, it’s, I think
Nick van Terheyden
it’s unusual, I’m gonna take that as a score for this interview.
Craig Joseph
As we started off with, you know, it’s all about the prompts. Right. And, you know, when you when they think that final modification of the prompt was, Don’t hallucinate things, you know, don’t reference things that aren’t in the aren’t in the interview, be succinct, use clinically appropriate terms. Because some of those would, you know, veer off into using non clinically specific terms that, and that’s a clear giveaway, that, you know, a doctor has not done this, right. So, but it did it improved quickly. And I think once we get those prompts done, that you’re getting, you know, maybe you’re starting with nothing at 80%, you can get to good 95% accurate and fast. And you know, then the question is, Well, are you losing something by not starting from scratch? Are you are doctors going to lose something because they don’t have the they start to lose the ability to generate these things in their heads. And all they’re really doing is approving something that the you know, that an AI is created? Are we are we creating a generation of physicians that will be in big trouble if that tool is gone?
Nick van Terheyden
So that sounds very much like the argument I hear repeatedly around automated flying, you know, the, the pilots losing the skill set, particularly with the new capabilities that are emerging, where, you know, their auto land is now available in a lot of places, it requires a lot of technology, and, you know, still supervised, but they lose the touch. I could tell you, I don’t know about you, but I’ve certainly been on some planes where I think whoa, he hasn’t landed that thing. You know why? Cuz we really come down, you know, pretty hard, and it’s not wind or anything, but, you know, huge respect. And I think they’ve addressed that by continuing. So they don’t just do it. I think that’s one of the key things. And, you know, the other thing that sort of strikes me through all of this is everybody’s listening to this or looking at this technology and going well, we don’t need doctors. That’s not what this says to me. It says, No, we really do. But we can make them much, much more efficient, to generate the content that’s necessary and to be succinct. Oh, and by the way, I think you can create two pieces of content from that same interview, one that’s dedicated to the clinical profession and yeah, we abbreviate use all these terms because it gives a really succinct you know, it used to be the four by six or Whatever cards that we used to write on, we could really get it in, I think you and I have talked about this, I remember you talking about one of your records, and it was like two lines, and there was a full clinical history in there. And it does, it works, I read stuff, and you know, go, oh, yeah, I get all that and some of the annotations, but then you also generate something that is equal to, but as in, you know, easy to understand comprehensible for the general public who don’t have that understanding, that can then be part of this sort of open notes forum that you and I think are both big fans of so this is, to me, this is hugely exciting, and brilliantly done, because they didn’t take real data and real patients and put it in because, of course, I feel like that might be going on. I know, five was working in this space, and I was there. And I know it’s not right, I’m not advocating it. But I just know, high pressure time, you know, it produces decent content. And as long as you don’t just take it and dump it in and use it as a sort of starting point. That’s cool. The problem is the private information. And we don’t want to have that shared. I don’t know about you, but my prompts have changed when you log into the system. Now it says, Warning, you know, we’re looking at this we’re checking, don’t put any private, you know, so they’ve really sort of jumped on that. I guess we need private instances, maybe there’s a better version of this. But to me, it’s really exciting.
Craig Joseph
No, it is exciting. And I have absolutely uploaded I think I mentioned this on a previous show, I’ve uploaded patient information, and I don’t care who knows about it, because it’s my patient information. Right. And, and so if anyone is reading about, you know, a pediatrician, that is a George Clooney lookalike, like that’s probably based on the information that I that I’ve uploaded. Everyone. Everyone says it, you know, another another? Yeah, so I don’t think anyone’s talking about getting rid of physicians. I think we’re talking about making physicians more efficient, also, kind of adding expertise to physicians that might be outside their, their areas. And I, you know, I think of a space I think of when I, you know, as a pediatrician, I learned many things that focused on kids and are that are rarely seen. But when they’re seen, you never want to miss them, even though they’re very uncommon, right? I think good example of that, I’m just gonna give this rather odd reference, I remember being a second year resident, and we were doing board review with the chief resident, so there’s, you know, 30 of us in a room, and they put up a slide, and it’s a, it was a picture of a young child, and without clothes on facedown, and there’s this big rash all over, you know, the lower back and the buttocks and the upper legs. And, you know, we’re we’re kind of just looking and the question, of course, was what what’s causing this rash? This, the senior resident said, Listen, I’ll give you a little hint. Any rash of the buttocks on your board exam is Henoch shoreline prepper until proven otherwise. And, and that’s, that was a key thing, because you don’t generally see that kind of a rash, that kind of a distribution. And, you know, I certainly got that because I did a pediatric residency but family medicine, doctors often don’t get that much pediatrics because they’re doing a lot of other things. And certainly adults aren’t going to get that kind of a internists, I’m sorry, adult specialists are not going to get that kind of so you know, when those words are maybe said during a transcript of well, the, you know, the rash started on the butt or describing what those words to me are going to jump out as an expert and say, I have to think about this weird thing called HSP. That I don’t normally see that commonly. But to others that might, that’s where the I think that some of this technology can really come in, because it is quite good. When you’re asking it to give, you know, what are some of the diagnoses I should think about that I don’t normally think about. It will make some suggestions like that.
Nick van Terheyden
Yeah, I think 100% And, you know, we short shortcode that to zebra hunting, I think that’s the thing that the clinicians know understand, you know, when you hear hooves, think zebras, not horses, you know, find the rarities. And, you know, again, one of the great opportunities that you know, this represents to extend our capabilities as we become more and more specialized. For those of you just joining, I’m Dr. Nick the incrementalist today as I am each and every month. I’m delighted to be joined by Dr. Craig Joseph. He’s the chief medical officer at Nordic consulting partners where news you can use for this past month we of course spent a fair amount of time on chat GPT and I think a great piece of research demonstrating the value proposition done in a, I think, very clever way with good detail, you know, diving into and, you know, we both concur in this particular area and I’ll sort of re emphasize it, you know, the prompt engineer or prompt Whisperer is going to be a critical skill set. Going forward, it doesn’t matter where you work as a clinician, engineer, programmer, you name it, I think it’s, it’s absolutely key to all of that. So all good news, let’s move on to some other things. Whilst we still have a bit of time, there was a publication in nature talking about digital health for aging populations. We are seeing more and more folks trying to push away from the hospital. I know, we’re still building them, that still surprises me when I see this, but nobody really wants to spend time there. I know, I don’t. And, you know, to be clear, I they’re essential and important. But you know, from an aging standpoint, we want to stay out of them as much as possible and use them for the very specific purposes. But how do we support that activity? And, you know, the reality is that there’s all these technologies and the wearables and various digital innovations that have this enormous potential to drive independence. But are we using it correctly? Do we have the right pieces that are in place? Is the you know, what categorization? I think we’ve established at this point. That, you know, they have to be clinically valid. I know I certainly in the past have sort of, not dismissed it, but discounted it and said, you know, if you’re just getting some measure, but I’ve seen too many of these measures where I go, Oh, my God, really? All right, I had a blood pressure cuff to be clear, one of those, you know, cheapos that I bought, and I was doing really well. And I’m thinking, Oh, this is really cool. And then I think it might have been my, my daughter who’s an emergency room physician, and she was like, you should check this properly by we got my old sphygmomanometer out, and how it was way, way off. So I’m super careful with it. So you know, there’s those elements. I think there’s fantastic technology I’ve used the CGM don’t know about you, but you know, I use it. It’s informative. I don’t have it on all the time. But you know, and I have a ring that’s tracking everything. It’s been fantastic. I’m, you know, recovering from some procedures that were done. I’ve seen the sort of steady progression of sleep and the improvements. How do we get all this reconciled? What are your thoughts around all of that? And, you know, indeed, the paper?
Craig Joseph
It’s good, Nick. Now, I was trying to think of something sarcastic to say, cuz you’re asking these very big questions to which there are no easy answers. I think, firstly, you know, we have to acknowledge that many of us stereotype the elderly as being tech phobic, right? They don’t, they don’t understand. They don’t want to be around, it’s too confusing. And they’d rather you know, kind of stick with their old ways. And my experience, and I think the experience of many years that that is not the case, it’s often the opposite of that. In fact, my, my mother in law got kicked off Facebook. I’m not sure exactly what she did to do that. But I picked up Facebook, and you know, I called her recently on her birthday. And I said, she said, You know, I don’t know how to get back on or what to do. And, and I said, Well, let me take a, I’ll take a, you know, at least one stab at trying to get a better and she goes, she said to me, that’s the best birthday present I’ve had in a long time. And I said, Well, I don’t even know if I’ll be able to get it to work. And she well just someone trying my point being that that technology that it’s just very kind of low, low tech at this point, is something that she really relies on and likes and and I think a lot of these things, you know, I’m looking at one of the pictures from that article and you know, it shows some wearables that are really not like we’re not talking about full time getting your blood pressure checked every two minutes like when you’re in the hospital, but hey, are you moving around enough? Like we because if you fall in maybe we could detect that fall, either via a camera or something that you have on you, even on your watch can tell us so if you know have you taken a violent fall, are you are you moving the way you typically would? Those kinds of things and so, you know, a wearable is not just a watch or a ring It can be shoes or socks or, you know, motion detector that’s out there. And so yeah, there’s lots of things to both kind of find problems before they find little problems before they come big problems. And also to let you I think, as you started off this conversation, Hey, why do you need to be in the hospital? Or why do you need to be in a residential care facility? No one, as you mentioned, no one wants to be in those places generally. And even if they’re great, you still don’t want to be there, you want to be in your home. And I think we can do to kind of get you there with the same or even better, quality outcomes would be as well appreciated.
Nick van Terheyden
I clicked my red heels all the time, there’s no place like home. i It’s and you know, having had, you know, my intersection with the facilities and you know, in a space that I actually perform the procedures involved, where it was a 15 day stay. And I was out same day was just and it was, I could not have been more grateful to be at home. And I was doing more work. I just to be clear, I was hooked up to a monitor, but they got tired of the beep or actually, I got tired of the beep beep So they just unplugged it. So what was the point where as I got, you know, constant monitoring, I’m really excited by sort of continuous blood pressure. I think the important point you brought out there that’s, you know, worth highlighting is this, you know, designed for the purpose. And you know, with the elderly in mind, they’re absolutely not technophobic. My mother was the same when she was alive. She was absolutely she jumped on this stuff. She the iPhone, and Siri, by the way, was a huge positive. You know, it achieved so many things. And a lot of it, you know, one of the simplest things I learned around that time was, you know, those little pens with the little rubber thing me, Bob to touch the screen, proved to be a huge positive for interacting with the screens, because she couldn’t do it very well with a finger didn’t have the dexterity, but you put it in a pen, and suddenly it was much, much easier, simple thing, but you know, give her that. And then suddenly, she was I think she would have had a watch if she’d been around for that period of time. So anyway, let’s talk briefly if we can about the epic research, they said, Nope. bloat. So, you know, you and I have seen that that’s a like a huge problem copy forward epics. I think we both love the epic research, lots of data to sort of dive into and people are doing some interesting things. Any thoughts?
Craig Joseph
Yeah, you know, I think you mentioned open notes before, that’s the movement to allow patients to see their progress notes, be it inpatient, or outpatient, make it easy to do over the Internet through the patient portal. And when people are looking at them, they’re seeing these these Tomes, these four or five, six page notes, when typically when we wrote them, they were maybe 789 10 lines. And why is that happened? There’s multiple reasons. But one of the main reasons was that there were rules in the United States about how you got paid to prove the work that you were doing, you had to jump through hoops, you had a checkboxes, you have to do lots of things, and physicians have become used to writing these big things. And then two years ago, CMS, the, the organization that’s responsible for some of these rules, said, you know, what, we’re gonna make it a lot easier. And they did. And they, they really changed the focus. So it’s a lot less of those boxes that you have to check and words that you don’t really need that don’t clinically say anything. And we were all very excited, because we’re like, Okay, now we’re gonna get notes back to succinct and the way they were when we’re on paper, and epics research, this is a, they were able to look at 1.7 billion clinical notes, that’s B, that’s B, like Bravo, wow. 1.7 clinical 1.7 billion progress notes in their electronic health records of their customers, and did research on those and found that, well, the actually, the notes didn’t get shorter in that timeframe. In the last few years, they actually got 8% longer. Oh, gosh. So that’s not good. But they did find though, that on average, there was an 11% decrease in the amount of time doctors took to write the notes. So I’m not sure why that is.
Nick van Terheyden
Oh, come on, copy forward, or whatever you want.
Craig Joseph
But they’ve been able to do that for for for years and years. This isn’t the last two years why did it get faster but under any circumstance? I’m not sure it’s so they’re spending less time but I’m not sure we’re producing a more a higher quality product. But again, this is you know, it’s quite interesting because, no, we would never be able to do a study with right when we said you know, when you said you’re gonna look at a billion notes cuz like that’s insane and
Nick van Terheyden
it’s really cool. Yeah. And you know love, love the fact that it’s being published people can see this I think you know great stuff. Unfortunately as we do each and every week we’ve run out of time. Always a great conversation. Always good to catch up. Just remains for me to thank you, Craig, for joining me on the show, Greg. Thanks for joining me.
Craig Joseph
Always a pleasure. I can’t wait till next month.
