CMT Blog

Can AI help diagnose rare diseases like Charcot Marie Tooth and other polyneuropathies?

I recently fed a popular AI tool a bunch of my medical records to see what I might learn from the results. I had no expectations, but I was still impressed. I had already been using AI for research, creative, and technical work — and to better understand its usefulness and limitations. (I’ve written about AI in relation to writing, marketing, WordPress and the future of the web.) As it turned out, ChatGPT’s interpretations of my medical data and diagnostic directions were clearer and much easier to interact with than most of the doctors who provided them.

Unlike two of the doctors I’ve dealth with in relation to CMT or a possible CMT diagnosis, ChatGPT didn’t provoke anxiety or put me down. What I’ve learned from it has been much more helpful, on reflection, in my own rare disease journey. AI seems like it could be a more objective corrective and complement to emotionally charged situations where you may have only a brief moment to ask frequently evasive doctors your questions in the face of all the things you don’t know.

What is ChatGPT?

You probably know by now that ChatGPT is a Chatbot you can talk to like you might converse with another person — a person with a lot of knowledge and infinite attention, availability, and patience. Behind this chatbot is a Large Learning Model (LLM), currently GPT-4. GPT-4 has absorbed a massive amount of pre-September 2021 material from the internet. It’s passed a load of standard tests for university and post-graduate admissions as well as the licensing exams for lawyers and doctors. It’s impressive, and it’s what I used to review my own medical diagnostics.

ChatGPT still gets a lot of things wrong, including simple and silly things. It’s controversial, but it’s also very, very useful and compelling as a tool for completing simple and hard technical tasks. It’s promising as an aid to human learning, a virtual research assistant, and maybe even a partner for thinking.

Don’t believe the hype? There’s been a lot of it about “artificial intelligence” since OpenAI made ChatGPT publicly available. Headlines like these: The newest version of ChatGPT passed the US medical licensing exam with flying colors — and diagnosed a 1 in 100,000 condition in seconds and GPT-4 Saved My Dog’s Life! are highly optimistic and sensationalist. But even this New York Times article acknowledged ChatGPT’s potential as a medical tool. And in this scientific study, even GPT-3 (an older version) performed well as a medical diagnostician up against real doctors (who did better) in the context of common illnesses. On the other hand, here’s an ER doctor who thinks AI tech in the ER would be deadly. I myself am not a technophile or only reluctantly. I tend to be a late adopter. I am as archly critical as an anarchist or classic Luddite of the modern institutional forms of everything — from politics and religion to science and medicine. At scale, technology (particularly under the pressure of the profit motive) always depersonalizes, damages, harms, and even kills — at scale. And yet this is the world we are in, the professions we’ve chosen. Might as well be curious — and critical — about the good that may be possible.

This is a long one, so here’s a table of contents:

What is ChatGPT good for?

I’ve been using ChatGPT for work and for fun. I’ve found it useful but very limited in what it can contribute to research and writing tasks or technical problems. It seems to generate the most helpful and accurate results under the following conditions:

• I understand the topic well enough to form a clear, accurate, and precise statement of the problem or question. I could probably produce my own answer, but it would take me much longer. I am confident I can verify or correct the AI response to my prompt.
• The question I am asking is focused, not broad or open-ended. It’s answerable based on public knowledge that’s well-established or at least several years old. If the question calls for some creativity, it must also have very strict constraints. Or, the topic lies within a well-defined technical field where answers are clearly right or wrong; they work or they don’t work.

ChatGPT didn’t glare at me and tell me I must be faking something (until it saw my feet) because I could walk into the EMG lab. ChatGPT didn’t go on and on about how a genetic test might generate medical records disqualifying me from private insurance, especially in the US.

In essence, you must be the expert, and the AI must remain an assistant that is “unreliably competent” (sometimes helpful, sometimes not) outside very well-defined and well-understood subjects. It even seems to stumble into creativity while writing (mostly bad, dad) jokes and humorous poems with a strict rhyme scheme. It can’t do free verse or blank verse without mimicking a human model (in obviously derivative ways) so it’s good at doggerel, pastiche, and embarrassingly moralistic or ideological “creative” work. As ChatGPT will tell you, it has no emotions or experience to draw on. It’s about erotic as a rock, and there are probably censors and filters intent on keeping it that way. It is the not-hot librarian or stereotypical professor, and as such it has a lot to offer.

ChatGPT as a research assistant or learning tool

GPT-4 is much better than previous models at generative writing, but if you’re not in a position to know if it is making inaccurate claims, it’s foolish to believe them. GPT-4 is not a reliable authority or expert on anything. It does not think or reason — it mimics these aspects of the human mind. It predicts the most plausible response to your prompts by drawing on a massive amount of human-authored source material. That material comes with human biases and blind spots. However, by researching and fact-checking every claim it makes, or by asking it to cite its sources and then reading them yourself, you can help ChatGPT help you learn.

ChatGPT-4 is good at explaining where it is getting answers from, especially in technical areas where it can “show you the math” in each step. If you position yourself as a learner using ChatGPT as a learning tool, you should not treat it like a textbook but as an uncorrected manuscript written by a generally knowledgeable person with dementia or an average student who sometimes plagiarizes and bullshits their way through an essay. You will learn only by finding the nonsense and errors.

What is Chat GPT not good for?

Normally, ChatGPT is a good summarizer of topics and specific articles, but if they include a lot of nuanced distinctions and expert knowledge, I’ve found that ChatGPT does a poor job. Scientific and technical fields are less likely to fit this description, so it’s not surprising ChatGPT does a better job in those areas. It is very much a left-brain hemisphere without much of a right hemisphere. When it comes to asking ChatGPT to generate functional software code, regular expressions, or spreadsheet formulas, ChatGPT performs very well, but as complexity increases, so does its apparent ability. It’s better if you break large tasks or problems down into smaller parts because at some point the AI loses sight of the big picture and begins “hallucinating.” Or it may not remain consistent with your overarching goals.

How is ChatGPT useful for medical diagnostics?

Because it’s absorbed a lot of medical literature where diagnostic pathways are often well-defined, GPT4 can produce what might be called an “orthodox” or standard account of how diseases like CMT (the main topic of this blog) typically ought to be diagnosed. There’s not much disagreement about it. But if that process has brought you to a dead end, you either have an undiscovered type of CMT or something else that may or may not be known in medical science. (That’s my situation.)

Adding to the challenge, CMT itself is a hazy, maybe even increasingly controversial category among the many types of neuropathic diseases and disorders, which are often not well understood.

Obviously we can’t ask anyone answerable questions about things that are undiscovered or unknown unknowns. But “something else” — a known unknown, or an unknown known is a possible path to explore. What other diseases, disorders, or conditions exist that might be mistaken for CMT (especially CMT2), that might present similar symptoms and test results? There’s existing medical research literature on this type of question, and if we exclude CMT as a possible diagnostic result, ChatGPT can match symptoms with the next most likely known diseases.

This type of diagnostic reasoning is simply a process of elimination. That’s how diagnostics tend to work in every field. Test for the most likely cause of a problem. Then the next most likely. What this process cannot do is challenge underlying assumptions (e.g., the test data is valid) or reframe the problem to approach it in a different way (e.g., consider artificial or environmental rather than genetic causes). And it cannot make a new discovery. That would take a human expert creatively going outside the existing rules and assumptions.

ChatGPT as a Medical Diagnostician

What happens if you ask ChatGPT to provide a range of possible diagnoses for a quick list of symptoms?

Let’s see!

This is actually a pretty bad prompt, especially since it omits the age of the patient relative to the onset of symptoms:

Provide differential diagnoses for a patient with pes cavus, distal muscle loss, and reduced (39 m/s) sensory as well as motor nerve conduction velocities in the peroneal area. Unusually, the patient has no family members reporting similar symptoms.

First of all, you get a standard disclaimer:

I’m not a doctor, but I can try to provide you with some information on potential differential diagnoses for the symptoms you described. Keep in mind that only a medical professional can accurately diagnose a patient.

Then we continue:

This is a pretty good response — CMT comes first, which is what most doctors would conclude if they know about CMT, simply because the foot deformity is so commonly associated with it.

Notably, GPT-4 thinks CMT is less likely in this case because I told it nerve conduction isn’t (highly) impaired, which indicates predominantly axonal (or simply non-demyelinating) damage to nerves or no damage at all. Low conduction velocity indicates demyelination which is associated with the most common (70-80%) and best known types of CMT in the CMT1 category. A patient with my clinical details is normally assumed to be exhibiting a subtype of “axonal” CMT or CMT2, which accounts for 12% to 36% of all CMT cases, according to the MDA. ChatGPT seems to understand axonal CMT with moderately impaired nerve conduction velocities is rarer, and perfectly normal nerve conduction would be very rare (but not unheard of) in CMT cases.

But then again, do we really know how rare the types and subtypes of a rare disease are — within specific populations? 12% to 36% is a huge range — why? Since MDA footnoted that statistical range and its source, I could look it up: a 2016 international (European) study of fewer than 1,000 CMT patients that found the incidence of CMT1 vs. CMT2 patients varied greatly from country to country. Perhaps we can’t generalize about how common a type is without more patient data in many global populations. Perhaps CMT1 is overrepresented in the literature because it can be genetically verified and has clear clinical presentations. Mild axonal motor neuropathy can be ignored, hidden, and evade detection altogether — at least for many years. Even decades. Such is my experience.

When I asked ChatGPT for more diagnostic possibilities (other than CMT) stressing axonal damage, I got another reasonable list promoting CIDP to the top:

1. Chronic inflammatory demyelinating polyneuropathy (CIDP): CIDP is an autoimmune disorder that affects peripheral nerves and can cause muscle weakness, sensory disturbances, and atrophy. It primarily involves demyelination, but axonal damage can occur as well.
2. Hereditary sensory and autonomic neuropathy (HSAN): HSAN is a group of rare genetic disorders that primarily affect sensory and autonomic nerves. Some subtypes of HSAN might present with axonal damage, muscle atrophy, and sensory disturbances.
3. Acquired peripheral neuropathies: Various acquired peripheral neuropathies can lead to axonal damage and symptoms similar to CMT2. Examples include diabetic neuropathy, alcoholic neuropathy, and neuropathies related to nutritional deficiencies or toxins.
4. Amyotrophic lateral sclerosis (ALS): ALS is a progressive neurodegenerative disorder affecting motor neurons, leading to muscle weakness, atrophy, and eventually respiratory failure. While ALS typically affects both upper and lower motor neurons, some cases might present with predominantly axonal damage in the early stages.
5. Spinal muscular atrophy (SMA): SMA is a group of inherited neuromuscular disorders characterized by progressive muscle weakness and atrophy. Some forms of SMA might present with mainly axonal damage in motor nerves.
6. Guillain-Barré syndrome (GBS): GBS is an autoimmune disorder that affects peripheral nerves and can cause rapid onset muscle weakness and sensory disturbances. Although the classic form of GBS is associated with demyelination, axonal forms of GBS (such as acute motor axonal neuropathy or AMAN) can also occur.

I’ve never had a doctor openly go down the checklist of possibilities with me given the symptoms I have — so I haven’t been very confident about my findings via Google. SMA, HSAN, and CIDP are diseases I’ve come across in my own research for alternatives to a CMT2 diagnosis, however, so it’s good (but not surprising) to see ChatGPT is consistent with Dr. Google.

Ask for more and you’ll get more

Going back to the first list of alternative diagnoses, the final two answers are straight from the textbooks and very general ones. I asked ChatGPT what the “other” category could include. I got this:

1. Distal hereditary motor neuropathies (dHMN): dHMN is a group of rare genetic disorders characterized by progressive distal muscle weakness and atrophy, mainly affecting the lower limbs. Some subtypes of dHMN might present with pes cavus and normal nerve conduction velocities.
2. Hereditary spastic paraplegia (HSP): HSP is a group of inherited neurological disorders characterized by progressive stiffness and weakness in the lower limbs. Some forms of HSP might present with muscle atrophy, pes cavus, and distal motor impairment.
3. Giant axonal neuropathy (GAN): GAN is a rare genetic disorder characterized by progressive nerve dysfunction, leading to muscle weakness, sensory loss, and central nervous system involvement. GAN might present with muscle atrophy and pes cavus, but it typically affects both the peripheral and central nervous systems.
4. Congenital muscular dystrophies (CMD): CMD is a group of inherited muscle disorders characterized by muscle weakness and hypotonia from birth or early infancy. Some forms of CMD might present with distal muscle involvement and pes cavus.
5. Various forms of muscular dystrophy: Some less common forms of muscular dystrophy, such as distal muscular dystrophy or limb-girdle muscular dystrophy, might present with distal muscle weakness and atrophy, along with pes cavus.
6. Metabolic or mitochondrial disorders: Some inherited metabolic or mitochondrial disorders can affect the nervous system and cause symptoms like muscle weakness, atrophy, and pes cavus.

HSP, GAN, CMD, etc. are all interesting new paths to consider too. I’m more familiar with dHMN, so I recognize ChatGPT has run into some limitations due to the contradictions in a changing medical field. dHMN is very difficult to distinguish from CMT2 and a third disease, distal spinal muscular atrophy (DSMA), which is now commonly lumped in as a type of dHMN. Some clinicians believe CMT2 and dHMN (and DSMA) should be considered the same thing since they can be caused by the same pathogenic genetic variants. ChatGPT disagrees. (More on that below.)

What does ChatGPT know about diagnostic tests?

At one point, I asked ChatGPT what tests might help pin down a definite diagnosis. These are correct and pretty standard:

1. Electromyography (EMG): This test measures the electrical activity of muscles and can help detect abnormalities in muscle function and nerve-to-muscle communication.
2. Nerve conduction studies (NCS): Although you mentioned the nerve conduction velocity is normal, further nerve conduction studies might be helpful to evaluate other aspects of nerve function, such as latency and amplitude.
3. Blood tests: Blood tests can be used to assess for signs of inflammation, autoimmunity, vitamin deficiencies, metabolic abnormalities, or infections that could contribute to the patient’s symptoms.
4. Genetic testing: Genetic testing can be performed to identify mutations associated with inherited neuromuscular disorders, such as Charcot-Marie-Tooth disease, spinal muscular atrophy, or hereditary sensory and autonomic neuropathy.
5. Cerebrospinal fluid (CSF) analysis: A lumbar puncture can be performed to collect a sample of CSF, which can be analyzed for signs of inflammation, infection, or other abnormalities.
6. MRI or CT scan: Imaging studies of the spine, muscles, or other relevant areas can help identify structural abnormalities, tumors, or other potential causes of the patient’s symptoms.
7. Muscle or nerve biopsy: In some cases, a small sample of muscle or nerve tissue may be removed and examined under a microscope to identify any pathological changes.
8. Quantitative sensory testing (QST): This test can help evaluate sensory nerve function and detect any abnormalities related to sensory neuropathy.

Even more useful is how ChatGPT can interpret data produced by these tests! (More on that below too.)

How does ChatGPT respond to very specific CMT Questions?

I gave GPT-4 the results of a physical exam and the first of three nerve conduction velocity tests I’ve had. I had ChatGPT interpret a data table by itself, which it did perfectly. I also included the neurologist’s summary of physical and NCV test results but not his overall diagnosis.

ChatGPT can interpret and explain test results

I asked ChatGPT to interpret this information separately, which it did correctly and in detail. It noted the implications of each data point and observation. For example, “The unobtainable right ankle jerk may indicate some neurological impairment affecting the lower extremities.” I had to ask for a specific diagnosis, and it came up with the following:

1. Charcot-Marie-Tooth disease (CMT) – A hereditary neuropathy that affects the peripheral nerves, leading to muscle weakness, atrophy, and sensory loss.
2. Hereditary sensory and autonomic neuropathy (HSAN) – A group of rare genetic disorders characterized by sensory and autonomic nerve dysfunction.
3. Peripheral neuropathy due to various causes, such as diabetes, alcoholism, nutritional deficiencies, or toxic exposures.

ChatGPT also provided guidance about the standard tests that would eliminate #3 and possibly #1 and/or #2 as possibilities. The neurologist who ran the original NCV tests also did that. His tentative diagnosis of “axonal polyneuropathy such as Charcot-Marie-Tooth Type 2, or other motor neuropathy” was basically the same one ChatGPT came up with based on that neurologist’s report. However, I’ve never had a second possibility proposed, and HSAN never entered my awareness until I had results from the Invitae Comprehensive Neuropathies Panel, which covered 102 genes at that time. (Now it includes 111.) More on that below.

ChatGPT can include or ignore certain data points. Can this helpfully reveal human biases passed on to machines?

The classic pes cavus foot description is probably still the most decisive symptom for a CMT diagnosis — probably because humans are fixated on things like physical deformities. This is a problematic bias for rare disease research.

When I asked ChatGPT to consider the nerve conduction data by itself as the basis for a diagnosis it was not more specific than “a peripheral neuropathy or neuromuscular disorder.” But when it took the clinical exam into account too (and when it considered the clinical exam by itself) CMT and HSAN were the top picks and diagnoses.

This tells me the clinical exam and probably the classic pes cavus foot description is most decisive for a CMT diagnosis, as it always has been — probably because humans are fixated on things like physical deformities. I suspect this is a problematic bias for rare disease research.

The same physical outcomes can be rooted in different genetic causes or other pathogenic mechanisms. As well, a genetic anomaly can present very different physical outcomes in different people, even within the same family due to a concept ChatGPT introduced me to — incomplete penetrance. Here’s a study that mentions two brothers diagnosed with CMT2, one of whom exhibited pes cavus or “high arches” and one who did not. (We’ll come back to this.)

Genetic data outweighs other diagnostic paths

If ChatGPT has a diagnostic bias, it’s for genetic data — as it should. So is the medical science-industrial complex. By simply adding two genetic anomalies to the clinical and NCV exam data I had already provided, I made ChatGPT completely alter its diagnosis. This may reflect the weight currently put on genetics for diagnostics and rare disease research. It may also reflect the connection between both of the genes I mentioned and HSAN, which ChatGPT had already included as a possible alternative diagnosis to CMT. That’s a significant commonality that wasn’t emphasized in Invitae’s report or follow-up.

Given the full picture (or at least the most salient details) of my painfully slow diagnostic path over the past 5-6 years, ChatGPT came up with the following non-CMT suggestion:

This is actually very similar to the findings Invitae reported on my genetic test, but it did not specifically name Hereditary Sensory and Autonomic Neuropathy (HSAN). Instead, it mentioned HSN, a closely related disease category — HSAN without the ‘A’ — which is also considered a type of CMT2 by some researchers. Specifically, my genetic report and follow-up named HSN1E/HSAN1E and HSN2C, which I clearly do not have in their typical, documented presentations, but they are related to CMT as types of neuropathies with some overlapping genetic variants. In fact, the naming is confusing because HSAN and HSN are types of the same (not terribly well-defined or understood) disease category.

“Given enough eyes, all bugs are trivial” is an idea that drives open-source software. It also drives the sharing and open-sourcing of genetic data. I’m pretty sure this is one of my contributions to the human genome bug tracker. It’s anonymized of course, but now ChatGPT and everybody knows about it. Privacy concerns are a major issue for Artificial Intelligence that harvests and retains the things we tell it and publish online. I just don’t happen to care when it comes to my own data because you can’t easily learn about a thing you’re trying to keep secret.

Naming is hard — in every field

I asked ChatGPT about the naming issue — whether HSN is the same as HSAN — and it confirmed this:

Yes, HSAN (Hereditary Sensory and Autonomic Neuropathy) and HSN (Hereditary Sensory Neuropathy) are often used interchangeably to describe the same group of rare genetic disorders characterized by sensory and, in some cases, autonomic nerve dysfunction. These disorders primarily affect the peripheral sensory nerves and can lead to varying degrees of sensory loss, pain insensitivity, and other symptoms depending on the specific type of HSAN/HSN. There are several subtypes of HSAN/HSN, each with distinct genetic causes and clinical presentations.

However, ChatGPT denied HSAN/HSN are CMT2:

It looks like ChatGPT may take a conservative view and rely on the most established or “standard” literature with some more recent contributions. For human academics, this is usually the prescribed and safe middle course — until revisionist research accumulates and the prevailing orthodoxy is revised.

These disagreements over gray areas in every field are like faults between tectonic plates — small rumblings over a long time and then one day all at once the ground shifts. It’s the most interesting and delicate point in a community, institution, or collective knowledge archive.

So I thought I’d explore that a little more.

ChatGPT takes sides in controversies among the experts

When I asked for elaboration on a specific CMT type, ChatGPT explained why “axonal” CMT2 was most likely given the NCV data I had given it. This would be the standard interpretation 10-20 years ago when 38m/s was the line drawn between “demyelinating” (<38m/s) and “axonal” (>38m/s) CMT.

So what if you have a result of 38m/s — right on the line — or like the exam data I gave ChatGPT, 39m/s?

Today, the cutoff clinicians use is <35m/s (or even <25m/s) and <45m/s with a 25-45 m/s or 35-45 m/s intermediate threshold for a third category of “Dominant Intermediate CMT” (DI-CMT). (Some researchers think this is a special type of CMT2.) This intermediate category was created after research showed families with the same type of CMT with the same genetic cause could have very different NCV results. It also opened up the door to further research indicating there is a mix of demyelination and axonal damage in nearly all types of CMT.

When authoritative sources disagree, what’s an AI to do?

When I queried ChatGPT further, it insisted CMT2, DSMA, and dHMN are distinct diseases (it also used “disorders”), but then it noted DSMA is seldom used as a term now — it’s often included in dHMN. When I asked, “Do CMT2 and dHMN have common genetic causes?” I got a similar reaction. ChatGPT insisted they are distinct diseases even though they can have the same genetic causes:

This reasonably reflects the contradictions and debates in the medical literature at this point in time, but if you don’t know that (and even if you do!) it is confusing. The clarifying paragraph ChatGPT adds is a very helpful and important reminder — the same genetic mutation can lead to very different physical outcomes.

What did I get out of this?

I got a lot out of this interaction with ChatGPT. I reread (more carefully) my medical records. I was reminded and learned more about some of the diagnostic gray areas. I got a sense that most of my Dr. Googling over the years has been effective at surfacing most of the possible alternative diagnoses to CMT. I discovered some new things in the big, changing world of neuropathic disease and genetics research and got some new topics to explore: HSAN, HSAN2, HSP, ADCA-DN, and other rare, unknown, or little-understood disorders. Chat-GPT offered this kind of specificity that doctors generally avoid but patients want. However, nothing Chat-GPT produced in response to my questions resembled anything like a high confidence diagnosis.

None of these diseases in their textbook forms (probably biased toward the most extreme cases) are strong matches for me. My answer probably lies in the realm of unknown unknowns that may or may not become known tomorrow. Most of the conditions I’ve mentioned here touch something recognizable, especially as age and stress seem to have contributed to a big falling off in function and increasing fatigue. It’s a stressful reminder that stress can activate genetic conditions that have been quiet previously.

It’s impressive that ChatGPT can do in seconds for me what I didn’t quite get after six years with a bunch of useless MDs, three neurologists, and a CMT-informed genetic counselor. They’re actually in total agreement, but it’s the machine that was willing to be clear and concise, name alternative diagnoses, and explain them — answer any question, any time.

There’s also this — the category of humans doctors include a fair share of people whose bedside manners are shit or they’re just straight-up assholes. That can land hard and really damage things with patients. Why should they be the keeper’s of the keys to diagnostic science? Why should we even have to visit them in person, strip down, get poked and prodded, etc. just to get a consult?

ChatGPT didn’t glare at me and tell me I must be faking something (until it saw my feet) because I could walk into the EMG lab.

ChatGPT didn’t go on and on about how a genetic test might generate medical records disqualifying me from private insurance, especially in the US. (That led me to avoid genetic screening for years, and other potentially family members won’t even talk about it.)

ChatGPT didn’t have me answer very personal life questions to a resident and then interview the resident in front of me, referring to me in the third person, while I sat there in a smock.

ChatGPT might have been a huge difference maker in my childhood through middle-adulthood when a partner and family members ignored, minimized, and generally denied the existence of any real problem at all. (See Disease, Depression, and Family Denial.)

ChatGPT doesn’t experience stress or burnout. I don’t think it can develop OCD and borderline or narcissistic personality disorders. It will operate the same under our failing provincial and national healthcare systems no matter how bad they get. That’s not awesome, but it’s not a bad thing — not bad at all.