The misdiagnosis of our son in utero is an excellent illustration of a fundamental flaw at the heart of medical science - the false analogy between people and sets of numbers

A wrong diagnosis: our story 

‘What is the chance our child will be normal?’
‘Hmmm … a very low probability.”
             - conversation with Dr O.

Intense emotion often sears the memory with incidental details. Two years later, I can still remember every inch of Dr M.’s office. His chair sat under a window that had regal views over Hyde Park; his desk was an enormous, darkwood affair that contrasted with the compactness of the Mac computer that lived  on top of it. There was a large, garish piece of modern art on the right hand wall, showing a Picasso-esque mother holding a baby; a small bookshelf that held stacks of paper and a variety of medical supplies and gloves; and in the far corner of the room behind us, an adjustable bed and a small ultrasound machine with its display screen.

Dr M. was our obstetrician, and he had a reputation as one of Sydney’s best. His fees – a flat fee of a few thousands, and hundred dollars each for consultations that often took less than five minutes – seemed to confirm that. Like most first-time parents, we felt that we were doing the right thing by seeing him. By having a specialist of our choice involved in the pregnancy, we were offering our new child the best of medical care. We were vaguely aware of the alternative – going through the public health system and seeing whichever doctor happened to be free at the time – but as people with some disposable income, it seemed a no-brainer. We had the money so we would “go private” and see our own doctor.  

Dr M. used two offices, so that he could line up patients in quick succession and waste no time waiting in between appointments. Our consultations with him were usually quick and pleasant. A simple scan to check the growing baby, then general enquiries about my wife’s health and a promise to see us again in a few weeks. 

But in the fifth month of the pregnancy, something went wrong.

 

Arjun in the womb at 22 weeks

It all began with the 22-week scan, a standard, detailed ultrasound examination of the fetus that is now a routine part of prenatal care. While we were mesmerised and delighted by the pictures of our (literally) bouncing little baby boy, our sonographer seemed more circumspect. She waited until the very end of the examination to tell us that something had “not measured right”. We had noticed her using a computer cursor during the examination to measure different parts of the baby’s body – bone lengths, head circumference, size of organs and so on – but now we suddenly paid a lot more attention. What did she mean not “right”? What had she seen? Was there something wrong with our baby?

“It is probably best,” she told us as she ushered us out, “to see your doctor.”

We left the examination in a daze. We had tickets to the Caribbean the next day for a four-week holiday; we knew that there was no chance of us getting an appointment with Dr M. before that. As we drove home, our anxiety grew as we began to feed off each other’s apprehensions. What made it worse was not knowing any details. It seemed stupid that we had not asked the sonographer for more explanation, but it had not occurred to us at the time. The shock of hearing that all may not be well with the baby had temporarily paralyzed our thought processes. Now, by the time we had recovered from the initial shock, it was far too late. We couldn’t see our doctor, and it would be four weeks before we would next get a chance.

We flew to the Caribbean with a bittersweet taste in our mouths. We convinced each other that is was probably nothing, and that worrying would not get us anywhere. But  at the airport, we used international roaming on our mobile phone to call Dr M. There was no answer, and we left a message. Every new country where we disembarked, I would turn on the phone and watch the little screen in hope, waiting for a return message. In the plane, my wife’s little bump suddenly had a significance and a poignancy that it had not had before. 

Finally, in Miami, the message came. The baby’s long bones [arms and legs], Dr M. told us, were a little short. No big deal right now; come back next month and we would review it then.

It was enough, this little nugget of reassurance bounced to us on the other side of the world. We enjoyed our holiday in relative peace, reassuring each other that it was all a false alarm. We watched cricket, drank rum and lay on the beach. On a boat cruise, fellow tourists exclaimed at how small my wife was for a second-trimester pregnancy.  I told myself it meant nothing – she was a small girl anyway, so what did they expect? The sun was shining and all was well with the world. We lay at night in the tropical dark and pretended to each other that we weren’t thinking about it. 

But we were. And in a spare moment of weakness, I logged onto the Web at our resort and searched Google for “short long bones fetus” - and stumbled onto a chamber of horrors.

Skeletal dysplasias are abnormalities of bone growth that typically lead to what is commonly termed “dwarfism”. Although the most common and milder forms such as achondroplasia are often compatible with an active and long life, many of the more severe forms are very disabling and lead to death at or shortly after birth. It turns out that abnormally short long bones in the fetus are a prime prenatal marker for skeletal dysplasias. Everything I read pointed to the same conclusion: that these results at this stage of the pregnancy put our baby at real risk of having a form of skeletal dysplasia. As I scrolled through pages and pages of diagrams and graphs, research papers and photographs of crippled children, it really hit me for the first time that there was a possibility that my boy would be very visibly abnormal. That all the dreams I had about his first steps, playing cricket in the park and walking him to school would have to be shattered and rebuilt. That the future that once lay bright and undisturbed in front of us now bubbled with uncertainty and fear. 

We returned to Sydney, and went to see Dr M. as soon as we could get an appointment. I was relieved to see that he seemed unfussed. Google had not lied to me; there was a possibility of skeletal dysplasia, but at this stage it was nothing more than a suspicion, he told us. The only way to know was to follow the baby’s growth over the next few months. If his little arms and legs grew at a decent pace, we could be reassured that everything might be okay. If, however, we saw the growth dropping off the graph, then there was some cause for concern.

The  next few months were a  merry-go-round of medical appointments. There were bi-weekly meetings with Dr M., and ultrasounds scans both at his office and at the ultrasound center that did more detailed scans.  Time dulled our initial anxieties, and we went into a kind of holding pattern of hope and fear. Our appointments were as jovial as ever, but the banter was now forced. I watched the ultrasound screen with feigned disinterest, waiting for the new measurements to appear every week and mentally calculating the rate of growth as soon as it appeared. Our baby was reduced to points on a graph and a percentile reading on a growth chart. And though I tried to fight it, at nights I would sometimes pore over our ultrasound pictures and surf the Web for information, looking for reassurance but also afraid of what I might read.

Things came to a head in late July, just a month before our baby was due. Dr M. had danced around the truth for months, but now he told us straight out: our baby had a skeletal dysplasia, and he was referring us to a geneticist to start the process of “management”. The cubist art beside us now took on a sinister air; the doctor put on a professionally compassionate face; and in that moment our carefully constructed defences crashed down around us. In the waiting room as we paid our consultation bill, my wife wept openly and drew sympathetic stares. This was what we had desperately hoped to avoid. For the first time, we felt stripped bare and alone - and unprepared to face the future. 

My online research became obsessive. What might be the exact diagnosis, given the prenatal measurements that we had? What was the prognosis for our baby’s life? What was the chance that the diagnosis was false? I spent hours on the Web reading everything I could find, even spending money to get access to medical journal papers and specialist material. I began to read up about people with dwarfism and their lives and how they fared. I read about achondroplasia, the most common form of dwarfism, and the medical complications it could bring. I read the blogs of people with dwarfism and saw how normal they were, despite the many hurdles they faced. I looked up the American TV show “Little People, Big World” and laughed along with their stories. 

And meanwhile our little baby kicked and squirmed, ready to come into the world.

The appointment with the geneticist was the most nerve-wracking day of my life. His name was Dr O., and he was an expert in the field. He was the one, we thought, who would give us a definite diagnosis for our baby’s condition. I remember the fetal medicine department as being the darkest, shabbiest, most depressing part of the entire hospital. We sat in the waiting room surrounded by a sea of glum faces, each mother-to-be trying to guess why the others were there. The receptionist laughed and joked with her friends over the phone, oblivious to our private tragedies. Dr O., when he finally came to call us, turned out to be a small, obese man with close-cropped hair and womanly hands. His solicitious tone suggested that he was used to dealing with anxious parents. He ran the ultrasound monitor over my wife’s belly, and I watched with my heart in my throat. The examination couldn’t have taken more than a few minutes, but it felt like hours. My wife gave me a questioning look; I squeezed her hand for reassurance. When I couldn’t take it any longer, I asked Dr O. what he thought. He hummed and hawed and scrutinized the computer monitors.

 ‘It’s hard to say. There is definitely some kind of skeletal dysplasia. But I don’t think it is lethal.’

Oh, how I had hoped for a different reaction! A dismissive wave of the hand, perhaps – ‘Nothing to worry about!’. Or a puzzled look - ‘Why are you even here? He’s normal!’ Normal, normal – those were the words I had so wanted to hear. Instead - he didn’t think it was lethal! To hear that your baby will not die is not something that a parent should take lightly, but I hardly focussed on that. What Dr O. had told us was being measured against our expectations, not against any absolute reality. On our scale of possible outcomes, I could see both the best-case and worst-case scenarios slipping away. And those words – “definitely a skeletal dysplasia” … 

‘What is the chance he might still be normal?’ I asked. Dr O. furrowed his brow in reply, as if calculating the odds there and then.

‘Very low probability,’ he said.

Our baby would not be normal. I tried my best to internalize this new reality. A definite diagnosis would only be possible after birth, and we went home to wait it out. We told our families the news, and although they lived interstate we hid nothing from them. My parents tried to reassure us with anecdotes of short people in our family history; how so-and-so aunt or uncle had hardly reached five feet and still become a success. My father-in-law railed at the “know-nothing” doctors and their new-fangled machines. They all booked tickets and flew to Sydney to be with us. Like a lotus flower, our family closed itself protectively around us. 

And then in late August, our son arrived.

He was in a rush – it was a two-hour labour. Things progressed so fast that there was no time to hook up the monitoring equipment that Dr M. had recommended be used. Little Arjun appeared and was whisked off to have his lungs cleared. I was thrilled and thankful. Under the bright lights of the labour ward, I looked into his gorgeous blue and brown eyes and almost forgot all the dramas of the last few weeks. 

Almost - but not entirely. I couldn’t help but notice his limbs, which were short; but not unusually so. A little flame of hope that I had kept kindled inside burst alight. When Dr M. arrived he was, despite his thousand dollar fees, more than half an hour late.

‘What do you think?’ I asked him.

‘I think you have a healthy baby boy,’ he said – ‘enjoy him.’

We did. There were some minor complications over the next few weeks – many of them a result of over-officious medical care – but it gradually became clear that the prenatal diagnosis of skeletal dysplasia was wrong. Dr O. came for a look in Arjun’s second week. He held our baby at an arm’s length, inspecting him as dispassionately as a meat inspector at an abattoir. Nothing seemed obviously wrong, he admitted grudgingly. That’s as close as his kind would get, I suspected, to admitting he is wrong. 

In September we were finally discharged from hospital. Putting Arjun into the car for his first trip home brought with it an exquisite feeling of relief. It was as if by physically removing him from the hospital was to finally extricate him – and all of us – from an enormous machine that had held us all captive for so long.

I still believe that feeling was accurate, for that is what we had done.

We would have loved Arjun, no matter what shape his skeleton or how short his stature might have been. We know that now. But I won’t deny the stress, worry and fear that I went through with my wife as a result of his misdiagnosis. For those parents who go through the same thing with a less fortunate outcome, I can only express empathy and an infinite admiration. And the memory of our ordeal gives us a reason – as if we need yet another, in addition to the little man himself – to thank God for the gift that He has chosen us to receive.

Measuring the unborn child

At this point it is worthwhile going over Arjun’s diagnosis in some detail. Not because it is inherently interesting, but because it illustrates some general features of prenatal diagnosis which I believe are fundamentally flawed. Although this discussion is somewhat technical, it is important to go over the details to get a good understanding of what those flaws are.

Arjun’s diagnosis of skeletal dysplasia was based on what is called prenatal biometry. This basically means measuring the dimensions of a fetus’ body. Because the actual lengths of a baby’s body vary according to genetics and stature, measurements are plotted on a standardized curve and often expressed in terms of percentiles. Measurements that fall out of a “normal” range are used  as markers for particular conditions. Doctors also look for gross features, such as malformed bones or organs or certain fetal behaviours. A diagnosis is usually made on the basis of the aggregate of all this information; but in the case of skeletal dysplasias, biometric measurements make up the most important evidence in support of the diagnosis.

Let’s look at a specific example. In Arjun’s 22-week scan, his femur [thigh bone] measured 34.6 mm in length, while the “average” length for a 22-week old fetus is 38.0 mm (more on the calculation of “averages” later). This places Arjun at roughly the 3rd percentile, which means that he is in the bottom 3% of fetuses when ranked by femur length. It is this percentile ranking which is used as a metric rather than the actual measurement. Thus, although Arjun’s femur was only 3.4mm smaller than the “average”, his ranking in the bottom 3rd percentile immediately singled him out as being “of concern”. 

(Statistical note: 3rd percentile puts him almost two standard deviations below the mean (-2SD). Note that because the distributions are approximately normal, mean and median coincide in this case.) 

In general, this is the pattern followed by prenatal testing and diagnosis: if measurements fall sufficiently outside the “normal” range, that in itself is taken as prima facie evidence of a problem.

Once singled out, Arjun was measured at regular intervals throughout gestation and his femur length was plotted on a  graph. Over time, this created a “growth curve” for Arjun that could be compared to the “average” growth curve. Similar graphs were made for the length of the other “long bones” – the humerus [upper arm bone], radius/ulna [lower arm bones] and the tibia/fibia [lower leg bones].

Arjun’s femur length in utero compared to the 50th percentile “average” curve

On all these graphs, Arjun’s growth was below the average and dipped away as he approached birth. In the case of his femur, his measurements stayed below the 3rd percentile and even dropped to the 1st percentile at a later stage. Often, these growth curves are shown with a 95% interval around the mean which is deemed as the “normal” range. Arjun’s long bone measurements fell outside of “normal” strip. At birth, his femur was almost 10mm shorter than what it would be “expected” to be.

These were the graphs that Dr M. and Dr O. looked at. It is on the basis of these graphs that they made the diagnosis of an “unspecified skeletal dysplasia”. There were no other indications of bone deformities apparent. In a nutshell, they were saying: Arjun has a skeletal dysplasia because his long bones are too short for a “normal” baby.

Arjun’s diagnosis of skeletal dysplasia turned out to be wrong. But so what? False diagnoses happen all the time and are acknowledged to be a risk in any diagnostic procedure. In itself, the misdiagnosis may not have any significance at all. Medicine is an imperfect science. It may well have been “one of those things” and nothing more. 

But I don’t believe so. Arjun’s case in itself doesn’t matter, other than being a part of our personal history; but it is a good illustration of something fundamentally flawed in the way statistics are used in medicine. Remember, this was not a case of human error. Throughout the entire process, the doctors involved did not do anything “wrong” – they followed procedure. Any geneticist or obstetrician would have come to similar conclusions when looking at the evidence. 

Nothing “went” wrong. But something “is” wrong with the procedure itself. 

In the next few sections I will explore the two fundamental flaws in the system of prenatal diagnosis – and, more broadly, in much of medical science - that I believe are illustrated by our particular case. The first, relatively minor flaw is the misapplication of statistical testing: using the wrong data and not understanding the limitations of the testing itself. The second and deeper flaw is the mistake that permeates so much of medical science: the false analogy between people and numbers. 

What is “average”?

Growth curves are made by collecting data from a large number of people and then fitting a mathematical formula that best describes a “middle path” through all the data points. The sample of people used is drawn from a “population”. If a sample is large enough, it is assumed that repeating the sampling process on the same population will yield almost the same results. However, different populations can give very different results. And here lies the rub: the growth charts used during prenatal diagnosis are assumed to be universal, but they are not - they are specific to a particular population.

Take the femur length chart shown before. It is based on a sample of over 1000 fetuses taken from a primarily European and American population. Now, it may well be that European and American babies (majority Caucasian in race) measure the same as Indian babies; however it may well be that they don’t. At least anecdotally, Indians feel that their babies are smaller. Is it possible that some of the variation seen in Arjun’s case was due to racial factors?

It turns out to be surprisingly difficult to find out. Femur length charts from an Indian population are not available to me, so as an analogy we can look at the work of Shinozuka who has published  growth curves for Japanese populations. Shinozuka’s femur length chart is follows a significantly lower trajectory  than the  standard chart used in the West. His Japanese babies end up with femurs that are about 4.0 mm shorter than their Western counterparts. In fact, using Shinozuka’s chart Arjun’s measurements fall mostly within the normal range, although they are still well below the average. 

We don’t know if Indian populations are comparable to the Japanese, but it seems likely that they would be. If Arjun’s race had been taken into account, he probably would have been close enough to normal to have caused no concern.

Arjun’s femur length in utero compared to the Shinozuka data

Casting the net too wide

Diagnosis is in many ways a mechanical science - doctors often do no more than follow rules. When Dr M. referred Arjun to a geneticist with a suspicion of skeletal dysplasia, he was following a rule as faithfully as a computer program: look for femur length at or less than the 3rd percentile. By definition, about one in thirty fetuses will fall into this range. Although exact figures are hard to estimate, the birth prevalence of skeletal dysplasia is probably around 1 in 5,000. This means that of all the people who are “suspected” of having the condition, 99.4% of them will not have it!

It’s simple maths, but it makes you wonder. The usefulness of this prenatal marker must be compromised by the large number of people who are suspected without cause (“false positives”). On the other hand, almost all people with skeletal dysplasias do indeed have very short femurs in utero – less than the 3rd percentile. So the question is, should we test a lot of healthy babies for this condition to find the few who do indeed have it? 

In statistical parlance, these concepts have names: “specificity” and “sensitivity” . Specificity measures how good a particular test is at finding positives within a population without catching a whole lot of irrelevant guff. Sensitivity measures how good the test is at finding positives and not missing other positives that might be floating around. In fishing terms, you might say that a harpoon is highly specific but not sensitive, and that a trawler net is sensitive but not specific.

The prenatal procedures for detecting skeletal dysplasias are a trawler net; they have good sensitivity (most skeletal dysplasias will be caught this way) but very bad specificity (most babies who are suspected are free of the condition).  Large numbers of babies (and parents!) are scooped up and run through the mill of testing, but only a very few of them are actually found to have needed intervention. 

In my experience, the doctors involved in prenatal testing do not have a good understanding of these statistical concepts. They operate as if their testing procedures had both high sensitivity and high specificity. But testing involves a trade-off between these two measures, and usually both cannot be high. The truth is, almost all prenatal testing has good sensitivity but low specificity. They are often good at finding those babies with problems; but they also catch a lot of false positives and put a lot of parents through a lot of stress for nothing.  This is something that people know anecdotally, even if they do not understand the technical reason.

It’s not surprising that the medical profession is willing to sacrifice specificity for sensitivity. In this litigious age, the potential “cost” of a false negative is far greater than that of a false positive. The worst a parent like me is likely to do is write a long-winded article to vent my frustration. The parent of a baby who is declared normal but is then born abnormal, however, is far likelier to  take things a lot further. This is one reason the culture in diagnostic medicine in general has become one of extreme risk-aversion: 

“Let’s test the lot so we don’t miss anything; because if we do, we could be in for a lawsuit.”

And finally, an even more cynical observation. Low specificity in diagnosis leads to a lot of healthy babies (or patients) being put through a lot of tests that they don’t need – and someone paying for it. In our case, we shelled out at least an extra two thousand dollars over the course of three months to keep an eye on Arjun’s femur. Surely there is a commercial imperative at work here, even if no one will openly admit it. I am not suggesting that individual doctors are thinking about money when suggesting monitoring. But the fact remains that anxious parents  - and a fetus that is being kept under strict observation -  is undeniably good for business. 

People and numbers: the false analogy

The whole time we were in Dr O.’s office, his eyes hardly left the computer screen that showed Arjun’s growth graphs. In fact the consultation could easily have been done remotely; as far as the doctor was concerned, the only important thing was those graphs. We could have sent them to him via an internet link. He hardly looked at us or dealt with us as people. For him, Arjun was not a growing baby but a set of points on a Cartesian plane. 

This is how modern medicine often works. The individual patient is abstracted to a set of data; then that data is compared to a body of global population data to draw conclusions about the condition of the patient. Thus cancer patients are nowadays told their chances of survival in terms of probabilities: a 80% chance of surviving five years, for example. What has actually happened is that the patient’s cancer has been converted to a set of data and then classified (say, as Type II); then historical data is consulted which shows that 80% of Type II cancer patients live for five years after diagnosis. This is then projected back onto the individual patient and expressed as a prediction about his future.

And this, I believe, is the fundamental flaw that is at the heart of the misuse of mathematics in medicine.

It is a flaw because the practice rests on false assumptions. To do medicine this way presupposes that a narrow abstraction of a human being can serve a useful purpose in isolation from the individual himself. It is an artefact of the reductionist, mechanistic view of science that has ruled the roost for so long; a model that has all but broken down in other branches of science but is still dominant in medicine. To look at such narrow abstractions is to treat a person as a sum of his parts alone. There is a false analogy at work: the analogy between a real, breathing human being - with all his history, personal traits and particular conditions – and a set of numbers that describe a very specific part of that person’s phyisiology. To believe the analogy is, I think, an absurd extreme of reductionism. In reality, a person is much , much more than the sum of his parts. We look at one small part in isolation and lose the holistic view of that individual as a working system.

Coming back to Arjun, was his femur really that small? Yes, if you measured it and placed it next to the particular population distribution that was available. But what if he were to be treated in context as an individual? A baby of Indian stock, who was small all around – even his head and torso were at the very small end of the scale – and who comes from a family where most of the older generations were stocky and short. A baby who had no other signs of any skeletal problems, was active and did not exhibit any of the other in utero signs of skeletal dysplasia. What would be the diagnosis then?

Surely not a “very low probability” of being normal?

The false analogy is used in both directions. Not only is the individual reduced to a string of data, but statistical data is often treated as if it represented something real. For example, the “average” biometric values are always used as a point of reference; yet there is no such thing as the “average” baby. There is no baby who has an exactly “average” length femur, an exactly “average” circumference of head, an exactly “average” bloodflow reading etc. Every individual baby, will exhibit some variation from the “average”. The “average” has a real existence in the abstract, mathematical world. But it is a human construction and an artefact of our mathematics. We have theories about its connection to the real world, but it does not live a “natural” existence. Yet, philosophically at least, the idea of the “average” being real in the world does strongly exist. And that is why we are so quick to compare someone to the “average”, and so concerned when the deviation is found to be “large”.

And what does it mean when someone is far from the “average”? Why are we so ready to take that as evidence of disorder or pathology? The “average” is simply the mathematical centre of the data what has been measured to date. Think about it: where else in life do we put such store in the abstract and the aggregate at the expense of the individual case? Do we tell a student about to sit his exam that he has a 20% chance of failing? Do we tell a batsmen as he steps out of the pavilion that he has a 7% chance of scoring a duck? Do we sign insurance policies at our wedding to cover the 30% chance of divorce?

We don’t, because in each of these cases we see the individual as possessing qualities that may place him or her outside of the normal historical range. People are different in ways that regular normal distributions know nothing about. You simply cannot project large-scale statistical measures onto individuals and call them predictions.

One last point. To compound the problem of the false analogy, there is also the phenomenon of “the more you look, the more you find”. If each of us, as adults, was put through the regime of pinpoint biometric measurement that fetuses undergo, I would guess that most of us would come up with some measurements that would be well outside the normal range. And yet most of us are “normal”. There may indeed be a vicious cyle at work here: the more testing your baby  undergoes, the more likely that something of “concern” will pop up; which means more follow-up testing, and possibly more “concerns”. And so on.

“Anything can be proved with statistics”

I realize this has all sounded something like a rant, but I am not bitter about what happened with our son. I don’t, in general, doubt the good faith and intentions of doctors. I am not a doctor myself; but I do understand maths, and I think the way medical science uses statistics is wrong and harmful. Folk wisdom say that “anything can be proved with statistics”. A combination of risk-aversion, mathematical illiteracy and  false assumptions has produced a system in which the everyday is becoming pathologized. My particular example is in prenatal diagnosis, but it is happening in other fields too. It is a dangerous trend, and it should be arrested.

Permit me one last piece of cynicism to finish on. I don’t think things will change easily because there are real economic barriers in the way of change. To overturn the false analogy between people and numbers would mean treating patients as individuals and doing qualitative work; in other words, getting to know them as people. That would make medicine less efficient and less lucrative for everyone. Dr M. and Dr O. seemed like nice enough people, but I doubt that they would ever vote for that.