Finding the needle – without using one

This week the DFTB team have been invited to run a conference within a conference in Sydney. Resus @ the Harbour is a multidisciplinary resuscitation conference combining powerful patient stories with cutting edge care – just the sort of thing we love at DFTB.

Critical illness is rare in children in the developed world. Spotting the sick child has been likened to finding a needle in the proverbial haystack and, cliched though it may be, in truth that analogy applies to many paediatric emergency department presentations. It is well recognized that treating potentially septic children early and aggressively reduces mortality. The same can be said for various surgical emergencies, non-accidental injury, arrhythmic causes of syncope and a host of other nasties, but how do we work out which child is really unwell? In some cases there are clear red flags; the neonate with green vomit, the non-mobile infant with a bruise, the kid who fainted while running. In others there are not.

We have two widely used and debated schools of thought in our efforts to pick the sick kid from their peers with self-limiting afflictions, that based in science and that based in art.

There is something very attractive about a flow chart, which if followed accurately would allow any clinician of any level of experience to distinguish the sick child from the well child and much effort continues to go into developing one. There are many examples of Paediatric Early Warning Scores in use around the world. These tools ask the bedside clinician to input some combination of vital signs and other clinical observations and provide a recommended course of action based on what goes in.

These tools fall into two big families; Single trigger tools – where one vital sign exceeding a certain threshold will initiate a clinical review, and multi-trigger tools where the variables are combined into an overall score and it is the score that guides the bedside clinician as to what to do. Examples of single trigger tools in widespread use in Australia are “Between the flags” and VicTOR (Though VicTOR is applied differently in some institutions). The Children’s Early Warning Tool (CEWT) is a multi-trigger tool used throughout Queensland.

Each early warning tool has slightly different test characteristics but as a broad generalization single-trigger tools are more sensitive but multi-trigger tools are more specific. In other words, a truly sick kid is less likely to be missed by a single trigger tool but a hospital using a single trigger tool will work it’s medical emergency team (MET) harder, which may have consequences for the rest of the patients in the hospital.

Early warning scores are explicitly designed as “track and trigger” or screening tools, not assessment tools. They are supposed to flag to bedside clinicians that something has changed. Though they are commonly misused for this purpose they are not designed to dictate which clinical area a patient should be managed in.

Blood tests

Older papers on risk stratifying the febrile child talk of the role of blood tests, particularly white cell count, CRP and blood cultures. More contemporary references will also talk of procalcitonin and various PCR’s, used either to rule in nasty bugs or to detect less worrisome invaders and by extension suggest a reduced likelihood of an invasive bacterial process. While all of these have a place in differentiating the trivial from the terrifying, we need to understand what the results mean in the population we are caring for. Specificities and sensitivities tell us about the test, not about the child in front of us. What we really care about are positive and negative predictive values – what is the chance that a positive test means the patient has the disease, or what is the chance that a negative test means the patient does not have the disease. Tessa has written more about this here.

The thing you really need to know about positive and negative predictive values is that they change for the same test depending on the background rate of disease in a population. The more common a disease is in the population the more likely a positive test result is to represent actual disease in the child. Changes in immunization practice over the last couple of decades have led to significant changes in the background rate of invasive bacterial infection in our paediatric population and hence changes in how our commonly used blood tests perform as discriminators between life-threatening and self-limiting disease.

In Australia we introduced Prevenar 7 (the 7 valent pneumococcal vaccine) in 2005 and saw a subsequent 75% reduction in invasive pneumococcal disease. This was upgraded to Prevenar 13 (active against more strains of pneumococci) in 2011 and as those children grow up the protection offered by these immunisations works its way further through Australia’s paediatric population. As invasive bacterial infection gets less common so the positive and negative predictive values of our tests change and it becomes more likely that a positive test does not actually represent disease.

To complicate matters even further some bugs are better than others at getting a patient’s inflammatory markers all excited. Meningococcus, for example, is notorious for minimally affecting the patients white cell count and CRP and indeed most of the kids I have seen with proven menigococcaemia have had white cell counts between 5 and 15 and CRPs of less than 20 on presentation – should we really find those numbers as reassuring as we often do?

The other school of thought is based on art. How many times have you heard the question of how to spot a sick child answered with “they just look sick”? In truth part of what we are looking at when deciding that a child “just looks sick” is measured by those early warning tools, resp rate, work of breathing, peripheral perfusion, mental state. How we really develop this sense has not changed since the time of Hippocrates. We see a lot of kids and follow their journeys. As we see some kids get better and others not we reflect on what it was about those that didn’t get better that was different and apply our conclusions when assessing kids in the future. In doing so our clinical judgement gets better with time. This is what Kolb called “experiential learning” and is a theory that gets quoted by simulation educators often.

We have developed a couple of traps for experiential learners in developed world paediatrics however, and these are especially applicable to those of us who work in the emergency department. Trap number one is that most kids get better anyway so we get a lot of reassurance that our judgement is good when actually it has just never been tested. Croup is one of the common conditions that really scares me, yet staff can spend six months in our ED and see a lot of kids who just need a bit of a dexamethasone and maybe a couple who need some adrenaline. With this experience its difficult not to learn that croup is just something that gets better with steroids and we don’t need to worry too much. I have used my clinical judgement to exclude epiglottitis in lots of kids but I have never actually seen a kid with true epiglottitis so I have no idea if my clinical judgement would stand the test of picking one.

The other big trap is that we don’t close the feedback loop. With our four-hour rules, shift work and compartmentalization of care we often don’t get the chance to witness the whole of even the hospital-based portion of the patient journey. If something goes terribly wrong we tend to hear about it but the kid who returns to a different care provider who orders a chest x ray that finds the occult pneumonia that we missed will remain in our consciousness as a correct diagnosis because we never hear otherwise. One of our many cognitive vulnerabilities that Daniel Kahneman talks about in Thinking Fast and Slow is our ability to get more comfortable making a decision the more times we make it, even though we never find out if it was the right decision or not.

Spotting the well kid

We are all pretty comfortable with the idea of a null hypothesis in research so if we have trouble reliably spotting the sick child – how about trying the opposite? Edward Snelson and Shammi Ramlakhan recently published their Delphi study in which they basically asked clinicians with experience in acute paediatrics what features on a clinical assessment would reassure them that a child was well. Being energetic, smiling, playing and age-appropriate verbalisation topped the list of reassuring features. What this paper highlights though is a thought process that I suspect is common to a lot of us, even if we don’t recognize it. Sick or not is often seen as a binary variable. In truth we see some kids who are obviously sick, some who are obviously well and some who, with all the clinical expertise it is difficult to be sure.

For those children in the unsure category we have in years gone by used blood tests to help us risk stratify in many cases, however for reasons described above these are not giving us the same information that they used to do. For these children if one data point is providing insufficient information for us to decide then the logical thing to do is collect more. As Damian Roland has written about here, observation is an investigation. We, and our systems, need to understand that sometimes we can’t make a spot diagnosis and we need to observe the trajectory of a child’s condition to know whether or not they need cefotaxime with their cuddles.

How good are we?

Simply put, not very! Samuel Vallaincourt and colleagues looked at all the children in Ontario who were admitted to hospital with diagnoses of meningitis or septicaemia between 2005 and 2010. They found that of 521 children eventually being given these diagnoses, 114 had recently been sent home from an ED. By using health administrative data they were able to track children’s hospital visits across different facilities and found that as well as 1 in 5 needing to return to hospital, 1 in 3 of those returned to a different hospital, so it is unlikely that many of the clinicians who had discharged them at the original presentation would have been aware of their patient’s progression. Interestingly those children who had initially been sent home did just as well as those that had been admitted at the first presentation in terms of length of stay, need for ICU and mortality.  This suggests that we can’t reliably pick sick children out of the crowd and safety netting advice inviting parents to return and the symptoms they should be looking out for is critical.