It’s fair to say that febrile infants can be challenging. Often presenting with insidious symptoms but looking reasonably okay, they may still have life-changing or life-limiting illnesses like sepsis or meningitis. You could argue that we should take the view of eliminating risk, performing septic screens on all febrile babies, and admitting for IV antibiotics until their cultures are returned. The vast majority will have a benign viral illness but at least you can rest assured you didn’t miss a seriously sick infant.
And that’s what we did when I started my paediatric training back when the dinosaurs roamed the earth – every baby under 6 months (yes, you heard it right, 6 months) with a fever got a full septic screen, including lumbar puncture, and was admitted to the ward for at least 48 hours pending cultures. But, from a health economics point of view, this is, let’s just say, perhaps not the best way to allocate healthcare resources.
Over the years, researchers have tried to rationalise our approach to febrile infants. 2013 saw the first NICE fever in under 5s guideline; a year later a group from Spain published the Step by Step approach to identifying young febrile infants at low risk for invasive bacterial infection; and last year, the PECARN group published a clinical prediction rule for febrile infants under 60 days, which had excellent sensitivity and negative predictive values to rule out serious bacterial infections.
Last month, the Spanish group published an article looking at the external validity of the PECARN rule in their dataset.
Velasco R, Gomez B, Benito J, et al. Accuracy of PECARN rule for predicting serious bacterial infection in infants with fever without a source. Archives of Disease in Childhood Published Online First: 19 August 2020
Before we plunge into the paper, let’s stop and think about a couple of important definitions here:
Serious bacterial infection (SBI) is used to describe bacteraemia, meningitis and urinary tract infections, also including infections such as pneumonia, skin, bone and joint infections, bacterial gastroenteritis and sometimes ENT infections.
Invasive bacterial infection (IBI) are infections where bacteria are isolated from a normally sterile body fluid, such as blood, CSF, joint, bone etc. An IBI is a type of SBI in a sterile site.
Who did they study?
Velasco’s group looked back at their registry of infants with a fever without source from a busy paediatric ED (> 50,000 presentations a year) in a tertiary hospital. To match the cohort in the PECARN paper, they used the following inclusion and exclusion criteria:
Inclusion: infants younger than 60 days who presented with a recorded fever, or history of recorded fever, of >38 C over an 11 year period between 2007 (when they started measuring procalcitonin) and 2018.
Exclusion: any infants whose history and/or examination pointed towards a focus, whose results didn’t include those used in the PECARN rule (absolute neutrophil count, PCT, urine dip), who didn’t have culture results, who were critically ill on presentation or who had a past history of prematurity, unexplained jaundice, previous antibiotics or other significant past medical history.
What were they looking for?
The group were interested to see how the PECARN rule fared in their dataset by looking at how many infants were predicted to be low-risk and yet had an SBI or IBI to assess the external validity of the rule.
What did they find?
1247 infants were included in this study. Of these, 256 (20.5%) were diagnosed with an SBI, including 38 (3.1%) with an IBI.
Of the 256 infants with an SBI, 26 (10%) were considered low risk by the rule. Of the 38 with an IBI, 5 were considered low risk (13.2%) by the rule. The PECARN rule would have missed 10% of infants with an SBI.
The PECARN rule’s sensitivity dropped from 97.7% in the original study to 89.8% and specificity dropped to from 60% in the original study to 55.5%.
So, how did Velasco’s group calculate the sensitivities and specificities of the PECARN rule for different groups in their dataset? They’ve nicely shown their data in 2 x 2 contingency tables in their figures. This is the data for SBI.
So, we can see that sensitivity (those patients testing positive for the SBI as a proportion of all patients who definitely have SBI) = 230 / 256 = 89.8%. This means that 10.2% are falsely negative.
Specificity (those patients who test negative for SBI as a proportion of all of those who don’t have SBI) = 550 / 991 = 55.5%. This means that 44.5% are falsely positive.
What about infants with a really short duration of fever?
When the group looked at infants with a history of less than 6 hours of fever (n=684, a little over half of the cohort), the sensitivity dropped further to 88.6%.
Why did the PECARN rule perform less well in this study?
The authors offer up a number of suggestions, some of which are outlined below.
The populations may be slightly different. Although the authors attempted to exclude ‘critically ill’ infants from this study (as the PECARN study excluded ‘critically ill infants’), a precise definition wasn’t coded in the original Spanish registry. Instead, they excluded infants from this study if they were ‘not well looking’ or admitted to ICU. Because of the way the data was coded, some critically ill infants may have been included in this study’s dataset, skewing the results.
The Spanish database was of febrile infants without a source, excluding babies with respiratory symptoms, which may explain why the rates of SBI and IBI were much higher in this study than the PECARN database of febrile infants. So, although the PECARN rule was highly sensitive in their group of febrile infants, as in this study it may not perform so well in febrile infants without a source.
This study showed that the PECARN rule performed less well in infants with a short duration of fever. Overall, infants in the PECARN study had a longer history of fever at presentation – over a third of the PECARN infants had fever >12 hours compared to 11% in this study. Over half of the infants in this study presented within the first 6 hours. Blood tests are less sensitive in the first few hours of a febrile illness and this may well partially explain why the rule performed less well outside the PECARN dataset.
It’s important not to ignore this study’s limitations. The PECARN dataset recruited infants from multiple centres, while the registry for this study came from only one ED. As this study was a secondary analysis of a dataset, a power calculation wasn’t performed. Generally, a minimum of 100 cases is recommended for validating a model, but only 38 infants in this study had an IBI.
Study bottom line
This study showed that in the Spanish dataset of infants under 60 days with a fever without source, the PECARN rule performed less well than in the original study. This was especially true for infants with a short history of less than 6 hours of fever.
Clinical bottom line by Damian Roland
In Kuppermann et al’s original 2019 study febrile infants 60 days and younger were demonstrated to be at low risk of SBIs using 3 laboratory test results: Urinalysis, Absolute Neutrophil Count (ANC), and serum procalcitonin (PCT) levels. The study was well designed and therefore compelling in providing a framework in which to manage these challenging presentations. However, with respect to knowledge translation, external validity is critical. The availability of PCT is a significant limiting factor to being able to show the PECARN approach could be reproduced internationally. While PCT is used in Europe and Australia, it’s certainly not widespread in the UK where I practice, and then it is only used routinely in a very small number of hospitals. This makes Velasco and colleagues’ work really important as they were able to replicate the requirements of the original study and helps answer an important question: should centres start introducing PCT into their diagnostic pathology panels? The results of this study will be interpreted differently by different observers as ultimately the question is of risk tolerance. Personally, a 10% false-negative rate (if this is indeed the case) for an outcome that could result in long term disability feels uncomfortable. Counselling a parent that they could return home without treatment knowing this would probably be quite challenging. I am not sure many departments would be rushing to buy point of care PCT.
However, there are two very important caveats. Firstly, is the validation cohort different from my own local cohort? The prevalence of disease has a huge bearing on the accuracy of any test. Knowing the local incidence of SBI and IBI in your own institution is important (but actually getting the numbers is harder than you may think!). It is likely that the PECARN approach may perform more effectively in other centres. Importantly the original paper highlights that implementation may be more effective in the second month of life due to the impact of HSV and other peri-natal infections present at 0-30 days. Secondly, what is the threshold for undertaking the blood tests in the first place? Fever in an infant less than 3 months is an interesting area as it’s one of the very few presentations in which a solitary symptom or sign is independently predictive of risk. Regardless of how the child appears to a health care professional, there is a risk of SBI and IBI (of anywhere between 2-10%) just by having a fever. This does mean that sometimes there is variation in approaches when there is a history of fever rather than a documented fever (for fear of not wanting to do a battery on tests on a neonate who in front of you appears completely well and has normal observations). But more importantly, this has led to an approach where although blood tests are taken, the results are often disregarded as an LP will be done and antibiotics will be given regardless. There are many cultural practices that have evolved around the management of the febrile neonate both within individuals and institutions. While in a study situation these are controlled for, their influence in the real world can not be underestimated and this is why it’s so important we have some pragmatic studies in this area.
This study makes me more determined to define our incidence of SBI locally and work out what impact new approaches to management may have. I think all centres should probably be doing this. However knowing the potential uncertainty in the sensitivity of the PECARN approach means it’s unlikely to be adopted in the immediate future without further validation.
**post blog addendum 1st September 2020**
While this blog was in post production phase Kuppermann and colleagues have released further data on implementing their original predictive rule. This work has been summarised by Dr. Kuppermann below (click on to go to the original thread) and provides useful context to the discussion about external validity and implementation – DR.
