It’s fair to say that febrile infants can be challenging. Often presenting with insidious symptoms but looking reasonably okay, they may still have life-changing or life-limiting illnesses like sepsis or meningitis. You could argue that we should take the view of eliminating risk, performing septic screens on all febrile babies, and admitting for IV antibiotics until their cultures are returned. The vast majority will have a benign viral illness but at least you can rest assured you didn’t miss a seriously sick infant.
And that’s what we did when I started my paediatric training back when the dinosaurs roamed the earth. Every baby under 6 months (yes, you heard it right, 6 months) with a fever got a full septic screen, including lumbar puncture, and was admitted to the ward for at least 48 hours pending cultures. But, from a health economics point of view, this is, let’s say, perhaps not the best way to allocate healthcare resources.
Over the years, researchers have tried to rationalise our approach to febrile infants. 2013 saw the first NICE fever in under 5s guideline; a year later, a group from Spain published the Step by Step approach to identifying young febrile infants at low risk for invasive bacterial infection; and last year, the PECARN group published a clinical prediction rule for febrile infants under 60 days, which had excellent sensitivity and negative predictive values to rule out serious bacterial infections.
Last month, the Spanish group published an article looking at the external validity of the PECARN rule in their dataset.
Velasco R, Gomez B, Benito J, et al. Accuracy of PECARN rule for predicting serious bacterial infection in infants with fever without a source. Archives of Disease in Childhood Published Online First: 19 August 2020
Before we plunge into the paper, let’s stop and think about a couple of essential definitions here:
Serious bacterial infection (SBI) is used to describe bacteraemia, meningitis, and urinary tract infections, as well as infections such as pneumonia, skin, bone, and joint infections, bacterial gastroenteritis, and sometimes ENT infections.
Invasive bacterial infections (IBI)Â are infections in which bacteria are isolated from a normally sterile body fluid, such as blood, CSF, joint, bone, etc. An IBI is a type of SBI in a sterile site.
Who did they study?
Velasco’s group looked back at their registry of infants with fever without source from a busy paediatric ED (> 50,000 presentations a year) in a tertiary hospital. To match the cohort in the PECARN paper, they used the following inclusion and exclusion criteria:
Inclusion: infants younger than 60 days who presented with a recorded fever, or history of recorded fever, of >38 C over 11 years between 2007 (when they started measuring procalcitonin) and 2018.
Exclusion: any infants whose history and/or examination pointed towards a focus, whose results didn’t include those used in the PECARN rule (absolute neutrophil count, PCT, urine dip), who didn’t have culture results, who were critically ill on presentation or who had a history of prematurity, unexplained jaundice, previous antibiotics or other significant past medical history.
What were they looking for?
The group was interested in seeing how the PECARN rule fared in their dataset by examining the number of infants predicted to be low-risk yet had an SBI or IBI to assess the rule’s external validity.
What did they find?
1247 infants were included in this study. Of these, 256 (20.5%) were diagnosed with an SBI, including 38 (3.1%) with an IBI.
Of the 256 infants with an SBI, 26 (10%) were considered low risk by the rule. Of the 38 with an IBI, 5 were considered low risk (13.2%) by the rule. The PECARN rule would have missed 10% of infants with an SBI.
The PECARN rule’s sensitivity dropped from 97.7% in the original study to 89.8%, and specificity dropped from 60% to 55.5%.
So, how did Velasco’s group calculate the sensitivities and specificities of the PECARN rule for different groups in their dataset? Their figures nicely show their data in 2 x 2 contingency tables. This is the data for SBI.
So, we can see that sensitivity (the proportion of patients testing positive for the SBI as a proportion of all patients who definitely have SBI) = 230 / 256 = 89.8%. This means that 10.2% are falsely negative.
Specificity (the proportion of patients who test negative for SBI compared to all those who don’t have SBI) = 550 / 991 = 55.5%. This means that 44.5% are falsely positive.
What about infants with a really short duration of fever?
When the group looked at infants with a history of less than 6 hours of fever (n=684, a little over half of the cohort), the sensitivity dropped to 88.6%.
Why did the PECARN rule perform less well in this study?
The authors offer up several suggestions, some of which are outlined below.
The populations may be slightly different. Although the authors attempted to exclude ‘critically ill’ infants from this study (as the PECARN study excluded ‘critically ill infants’), a precise definition wasn’t coded in the original Spanish registry. Instead, they excluded infants from this study if they were ‘not well looking’ or admitted to ICU. Because of how the data was coded, some critically ill infants may have been included in this study’s dataset, skewing the results.
The Spanish database contained febrile infants without a source, excluding babies with respiratory symptoms. This may explain why the rates of SBI and IBI were much higher in this study than in the PECARN database of febrile infants. So, although the PECARN rule was highly sensitive in their group of febrile infants, as in this study, it may not perform so well in febrile infants without a source.
This study showed that the PECARN rule performed less well in infants with a short duration of fever. Overall, infants in the PECARN study had a longer history of fever at presentation – over a third of the PECARN infants had fever >12 hours compared to 11% in this study. Over half of the infants in this study presented within the first 6 hours. Blood tests are less sensitive in the first few hours of a febrile illness and this may partially explain why the rule performed less well outside the PECARN dataset.
It’s important not to ignore this study’s limitations. The PECARN dataset recruited infants from multiple centres, while the registry for this study came from only one ED. This study was a secondary dataset analysis, so a power calculation wasn’t performed. Generally, a minimum of 100 cases is recommended for validating a model, but only 38 infants in this study had an IBI.
Study bottom line
This study showed that in the Spanish dataset of infants under 60 days with a fever without a source, the PECARN rule performed less well than in the original study. This was especially true for infants with a short history of fever of less than 6 hours.
Clinical bottom line by Damian Roland
In Kuppermann et al.’s original 2019 study, febrile infants 60 days and younger were demonstrated to be at low risk of SBIs using 3 laboratory test results: Urinalysis, Absolute Neutrophil Count (ANC), and serum procalcitonin (PCT) levels. The study was well designed and, therefore, compelling in providing a framework to manage these challenging presentations. However, with respect to knowledge translation, external validity is critical. The availability of PCT is a significant limiting factor to showing that the PECARN approach could be reproduced internationally. While PCT is used in Europe and Australia, it’s certainly not widespread in the UK, where I practice, and it is only used routinely in a very small number of hospitals. This makes Velasco and colleagues’ work really important as they were able to replicate the requirements of the original study, which helps answer an important question: Should centres start introducing PCT into their diagnostic pathology panels? The results of this study will be interpreted differently by different observers as, ultimately, the question is of risk tolerance. Personally, a 10% false-negative rate (if this is indeed the case) for an outcome that could result in long-term disability feels uncomfortable. Counselling a parent that they could return home without treatment, knowing this would probably be quite challenging. I am not sure many departments would rush to buy point-of-care PCT.
However, there are two very important caveats. Â Firstly, is the validation cohort different from my local cohort? The prevalence of the disease has a huge bearing on the accuracy of any test. Knowing the local incidence of SBI and IBI in your institution is important (but getting the numbers is harder than you may think!). The PECARN approach may likely perform more effectively in other centres.
Importantly the original paper highlights that implementation may be more effective in the second month of life due to the impact of HSV and other peri-natal infections present at 0-30 days. Secondly, what is the threshold for undertaking the blood tests in the first place? Fever in an infant under 3 months is an interesting area as it’s one of the very few presentations in which a solitary symptom or sign independently predicts risk.
Regardless of how the child appears to a health care professional, there is a risk of SBI and IBI (of anywhere between 2-10%) just by having a fever. This does mean that sometimes there is variation in approaches when there is a history of fever rather than a documented fever (for fear of not wanting to do a battery on tests on a neonate who in front of you appears completely well and has normal observations).
More importantly, this has led to an approach where although blood tests are taken, the results are often disregarded, as an LP will be done, and antibiotics will be given regardless. Many cultural practices have evolved around managing the febrile neonate, both within individuals and institutions. While in a study situation, these are controlled for, their influence in the real world can not be underestimated, and this is why it’s so important we have some pragmatic studies in this area.
This study makes me more determined to define our incidence of SBI locally and work out what impact new approaches to management may have. I think all centres should probably be doing this. However, knowing the potential uncertainty in the sensitivity of the PECARN approach means it’s unlikely to be adopted immediately without further validation. Â
**post blog addendum 1st September 2020**
While this blog was in the post-production phase, Kuppermann and colleagues released further data on implementing their original predictive rule. Dr. Kuppermann has summarised this work below (click on to go to the original thread), which provides useful context to the discussion about external validity and implementation—DR.
