The missing component of clinical practice

Cite this article as:
Damian Roland. The missing component of clinical practice, Don't Forget the Bubbles, 2019. Available at:
https://doi.org/10.31440/DFTB.19685

This is an extract of the talk I gave at #DFTB19 highlighting an important research ethos – the full talk will be released via the Don’t Forget the Bubbles at a later date.

The Doctor” is a painting by Luke Fildes and was first exhibited in 1891.

The Doctor exhibited 1891 Sir Luke Fildes 1843-1927 Presented by Sir Henry Tate 1894 https://www.tate.org.uk/art/work/N01522

The artist had lost his son Philip at the age of one and the scenes reflects the admiration that he had had for the doctor who had looked out for him. 

For some the painting may represent a stereotypical view of medicine in the past – the doctor rubbing his chin in a wise fashion, the child prostrate on a make-shift bed. And there is a parent figure in the background, watching on anxiously. 

This painting has had a revival recently despite being over 100 years old. It highlights the triad of care we all know exists in paediatrics – the child, the parents and carers, and ourselves.

This triad has received increased attention recently. The need for child centered care in respect of their engagement and involvement in their care. The need for positive communication with families; we remember the cases where parents haven’t acted as their child’s advocates but forget the vast majority of cases when they have. We so often let parents down when we should have been, not just listening to them, but honestly hearing what they were saying. And most recently the doctors themselves. An understanding of the importance of wellbeing and the shackles of rudeness. 

There is a fourth component, as well. One which perhaps will never get the attention it deserves because it isn’t a visceral part of our clinical care. It’s something we know exists but are quite willing to ignore. It’s something that perhaps has more impact on our practice than we would like to admit. It’s the variability in the actual care or treatment we provide or the fact that it might not be necessary at all.

When I became chair of PERUKI, Paediatric Emergency Research United Kingdom and Ireland, the international sibling of PREDICT and daughter of PERN I’d a personal vision that I would drive the organization forward in delivering ground-breaking new research highlighting novel interventions that would really make a difference to patients. What actually occurred is that I have realised that perhaps PERUKI has an even more important roll. One that does obviously include the need to develop, innovate and implement but one also that highlights where we could, and should do, better. It’s some examples of variation and the need for no treatment I would like to share. 

So this is an original selection of PERUKI members and those who helped us get PERUKI off the ground. I’d like a chance to pay particular tribute to Mark Lyttle at this point who has worked tirelessly at the outset to drive forward many early projects and is consistently named checked by our research partners for his ceaseless enthusiasm at collaborating and engaging. PERUKI took part in a prioritisation process published in 2015 with members putting forward their preferred research agendas and PERUKI publishing the top 20 via a Delphi process.

Number 4 on this list was: what is the best IV medication for Acute Asthma. PERUKI started on this work with essentially a two phase examination of the management of wheeze in March 2013. In the first phase a written questionnaire was undertaken. PERUKI sites responded as departments and 183 consultants responded individually on their wheeze management.

In study 99 (54.1%) use salbutamol as first-line intravenous therapy, 52(28.4%) magnesium sulfate and 27 (14.8%) aminophylline; 87 (47.5%) give these sequentially depending on response and 30 (16.4%) give them concurrently. Overall, 146 (79.8%) continue inhaled bronchodilators while on intravenous therapy.

When commencing on intravenous bronchodilators there were 10 different infusion rates with over 10-fold variation between the lowest and highest.

Everyone tends to have their little foibles about which treatment they prefer. And given the range of phenotypes and genotypes that exist in our wheezy cohort in can’t be the case that there is only going to be one best fit treatment for all patients. But a 10-fold difference probably pushes the bounds of flexibility.

What makes this more interesting is the second study. Also completed at the time (March 2013) was a prospective observational study. Data was screened from all patients presenting with wheeze and a detailed proforma completed for those who received intravenous therapies.Of 3238 children, 101 received intravenous therapies. Intravenous magnesium sulfate (MgSO4) was used in 67 (60.9%), salbutamol in 61 (55.5%) and aminophylline in 52 (47.3%) of cases. 

In 35 cases (31.8%), two drugs were used together, and in 18 cases (16.4%), all three drugs were administered.

More than half used salbutamol as the first-line intravenous agent, while fewer preferred magnesium sulfate or aminophylline, suggesting equipoise regarding which is most efficacious. To investigate this, participants were asked whether they would enrol patients to a randomised controlled trial allocating salbutamol, aminophylline or magnesium sulfate as the first-line intravenous agent, to which 148 (80.9%) responded positively. Asking clinicians who are regularly prescribing acute medications is vital for study design and subsequent implementation of study findings. With all due respect to respiratory paediatricians the question that they may be interested in, or want to explore, may well be completely out of keeping with the practice habits of emergency and acute paediatricians. PERUKI have welcomed increased engagement with our specialty colleagues in the last year and we hope we will reap the benefits of this. 

So a clear example of variation. I feel uncomfortable. Is there any reason to believe this variation has improved 6 years on? We have a challenge as the evidence base is not as strong as we would like. We look to Simon Craig and his work on developing asthma outcomes here – a PERN study I am very proud that PERUKI is part of. 

So what about where we think there is only a small amount of variation (a nationally agreed algorithm for example). DO we need to improve practice and CAN we improve practice? The EcLIPSE study was published a mere month ago and I am proud of the Don’t Forget the Bubbles team  for being part of the process of sharing this information widely. The Eclipse study compared levetiracetam and phenytoin in the treatment of status epilepticus. It was published on exactly the same day as the ConSEPT trial a similar study from our PREDICT friends. The EcLIPSE paper is available open access and there is a Don’t Forget the Bubbles summary. I also recommend the reviews by Justin Morgenstein and Casey Parker 

The primary outcome was time from randomisation to cessation of all visible signs of convulsive activity, defined as cessation of all continuous rhythmic clonic activity, as judged by the treating clinician.

Much debate has centred on what EcLIPSE and ConSEPT showed and at the heart of this is the difference between superiority and non-inferiority.

If these studies do nothing else it will to be to have spread the word about this construct. Because it is really important that people don’t glaze over or think because this terminology is used it’s someones else’s problem to analyse. I think this undue deference to academics probably perpetuates variation in care. I am not saying the theory is easy but neither is managing a sick neonate with congenital heart disease and we completely commit ourselves to doing that. 

Superiority trials aim to demonstrate that one intervention is better than other. The statistics, by convention, dictate that a difference between the interventions needs to be defined. In the case of EcLIPSE because phenytoin stopped status 60% of time and it was felt Levetiracetam may terminate seizures at a 75% rate the statistics calculated that 140 patients would be needed in each group. IF a difference exists this difference is likely to be a difference that is real and not by chance alone.

If they had wanted to show that levetiracetam was only 1% better then 1000s of patients would probably have been needed as if there was no difference by chance it would easily be possible that levetiracetam happened to be 1% better in that cohort of patients. 

A few interesting facts come out of EcLIPSE.

The first is that the while this wasn’t a perfect observational study – i.e not all patients presenting were recruited across a wide range of hospitals over 1400 patients were screened. This is a good cohort of children with seizures. About 10% of those who needed second line treatment for status were first presentations of afebrile convulsions and 5% were as a result of CNS infection.

Median time from randomisation to start of infusion was 11 min (IQR 8–15) for levetiracetam and 12 min (8–17) for phenytoin

But median time from randomisation to seizure cessation was 35 min (IQR 20 to not assessable) in the levetiracetam group and 45 min (IQR 24 to not assessable) in the phenytoin group.

These interventions take time! 

In EcLIPSE convulsive status epilepticus was terminated by levetiracetam in 106 (70%) participants, and by phenytoin in 86 (64%) participants. Therefore by the statistics LEVETIRACETAM is NOT better

Because the results are broadly the same it doesn’t mean they are equal – a non-inferiority study looks at two drugs and aims to calculate what is the minimum number of patients needed to be recruited into each intervention arm to demonstrate that one drug is not more than a certain % worse than another. By convention that number is normally 10%. The reason why 10% is important is that in EcLIPSE while it appears levetiracetam may have passed this test if the study had been designed as a non-inferiority in the ConSEPT study levetiracetam only terminated seizures (albeit as different end point) 50% of time; 10% worse than phenytoin. We don’t know yet what the meta-analysis may show us but this is planned.

A further suggestion is should we consider adding in levetiracetam with phenytoin; we could but that might delay some RSI intervention even further without overall benefit in seizure termination further. This is messy area where the complexity of clinical practice hints the required precision of research head on.

It might well be that you are happy for others to research novel drugs and techniques. You may well be content in supporting research through signposting or perhaps recruiting patients yourself. I would ask though that research itself is not scary. There is false divide between the ivory tower academic and jobbing clinician. Both these terms probably tribal and derogatory in their own way. We should all care about how effective our treatments are and where variability in practice is not in the patient’s interest. It is no more or less important than the three figures in Luke Fildes picture but perhaps it is less visible. 

Through PERUKI I’d like to champion this cause to make research feel more accessible. We are not doing research because we like to, we are doing it because we have to. 

Are there too few women presenting at paediatric conferences?

Cite this article as:
Davis, T. et al. Are there too few women presenting at paediatric conferences?, Don't Forget the Bubbles, 2018. Available at:
https://doi.org/10.31440/DFTB.16879

Sometimes we have a great idea for a paper but try as we might we cannot get it published in the traditional way. So what better means of disseminating knowledge than publishing it right here, on the Don’t Forget the Bubbles website? Given that this is the week of FIX18, the Feminem Idea eXchange, it seems like there is no better time like the present to discuss female presenters at paediatric conferences.

Steroids for pre-school wheeze

Cite this article as:
Tessa Davis. Steroids for pre-school wheeze, Don't Forget the Bubbles, 2018. Available at:
https://doi.org/10.31440/DFTB.14563

Wheeze must be one of the most common paediatric presentations to the emergency department and up till now most of us have been reassuring parents and sending them away without treatment. But should we be doing more?  A paper, released just last week, suggests that we could.

Big Picture Paediatrics : Adverse Childhood Experiences

Cite this article as:
Henry Goldstein. Big Picture Paediatrics : Adverse Childhood Experiences, Don't Forget the Bubbles, 2016. Available at:
https://doi.org/10.31440/DFTB.10082

So much of paediatrics, and medicine in general, is focussed on small experimental or observational studies. This series of posts takes the wider view; we’re talking here about some of the biggest and longest running studies that help us frame, measure and understand childhood through time and across the world.

Who & what was studied?

Kaiser Permanente is a large Medical Insurer in the USA; they collected data in two waves in the primary care setting with a view to describing the long-term relationship of childhood experiences to important medical and public health problems. The study initially rolled out in 1996 & 1997.

Felitti, VJ, Anda RF, Nordenberg D et al. Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults : The Adverse Childhood Experiences (ACE) Study. American Journal of Preventive Medicine. 1998:14, 245–258.

The study aimed to assess – both retrospectively and prospectively – the long-term impact of abuse and household dysfunction during childhood on disease risk factors and incidence, quality of life, health care utilization, and mortality for adults.

Here is the actual questionnaire:

Answer yes or no; all ACE questions refer to the respondent’s first 18 years of life.

Abuse

  • Emotional abuse: A parent, stepparent, or adult living in your home swore at you, insulted you, put you down, or acted in a way that made you afraid that you might be physically hurt.
  • Physical abuse: A parent, stepparent, or adult living in your home pushed, grabbed, slapped, threw something at you, or hit you so hard that you had marks or were injured.
  • Sexual abuse: An adult, relative, family friend, or stranger who was at least 5 years older than you ever touched or fondled your body in a sexual way, made you touch his/her body in a sexual way, attempted to have any type of sexual intercourse with you.

Household Challenges

  • Mother treated violently: Your mother or stepmother was pushed, grabbed, slapped, had something thrown at her, kicked, bitten, hit with a fist, hit with something hard, repeatedly hit for over at least a few minutes, or ever threatened or hurt by a knife or gun by your father (or stepfather) or mother’s boyfriend.
  • Household substance abuse: A household member was a problem drinker or alcoholic or a household member used street drugs.
  • Mental illness in household: A household member was depressed or mentally ill or a household member attempted suicide.
  • Parental separation or divorce: Your parents were ever separated or divorced.
  • Criminal household member: A household member went to prison.

Neglect

  • Emotional neglect: Someone in your family helped you feel important or special, you felt loved, people in your family looked out for each other and felt close to each other, and your family was a source of strength and support.
  • Physical neglect: There was someone to take care of you, protect you, and take you to the doctor if you needed it, you didn’t have enough to eat, your parents were too drunk or too high to take care of you, and you had to wear dirty clothes.


What does this mean?

The ACEs questionnaire accumulates a score from zero to seven based on yes/no responses to the above questions. These results in conjunction with a “Health Appraisal Clinic’s questionnaire” allowed correlation with risk factors such as smoking, severe obesity, physical inactivity, depressed mood, suicide attempts, alcoholism, any drug abuse, sexually transmitted diseases, parental drug abuse and a high lifetime number of sexual partners (>50), as well as the big swingers; mortality and overall morbidity.

The ACE score has been utilised to demonstrate a graded dose-response with more than 40 outcomes. You can see the entire list of publications here.

How good is this dataset?

Although there are almost all of the expected threats to validity from a questionnaire administered to people obtaining health insurance in the USA in the 1990s, the dataset is very good.

Of the 13,494 surveys, there was a 70.5% (9508) response rate, sent a week after standardised medical review. Respondents who did not respond to all questions were excluded from the final analysis. After non-responders and exclusions, a total dataset of 8056 responders was analysed. Alarmingly, more than half of the exclusions were for not answering the question about childhood sexual abuse. This certainly raises some concern for a risk of underreporting, particularly if this was the only question omitted! 


What meaning can be drawn from the results (so far)?

The dataset has lent itself to the associations between adverse childhood experiences and a veritable laundry list of medical, psychiatric pathology as well as social and public health problems.

This is data reports that 1 in 5 were sexually abused, nearly 1 in 4 lived with a “problem drinker or alcoholic” and that around 1 in 6 had a household member who was depressed or mentally ill.

It’s worth remembering that this study paints a picture of the adverse childhood experiences of the older generations in the USA – the mean age of respondents was 56.1 (19-92) years – in a study undertaken just over 20 years ago.

Rather than provide a snapshot of what childhood is like today, this data informs us about the childhood of parents of our patients. This gives us some understanding and frameworks by which to consider expectations of childhood from the parental & societal viewpoint – that most parents hope for a rosier childhood with fewer adverse experiences than their own.

With this in mind, and with a critical eye to some of the correlating outcomes, behaviours such as alcohol & drug abuse, smoking, over-eating, and sexual behaviours might alternatively be viewed as both coping strategies and symptoms of the anxiety, anger and depression that is likely co-morbid with high levels of adverse childhood experiences.

Primary prevention of adverse childhood experiences necessitates change at the societal level; with a focus on improving the quality of family and household environments through the childhood years.

Funding for the original study was combined between Kaiser Permanente (San Diego) and the US Center for Disease Control.

Where next?

The Centre for Disease in Childhood has taken over the study and, since 2009, transformed it into a national program across 32 states of the USA, called “Behavioral Risk Factor Surveillance System” (BRFSS). Data from the 2010 BRFSS has been published and includes more than 50,000 respondents. You can see more about the participating states, future timeline and previous data via the CDC website, here.

References:
Felitti, VJ, Anda RF, Nordenberg D et al. Relationship of Childhood Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults : The Adverse Childhood Experiences (ACE) Study. American Journal of Preventive Medicine. 1998:14, 245–258. 

Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, Division of Violence Prevention Adverse Childhood Experiences (ACEs)”.U.S. Department of Health & Human Services, Atlanta, USA. Accessed 27 September 2016. https://www.cdc.gov/violenceprevention/acestudy

Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, Division of Violence Prevention. “About Behavioral Risk Factor Surveillance System ACE Data”.U.S. Department of Health & Human Services, Atlanta, USA. Accessed 5 October 2016. https://www.cdc.gov/violenceprevention/acestudy/ace_brfss.html

Early Budesonide for the Prevention of Bronchopulmonary Dysplasia

Cite this article as:
Henry Goldstein. Early Budesonide for the Prevention of Bronchopulmonary Dysplasia, Don't Forget the Bubbles, 2015. Available at:
https://doi.org/10.31440/DFTB.7859

Bronchopulmonary dysplasia (BPD) is a common outcome in premature neonates, from 85% in 22/40 infants, to about 33% of neonates born in the 27th week of gestation. This recent study, published in the NEJM trialled a potential new therapy to reduce BPD.

Screening tests and statistics – we’re only doctors, what do we know?

Cite this article as:
Tessa Davis. Screening tests and statistics – we’re only doctors, what do we know?, Don't Forget the Bubbles, 2014. Available at:
https://doi.org/10.31440/DFTB.5537

A letter published in JAMA this week confirms what we already feared in medicine – we really do not understand statistics. Somehow throughout our university and training we have persuaded ourselves that statistics aren’t relevant unless you are conducting your own research. And yet we are delivering advice to patients every day based on our understanding of test accuracy.

We need to know what we are talking about. And we don’t…

Manrai et al asked 24 attendings, 26 house officers, 10 medical students and 1 retired doctor this question….

“If a test to detect a disease, whose prevalence is 1/1,000, has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease?”

Only 23% of those asked got the answer correct, and most common incorrect answer was 95% (an answer given by 44% of respondents).

What’s the explanation?

This question is asking for positive predictive value, which takes into account the prevalence. Most people assume that the false positive directly predicts the chances of the person having the disease if their test is positive.

In a population of 1000 people, there will be 1 person with the disease (prevalence is 1 in 1000).

If you tested the whole population then you would get roughly 50 false positives (5%). And we know that 1 person has the disease (so we can assume they will have a positive test result). So in that population, there will be 51 positive test results.

Therefore if your test comes back positive then there is roughly 1 in 51 chance that you have the disease – approximately 2%.

Does this research letter represent a real problem?

Most people think that patients have a 95% chance of having the disease if the test is positive, when actually they only have a 2% chance. This has huge implications for treatment following screening tests.

Although this research letter is based on a survey, we have recently seen the implications of this in real life too. In January a study was published which investigated use of the 15-minute SAGE tool (self-administered gerocognitive examination) in detecting Alzheimer’s. The newspapers became very excited.

The Telegraph:

Four out of five people (80 percent) with mild thinking and memory (cognitive) issues will be detected by this test, and 95 percent of people without issues will have normal SAGE scores

Sky News

SAGE could spot mild thinking and memory issues in 80% of those tested

And to be honest, it doesn’t seem reasonable to expect the press to bust the stats when many of us are equally confused.

What’s the explanation?

As the Telegraph correctly stated,  for 80% of people who have MCI, the test will be positive. This is a sensitivity of 80%.

And 95% of healthy people will get the correct diagnosis i.e. the test will be negative. This is a specificity of 95%

Whilst this looks good on first glance, to asses the usefulness of this being a screening test, the prevalence is crucial.

The estimated prevalence of MCI in the over 60s is 5%.

If there are 10,000 people, 500 (5% prevalence) will have MCI. Of these, 400 will test positive (sensitivity of 80%).

Out of the remaining 9,500 healthy people, 475 will get a positive test result (even though they are healthy). This is the specificity of 95%.

David Colquhoun provides some nice charts to explain this visually in his DC’s Improbable Science blog.

So, in a population of 10,000 people (over 60 years old), 875 will test positive. Of these, only 400 will actually have the disease (45%). If your patient has a positive test result, there is only a 45% chance that they actually have the disease). And if you change that to include the whole population regardless of age, only 14% of those who test positive will have the disease

Who is Bayes and what does he have to do with screening?

In the 1700s, Thomas Bayes was the first person to describe using evidence to update beliefs. He described probability as the ‘degree of belief’. When new evidence comes to light then the probability changes.

If your 8 year old tells you they did not eat the chocolate ice cream in the freezer then you would be inclined to believe them. But if you then notice chocolate down her dress, your belief might change somewhat. In many ways, Bayes theorem is just logical thinking.

In screening we aren’t simply asking ‘do you have the disease?’. We are asking ‘given that your test is positive, do you have the disease?’. And this must take into account the false positives and the prevalence.

In Graeme Archer’s Telegraph blog, he creates a lovely graph showing clearly that as prevalence increases, so does the positive predictive value.

Where does that leave us with screening tests?

Screening tests need to be considered in context. Specifically which section of the population is being screened, what the prevalence is for the disease being tested in that population, and how this affects the usefulness of the test.

Perhaps the more relevant question is – where does it leave us with statistics?

For those of use who were not taught it in med school, we need to take responsibility and self-educate. Bob Phillips is doing this in beautiful, bite-sized pieces for Archives of Disease in Childhood with his StatsMiniBlog; and Simon Carley is doing it eloquently as ever at St Emlyn’s – see his Risky Business posts for starters.

Selected references

Manrai AK, Bhatia G, Strymish J, Kohane IS, Jain SH, Medicine’s uncomfortable relationship with math: calculating positive predictive value, JAMA, online first, 21st April 2014, doi:10.1001/jamainternmed.2014.1059.

Scharre DW, Chang SI, Nagaraja HN, Yager-Schweller J, Mirden RA, Community cognitive screening using the Self-Administered Gerocognitive Examination (SAGE), The Journal of Neuropsychiatry and Clinical Neurosciences 2014; 00:1–7.

Colquhoun D, On the hazards of significance testing. Part 1: the screening problem, DC’s Improbable Science, 2014.

Arches G, False positives and Bayesian reasoning: have I really got dementia? The Telegraph Blogs, 2014.

Evaluating screening tests, the role of probability, Boston University School of Public Health, 2014.

Siegfried T, Doctors flunk quiz on screening-test math, Science News, 2014.