Curiouser and Curiouser: when we get the wrong answer from the right research by Damian Roland

Damian is a consultant and honorary associate professor in Paediatric Emergency Medicine. He is a research powerhouse as you can tell if you look at the list of his recent publications. He is also the chair of PERUKI – the Paediatric Emergency Research in the UK and Ireland – and contributes heavily to our monthly round up of interesting literature.

In this talk he asks to be wary of the conclusions we draw from just reading the abstracts. He reminds us, as have all of our great speakers, that we are here, ultimately, to make a difference for our patients.

So, sit back and listen.

Don't Forget the Bubbles
Don't Forget the Bubbles
Curiouser and Curiouser: when we get the wrong answer from the right research by Damian Roland

Navigating the Sea of Clinical Literature

Cite this article as:
Siobhán McCoy. Navigating the Sea of Clinical Literature, Don't Forget the Bubbles, 2018. Available at:


“If we choose to change nothing, then nothing will change”

I started with this because, there is no two ways about it, clinical research can be hard. It’s not the new Gucci piece of kit everyone wants to use but it is the foundation of healthcare as we know it. If we want things to change, we have to choose to change them. So, lets dive into our next instalment of the DFTB research section.

Healthcare literature is evolving at a rate of knots and it can be hard to keep up. The sheer volume of literature out there is, for want of a better word, baffling. Although you are not expected to have read it all, it is important to have an idea of what is going on within your chosen speciality in relation to developments in clinical practice, policy and education. Although the wonderful world of FOAMed helps us to generate a broader view, it does not cater for the nuances of your specific research question. This is where it is your job to dig a little deeper.

So, like others who have come before me, today I am going to wander through the quagmire that is the medical literature. In particular, I am going to look at how we navigate the literature as part of a literature review. In the last post, we reviewed the process of establishing your research topic and generating that all important research question. The literature review is the next logical step in the generation of your study design, funding proposal or trial protocol and a critical element in any funding application.


First things first, every literature review should be guided by a central research question for three reasons:

  1. It establishes boundaries for the reviewer
  2. It keeps the review focused
  3. It defines the topic and audience for which the review is intended


Why do we do literature reviews?

A literature review:

  • Provides a background on what is already known about your chosen topic
  • Places your proposed research within the context of your chosen field
  • Presents your analysis, interpretation and synthesis of the existing literature


Ok, time to get cracking! – Top tips for literature reviews

Problem Formulation

  • Define the topic and the intended audience
  • The topic has to interest you, otherwise you have made your job ten times harder before you even start
  • It should be clinically relevant – be sure to ask the question, has it been done before?
  • One of the key elements is manageability. There should be a balance. If a review is too broad or too narrow, it can cause significant issues for the reviewer; striking this balance can be difficult
  • Finally, the more obscure your topic the harder the review will be


Conducting the review

Finding the right studies is a critical element to a successful literature review. This requires you to embark on the dreaded search. The ever-familiar feeling of drowning in the literature is one we can all relate too. So, here are a few tips to keeping your head above water.

  1. Examine your resources, who can help you?
    • Your supervisor or colleagues
    • An information librarian (they do this as part of their job)
    • And remember, it is not cheating or ethically questionable to enlist the skills of a specialist. If you’re a Dermatologist you would not scrub up to undertake cardiothoracic surgery, would you? This is a similar situation with obviously less significant repercussions.
  2. Make a list of search terms related to the topic and the research question
    • Get a colleague/supervisor to check these, they may identify terms you have missed or did not think of
    • Remember some terms will have different spellings, depending on the study’s country origin, for example: Paediatric in Europe is Pediatric in the USA. Simple but crucial
    • Once you have established these terms make sure to keep a record of same. This will save time going forward
  3. Databases – this is where you will find the bulk of the literature
    • You will use multiple databases however, they should be relevant to the topic of interest. For example: If you are conducting a study on the use of pre-hospital TXA in stroke patients you will not conduct a search in PsycINFO or PEDro.
    • Keep a record of the search which yielded the most relevant results, as you can copy and paste it across multiple databases
  4. Your searches as a whole could result in thousands of studies, this is the point at which some researchers lose their way. This is because you’re afraid of excluding that one seminal paper which in turn voids all your hard work. Be not afraid my friends, there are a number of ways to reduce the chance of this happening:
    • Be thorough and remember the focus of the review
    • Develop an eligibility criteria for studies you deem fit for inclusion in your review
    • Read study abstracts carefully
  5. Once you have identified your eligible paper/studies there is a secondary search which should be completed. This involves:
    • Looking at reference lists of eligible studies/reviews earmarked for your review. This process could highlight studies you have overlooked
    • Also, to circumvent a massive meltdown due to the sheer volume of studies to review, now is the time to enlist the help of a citation manager such as EndNote or Papers
    • And don’t forget the importance of seminal papers/research as they will always have a place. Dismissal of these key papers could have an effect on the validity of your review as a whole
    • Now you have identified the studies for the review, it’s time for the hard part, the term that strikes fear into the hearts of all researchers – the critical appraisal of the literature


Evaluating and analysing the data

  • Critical appraisal is not just summarising the literature. As a researcher, you are expected to offer your readers:
    • An interpretation of the literature by actively participating in the review and rationalising your opinion
    • Synthesising the literature – this is a process of structured critique of the relevant work and your coherent argument of the existing literature and how it relates to your study. This process creates an integrated whole for the reader.

This part of the review can be quite intense and can often be hampered by the reviewer becoming overwhelmed and losing focus. Following a structured, systematic approach to critical appraisal can prevent the researcher from meandering off course.

There are a number of critical appraisal tools which will facilitate this process:


Writing the review

How you present your review will very much depend on your own personal writing style. There should be a systematic structure to the delivery and the discussion supporting this argument, and it should be engaging for the reader. Presenting a critical appraisal of the literature can be a fine line to walk professionally. Just because it is a critical appraisal does not give you a licence to be overtly critical though nobody wants to read a review that presents the literature but does not offer any valid interpretations. It is your job to strike that balance being as objective as possible and offering a coherent synthesis of the literature which will be beneficial to the wider community.


A comprehensive review, like any research study, will have highs and lows. But it is the hard graft that you put in that will make it worth it. So, when you are sitting in front of your computer agonising over your interpretations of the literature just remember…

If we choose to change nothing, then nothing will change


There is one final thing I would ask every author to do before you press the save button and this is where you must be entirely honest with yourself. Does this review answer our central research question? The answer and the actions you take related to this question are 100% up to you.

Probiotics in review

Cite this article as:
Henry Goldstein. Probiotics in review, Don't Forget the Bubbles, 2016. Available at:

Like almost every other human entering a pharmacy in the ten last years, I was offered some probiotics when I collected a prescription recently. On my walk back to the car I mused about the evidence behind the shop assistants attempted up-sale. I reminded myself of the use of probiotics to prevent necrotising enterocolitis, and was starting to think of some other indications. Some days later, this review by Hania Szajewska in the Archives of Disease in Childhood popped up; here’s a precis of an excellent paper:

Szajewska, H.What are the indications for using probiotics in children? Arch Dis Child. Published Online First: 7 September 2015

Probiotics are “live microorganisms that, when administered in adequate amounts, confer a health benefit on that host.” The most common strains used therapeutically are the lactobacillus strains L. reamnosus GG (LGG) and L. reuteri DSM 17938 as well as bifidobacterium and saccharomyces. There are also some novel probiotics in development.

Probiotic preparations differ to standard medications as the dose, viability and even agent (organism) are harder to control. There is significant industry influence and, in my opinion, therapeutic development has likely suffered at the expense of populist marketing. Research into probiotics is strain specific; with that comes the challenges of extrapolating the findings to any over-the-counter product. Specifically, probiotics are not regulated as drugs, hence significant concerns exist with respect to labelling and quality.

In this paper, Szajewska reviews the evidence for a number of paediatric indications for probiotics. I’ve simplified and summarised the findings here;

What do we think works?

Necrotising enterocolitis – Multiple RCTs and a Cochrane review, mostly using L. reuteri DSM 17938 show a reduction in NEC in preterm infants. Additionally, there was a reduced time to full feeds, reduced admission length and reduced rates of late-onset sepsis.

Antibiotic associated diarrhoea – Szajewska references her own meta analysis – albeit primarily an adult population – which identified a NNT of 13 for antibiotic associated diarrhoea; the database is predominantly adults. Most effective probiotic agents for this indication are saccharomyces boulardii and LGG.

Infantile colicL. reuteri DSM 17938 was assessed in 4 RCTs; their combined results showed that the use of reduced crying times in breastfed infants with infantile colic. In one analysis (3 trials), L. reuteri DSM 17938 vs placebo reduced crying times at 21 days of life by an average of 43 minutes/day. Probiotics appear more helpful in breastfed by comparison to formula fed infants.

Functional abdominal pain – A meta analysis of LGG for a range of abdominal pain-related functional gastrointestinal disorders (FGDs) showed that LGG was significantly better than placebo in this population, with a NNT = 7. Szajewska doesn’t appear to have much faith in these results with respect to FGDs as a whole, but notes that patients with Irritable bowel syndrome showed the most benefit (NNT = 4).

Acute gastroenteritis – ESPGHAN (the European Society for Paediatric Gastroenterology, Hepatology & Nutrition) recommend consideration of probiotics (LGG > S. boulardii > L. reuteri DSM 17938) for children with acute gastroenteritis, in addition to hydration therapy.

What might work?

Nosocomial infection – The review considers a number of nosocomial infections, and briefly mentions the importance of rotavirus immunisation, where available. A handful of trials showed that probiotics (LGG) vs placebo had no significant differences for risk of post-admission diarrhoea in children under 2 years old; the results contradict some earlier trials in this area, which showed promise.

Prevention of allergy – This is controversial – two studies published by opposing peak bodies disagree. This includes maternal probiotics to reduce long-term outcomes.

H.Pylori – May improve eradication rate, but limited evidence in children.

IBD – Some evidence for inducing remission of Ulcerative colitis; insufficient evidence in Crohn’s disease. 

What doesn’t work?

Functional constipation – no evidence of benefit; not recommended via ESPGHAN 

Within the review, two positive studies jumped out at me, so I went back to the primary literature for a deeper dive.

Firstly, I was fascinated by the idea of preventing infections in daycare centres – Szajewska’s overall verdict was that there was not currently sufficient evidence, but that LGG and L. reuteri DSM 17938 may have some effect on community-acquired infections. Particularly, the review describes this study;

Gutierrez-Castrellon, P., Lopez-Velazquez, G., Diaz-Garcia, L. et al. Diarrhea in Preschool Children and Lactobacillus reuteri: A Randomized Controlled Trial. Pediatrics Mar 2014, peds.2013-0652; DOI: 10.1542/peds.2013-0652

P: 336 children born at term aged 6-36 months attending a daycare in Mexico
I: 5 drops L. reuteri DSM 17938 for 12 weeks
C: placebo drops
O: The primary outcome was the number of days with diarrhoea per child, which was defined as days when 3 or more loose or watery stools were passed within a 24-hour period with or without vomiting, both during the intervention and for 12 weeks afterwards.

  • About ¼ of families offered enrolment decline; which means we should question the (?social) acceptability of the intervention in this population.
  • Semi blinded – one of the authors was overseeing the block-randomisation.
  • Interesting exclusion criteria including birth weight < 2500 g, chronic disease, failure to thrive, allergy or atopic disease, recent (previous 4 weeks) exposure to probiotics, prebiotics, or antibiotics, or were participating in other clinical trials.
  • A reasonably well defined list of secondary outcomes.
  • Parents were educated about stool descriptors using the Bristol stool scale, and upon a loose motion had to contact the study centre, and then report for assessment. I wonder if this call-presentation process lent itself to underreporting (in both groups.)
  • All four primary outcomes: Number of diarrhea episodes, Episodes of diarrhea per child, Mean duration of diarrhea episodes and Days with diarrhea per child were significantly better in the treatment arm, both during the intervention and afterwards. With p values ranging from 0.03 to 0.01.

Secondly, the idea that probiotics could reduce infantile colic seemed immensely appealing; it’s an area that has had a myriad of debunked therapies over the last several millenia. T

Indrio F.,Di Mauro A., Riezzo G., et al..Prophylactic use of a probiotic in the prevention of colic, regurgitation, and functional constipation: a randomized clinical trial. JAMA Pediatr. 2014 Mar;168(3):228-33.

P: 589 term infants aged less than one week, in 9 centres across Italy.
I: 5 drops of L reuteri DSM 17938 (1×10^8 cfu) for 90 days
C: placebo
O: Primary outcomes were daily crying time, regurgitation, and constipation during the first 3 months of life. Cost-benefit analysis of the probiotic supplementation.

  • Infants receiving antibiotics in the first week of life were excluded; (in Australian maternity units, this would account for a significant number.)
  • Trial was independently randomized and double blinded.
  • Around ⅙ patients were lost to follow-up; a significant number were withdrawn from the treatment arm for protocol violations by the investigator.
  • Parents recorded data in a structured diary and sought advice as required via usual channels.
  • At both one and 3 months of life, the infants in the treatment arm cried for significantly shorter periods of time and stooled more frequently. At three months, there were fewer episodes of regurgitation in the treatment arm.
  • Although this is a single study, there are a number along similar lines; Szajewska’ paper mentions 4 in total. The results are most striking in this paper, hence my curiosity.

The organism of the hour, L. reuteri DSM 17938 was first cultured from breast milk of a Peruvian mother; it is patented by BioGaia whom provided the study drug and placebo for both trials above.


There’s a growing body of evidence for the potential benefits of probiotics in a number of paediatric conditions.

There is a bias in this review (and pretty much all of academic medicine) towards positive trials. That being said, I haven’t given the details of every study mentioned; Szajewska’s review does so nicely and I also recommend a read of the primary literature.

Most importantly, communicating with parents about the uncertainties about over-the-counter probiotics with respect to labelling, quality, dose and organism remain central to this discussion.

Key points:

  • Probiotics are “live microorgnaisms that, when administered in adequate amounts, confer a health benefit on that host.”
  • There are many vested interests & popular marketing with issues around labelling and quality in this area.
  • Research is strain specific.
  • Main strains researched are; Lactobacillus reuteri DSM 17938, Lactobacillus reamnosus GG (LGG), Bifidobacterium and Saccharomyces
  • Presently, benefit has been demonstrated in NEC, Antibiotic associated diarrhoea, infantile colic, functional abdominal pain and acute gastroenteritis.


Szajewska, H.What are the indications for using probiotics in children? Arch Dis Child archdischild-2015-308656 Published Online First: 7 September 2015

Gutierrez-Castrellon, P., Lopez-Velazquez, G., Diaz-Garcia, L. et al. Diarrhea in Preschool Children and Lactobacillus reuteri: A Randomized Controlled Trial.