Rx News

What's the Evidence for Evidence Based Medicine?

Anne Sagalyn, MD

November 1, 2015

Evidence based medicine (EBM) sounds like a good idea. Developed as a teaching method for medical students, it quickly gained traction within the medical establishment. Described by David Sackett, in a landmark 1996 British Medical Journal article, as "the judicious use of current best evidence in making decisions about individual patients, integrating individual clinical expertise with the best evidence from systematic research," critics counter that along the way EBM lost its focus on both the individual patient and the physician experience and expertise. Google EBM, and failure results in pages of articles excoriating EBM. Search for "EBM and success," or "EBM and positive results," and very little appears. How did we get here? Isn't EBM what physicians have always done? Haven't physicians always used the best available evidence in concert with patient wishes and clinical expertise to make decisions? Why do so many of us mistrust EBM?

What's in a Name?

The answer begins with the name, which physicians find both amusing and insulting, as if in the dark ages before EBM, physicians practiced without evidence. EBM's power comes in part, from branding. Imagine if EBM were named randomized controlled trial (RCT) based medicine, or systematic trial based medicine-it would have gone nowhere.

Do Experienced Clinicians Need EBM?

For novice medical students, who require formulaic, rigid methodology to learn patient care, EBM makes sense. For experienced clinicians, RCTs and systematic reviews are critically important, but so are other lines of evidence. Clinical intuition is evidence; patient goals and desires are evidence too. Observational studies are evidence. Physiologic mechanisms are evidence. Experience is evidence. What physician hasn't treated the patient who stubbornly refuses to improve as physician and patient march down the list of evidence based treatment options, only to finally improve when the clinician steps out of the box and devises a treatment that works. The art and alchemy of medicine reside not in evidence based guidelines, but in the mind and expertise of committed physicians.

The Triumphs of EBM

EBM transformed AIDS from a death sentence into a chronic disease. The women's health initiative proved that, contrary to widespread clinical practice, estrogen in menopausal women caused harm. EBM is responsible for changing the calculus of cancer treatment, both in the discovery that some malignancies are best treated with watchful waiting, and in the expansion of treatments for the truly life threatening malignancies. Several orthopedic procedures were found to be no improvement over the tincture of time. Sadly, EBM is no match for Jenny McCarthy, the anti-vaccine activist (and one who does not let scientific evidence get in her way) who has a SiriusXM radio channel as well as a reality show coming later in 2015.

The Gold Standard: RCTs

Smallpox yielded to Edward Jenner's observational studies of cowpox vaccine. Penicillin-cured gonorrhea was also the result of simple observation. The history of medical progress rests on observation, and observational studies suffice when the question and answer are obvious: do vaccines work? Is it better to surgically remove gangrenous tissue? RCTs are a science of marginal gains, or the "low hanging fruit (interventions that promise big improvements) that were picked long ago."

Large-scale RCTs wield considerable statistical power, which may conflate statistically significant, but clinically unimportant, effects. "RCTs evaluating treatments for cancer are reporting smaller incremental benefits than previously, amid growing recognition that RCTs underestimate and under-report harms from new cancer therapies." The majority of RCTs are funded by industry, with all the inherent biases. Industry funded studies over-report clinical efficacy while underreporting adverse effects. RCTs are a closed system, that is, they measure, by design, the truth of a hypothesis (this treatment is better than no treatment) for a tightly defined group, eliminating most but not all confounders. RCTs enroll a select population-younger patients without multiple morbidities. The trial results don't generalize to the population at large

The Hawthorne Effect

An understudied phenomenon, the Hawthorne, or observer, effect comes out of 1920s industrial science. A manufacturing plant, trying to improve production, tried various schemes. Shorter workdays, longer workdays, different lighting. No matter what the intervention, production increased, although not for the long term. Ultimately, the interaction with the observer/scientist was what improved productivity. Similarly, in medical research, a paradigm exists that may increase the apparent benefit of a treatment, at least for the length of the study. Who signs on for experimental treatments? Better educated patients who have failed conventional treatments and are eager to be cured. Who treats them? Friendly researchers, anxious to keep patients in the study. This is a new area of study, with conflicting findings, but it makes intuitive sense. If you are nice to your experimental subject, who is eager to improve, she may be more likely to improve for the life of the study, thereby yielding false positive results.

"Why Most Published Research Findings Are False"

This grim verdict is the title of a paper published in the journal PloS Medicine in 2005 by Dr. John Ioanides, a Harvard trained mathematician and physician who studies the studies. His paper, the most downloaded in the history of the journal, elaborates the reasons why studies are less likely to be true, in the following scenarios:

• If they are small.
• If effect sizes are small.
• In hypothesis-generating studies versus confirmatory studies.
• If the design, definitions, outcomes, and statistical modes in the study are flexible.
• if there are financial interests involved.
• If the study is of a hot scientific topic.

To paraphrase Winston Churchill, EBM is not the best system of medical practice,but it's the one we've got. Some years ago, I treated a frail, elderly man living a lonely existence. His wife and friends had died. His children lived far away. He was recently diagnosed with prostate cancer and a steeply rising PSA. His urologist wanted to treat with hormone therapy. My patient did not want the treatment and understood he would likely die of prostate cancer without treatment. His one pleasure in life was flirting with female volunteers at a soup kitchen, where he volunteered on Sundays. Losing his libido was a non-starter. Plug this man into an EBM guideline and get what may be a reasonable plan for some male patients. For my patient, RCTs and systematic reviews were meaningless, if they didn't take into account how he wanted to live and die. The challenge for EBM is keeping the patient front and center while negotiating the vagaries of the evidence.

SOURCE: