Say what?
You heard me…Don’t bother reading the research!
Are you wondering why I would possibly say something like that? I mean, speech-language pathology IS a science based career, right? Anyone who has read me here or on Facebook/Twitter, knows that evidence based practice and research is something that I drone on (and on and on) about…so why on earth would I possibly say don’t bother?
Reverse psychology
See…I have been fighting for research and research-based practices for a long time. I’m constantly talking about pseudoscience and the need for critical thinking and not being led by anecdotal evidence and I am constantly being shutdown by my peers. So out of desperation, I have turned to reverse psychology.
I (and others) go on and on about the need for research to direct our services and the importance of applying critical thinking to treatment methods, and I am ignored (by many) and the anecdotal evidence continues to be spouted and loved and practiced while the research is ridiculed or ignored. By now, I can almost guess which posts will generate the most anecdotal evidence and arguing for/against research. I can also (usually) tell which posts will generate the most discussion among which type of clinician. For instance, a question about PROMPT or some other expensive method will have lots of discussion among the private practice people while something that claims to save time (such as speech buddies) is big among school based clinicians. (Of course there is some overlap…but I’ve started making a game out of which type of posts will generate which responses.)
For instance…
On Facebook recently a post came up regarding Therapeutic Listening (but really, you can change Therapeutic Listening to any other treatment method such as Prompt, NSOME, Facilitated Communication, Rapid Prompting, kinesio taping, etc. and have the same conversation). During the discussion the lack of research based evidence was pointed out. For this particular post, there were links posted to research showing it wasn’t effective…but there were still the arguments supporting it.
“But I’ve seen it work.” “It makes sense.” “It can’t hurt…why not try it”
Another SLP brought up potential backlash of arguing against non-EBP practices. Her argument was that when individuals state a program is not evidence based practice, that there is a decided lack of research to support the efficacy of a treatment (or heaven forbid research that actually shows it is NOT effective), it can harm the profession because perhaps, maybe, years down the road, it may become evidence based but people won’t stay current so they’ll still think it’s not EBP.
I find myself…conflicted. In some ways, she is correct. To boldly state something is not EBP is a bit … negative (it’s true…but negative). We could say “there’s no research to support that yet, but they are working on it.” That’s certainly far less negative…but, it also sounds like giving tacit permission to go ahead and use the treatment method. That obviously isn’t the right thing to do either.
I understand and fully support the need to keep an open mind. I agree that there are many treatment practices that are in their infancy and with limited funding it can be challenging to get research completed. I also understand that just because something is pseudoscience doesn’t mean it doesn’t work…it may work, for some. However, we have to clinically evaluate – does it work BETTER than a researched method. Not does it work, does it work BETTER. If it doesn’t DON’T DO it.
I have heard the argument that “it’s what parents want so as a private practice SLP I have to do it or they’ll just pay someone else.” I’ve had another private practice owner (an SLP) tell the group that she wants all of her new hires to be certified in certain programs (despite the fact that they are controversial and non-EBP) because parents want them even if they don’t work.
What does that say about us? About our profession?
Why is it that so many of these controversial treatments are SO very expensive that to get trained to do them correctly (which is necessary for outside research to be valid) is prohibitive? Do they do that on purpose, so that outside researchers aren’t able to show that it is ineffective? There are other programs that are so convoluted in how they should be done that any research that is completed and shows it as ineffective could easily be discounted because the treatment protocol wasn’t followed exactly or there’s too many nuances (e.g., according to one SLP choosing the exact right music and prompts for TL falls into this category).
In some ways, I have to applaud the creators of some of these products. They are raking in the money on trainings, materials, certifications, etc…all with the secure knowledge that even though there is no research to show it works, someone somewhere will buy it because it’s marketed well…because parents who are desperate to “fix” their child ask for it and SLPs who are desperate for business buy into it or need to please the parent.
When did it get to be accepted to not only look for research to support our treatment methods, but to turn our backs on research that shows our chosen methods don’t work? When did the responsibility to our clients change?
I’d love to hear your thoughts regarding research and our responsibility.
Until then…adventure on!
Mary
Speech Adventures 
Thanks for saying this Mary! The use of non-EBP methods in our field is certainly a problem. So many SLPs don’t realize how heavily they’re being advertised to by companies who use very little evidence to support the creation of their products or programs.
I’ve been following you and other bloggers for years, and I think there are a good handful of you well-read folks who have made a FANTASTIC difference in shedding light on common non-EBP.
Going forward, there are a few things that I’d like to add that I think are important for remedying this problem.
1) This, not that.
Remember those “Eat This, Not That” diet books that were popular about a decade ago? That’s always the image that pops into my mind whenever EBP issues come up. I think many of us who are trying to make research more clinically visible need to make a habit of regularly providing a substitute practice instead of simply saying “That’s not evidence-based, so you shouldn’t do it.” Saying “Don’t do that” isn’t nearly as helpful as providing the person with something to do instead. I’m definitely guilty of telling people not to do something without also coaching them on what they should do. Why? Because it’s easier and faster! But I do think we’d solve more problems if we tried to add this component in.
2) Remove the shaming
We HAVE to remember the psychology behind trying to convince someone of something. If we use shame (“That’s not EBP, I can’t believe you can’t know that, it’s terrible that you’d use that,” etc.) we foster anger, bitterness, and ultimately people stop listening to us and instead choose to just “take sides” and go hang out with people who make them feel good about themselves. And then, we’ve lost our audience.
3) Acknowledge the good in (or WHY these techniques are being used at all)
Finally, we have to recognize the good in some of these techniques. Some of them actually have a few good components that ARE evidence-based, and we need to acknowledge these. Otherwise we risk seeming blind to the full picture. We also must recognize the good in because, in order to get a stranger to listen to you, they must trust that you’re approaching them from a place of understanding, rather than annoyance. Finally, some of these techniques are total quackery and the only “good” aspect is that they benefit the clinician (e.g. saving them time, money), not the client. We also have to talk about these factors, too. Perhaps even more so! When we don’t discuss these issues, the person trying to make a decision has a MUCH harder time trying to reconcile why some people vote “yes”, while others “no”. What helps us make sense of an issue is by actually talking about all of it. We have to acknowledge, “I know this technique REALLY saves you time, but…” so that the person making the decision on whether or not to use a practice can see the motivation behind why some people do use it. And, ultimately, identifying the reasons why people use certain techniques despite a lack of evidence only helps us make headway in finding an appropriate solution.
So, in general, I think if the conversation is honest, kind, and above all always provides something the clinician can do instead, I think we’ll start seeing even greater improvement in clinicians using research-based techniques in their therapy rooms.
P.S. Mary, serious kudos for the amount of continuing education and reading you do in order to better both your own clients and all of us as SLPs. I know it’s a LOT of work, most of it on your own time! I think ASHA seriously needs to consider setting aside some funding to sponsor the work of folks like you who are highly involved in peer-to-peer training 😉
Meredith Harold, PhD, CCC-SLP
http://www.theinformedslp.com
Thank you for the kind words and wonderful comment Meredith. You make excellent points regarding ways that we can help stop the overuse of non-EBP methods.
You are absolutely right to simply make a blanket statement of XYZ is non-EBP and bad isn’t necessarily helpful. Saying XYZ is not EBP, however, research shows doing ABC useful is. The challenge (to me) is to decipher viable alternatives that are EBP without always knowing what the lure is for the other.
Excellent points. Thank you.
I find these kind of rants pretty unhelpful and don’t cover new ground. And targeting independent practitioners is so easy a target. The point about EBP is one that has to be made to encourage new research funding and good design but slating fellow practitioners on a grand scale with no evidence except anecdotes is not a great thing to make public…what does that say about our profession!?
Thank you for your comment.
That’s interesting Hillary that you consider Mary’s carefully worded and thought provoking blog post a “rant”. Nowhere in it did it feel like she was ranting at any point. I also did not feel like she was targeting independent practitioners nor do I feel that independent practitioners are “so easy to target”. Perhaps that is because I myself have a part time private practice in which I assess and treat clients, and I do not find myself targeted in any way, shape, or form.
In fact, our field could certainly use greater accountability than it has right now, as based on the social forum chatter numerous SLPs continued to buy into and use non-evidence-based practices. I find Mary’s blog post highly useful and relevant to all parents and professionals who may be interesting in doing their own independent research of whether a particular pseudoscientific program or method actually work.
“However, we have to clinically evaluate – does it work BETTER than a researched method. Not does it work, does it work BETTER. If it doesn’t DON’T DO it.”
That carries for basically everything.
Well said. Excellent article.
I sure wish I could choose what the researchers choose to research. The reason so many people do things that are not evidence based is because so much of what we do doesn’t get studied. If I find something that works, I use it. If it doesn’t work, I don’t. I have found PROMPT and Talk Tools to be extremely helpful for many of my patients. Is it EBP, probably not. Has either been well researched. Absolutely not. I can’t stop people from ridiculing people like me who choose to use programs that don’t have strong evidence behind them but I am frustrated at how many clinicians are close minded about approaches that haven’t received a thumbs up from some research team. They don’t try them, they just make fun of people who do. And, I believe that the people who create these programs believe in their products, for the most part. They believe they are helping people because they have seen it work, not just to make a buck. I read research but I also use my clinical experience to make decisions, especially when none of the researched approaches are working.
This is from ASHA re: EBP for apraxia of speech:
Research Needs
The Committee’s primary conclusion is that, as with many other complex neurobehavioral disorders, research in CAS has not provided clear answers to the following five interdependent questions: (1) What are the biobehavioral origins of CAS? (2) What methods to diagnose CAS are valid and reliable for children of different ages and with co-occurring problems? (3) What is the prognosis for children with CAS? (4) What are the most effective ways to treat CAS? and (5) What might be done to prevent CAS and/or mitigate its impact on other areas of development? The Committee’s most compelling single finding is the lack of consensus among investigators on the core diagnostic features of this disorder, thus limiting the utility of all research on optimum assessment and treatment. That is, lack of one or more necessary and sufficient diagnostic markers of CAS limits studies of the origins and neural substrates of CAS, and in turn, the scope and depth of our guidelines and recommendations for service delivery issues.
The Committee’s second conclusion is that there is a need for large-scale, collaborative interdisciplinary research in CAS. CAS research clearly needs to expand to different, broader types of research models. Most of the CAS findings reviewed in this report reflect the research of individuals or relatively small groups of investigators using small numbers of participants suspected to have CAS. In comparison, emerging findings from research on such widely studied complex neurobehavioral disorders as autism, dyslexia, and language impairment reflect the research of many international, multidisciplinary collaborations. The only such research of this scope on CAS to date is the programmatic studies of the KE family.
Thank you for your comment. In no way was I “making fun of” or “ridiculing” anyone. I was stating my concern that there are many individuals who use programs even when research shows that they do not work any better than any other method. Individuals who, when provided with research that shows it is ineffective, say “I’m going to use it anyway.”
You are right in that it is frustrating that we can’t choose what is researched…except that we can somewhat. We can be an active part of the research. Non-Speech Oral Motors (which Talk Tools falls into the last I checked) has been researched and shown to have no known benefit over any other program.
I have not found any peer-reviewed evidence that PROMPT works any better than any other program. Dr. Rvachew explains the problems with the associated reviews here:
https://developmentalphonologicaldisorders.wordpress.com/2013/11/03/still-no-evidence-for-prompt/
The pathways to research are out there. I’ve posted before on all the ways to get research at little or no cost.
ASHA has links for how to participate in research and why it’s important.
http://www.asha.org/academic/CLARC/
http://sig16perspectives.pubs.asha.org/article.aspx?articleid=1754397
http://www.asha.org/Research/Pathways-Program/
Again, I am not targeting any particular person or type of clinician and I am not ridiculing anyone. I am stating my concern.
Dr. Rvachew also stated:
“The reason is that we found out, when we were using the very excellent Williams, McLeod & McCauley (2010) book for teaching, that students believe that if a treatment is mentioned in a text book that it must be a “good” treatment. Even when provided with information about levels of evidence and so on, the students could not or would not distinguish between treatments that were and were not evidence based.” (see the previous link). I would say it’s not limited to what new clinician read in a text book…but what both new and seasoned clinicians are reading in public forums as well.
By refusing to at least consider the research that refutes certain practices, we are doing a disservice to our clients whether we like to consider that or not.
When considering treatment I carefully review the evidence behind the methods and programs available to me and then ask a self question: Can using a far less expensive yet far more evidence-based method yield better or equal results? So far on all the occasions EBP won out. However, if I were talking anecdotal evidence that I could certainly share a number of stories at the center of which were different children receiving oral motor as well as PROMPT therapies and not progressing one iota both qualitatively and quantitatively beyond the subjective therapists claims. These very same children then switched their respective therapists and made rapid gains, which had markedly different in quality both quantitatively and qualitatively to both parents and independent observers (family members who had not seen the child in several months).
Great conversation! I believe we need to keep an open mind, as our field is ever-changing. Thanks, Mary, for catching our attention. [personal comment deleted]
When I saw the title of this post pop up in my email — I was interested.
Given that I am clinician-scientist (who also teaches an EBP class to SLP grad students every year), to say that I am passionate about the topic of EBP is an understatement. BUT, in a crazy way, I do sort of do agree with the statement — “don’t bother reading the research.” First, much of the research tells us what doesn’t work, rather than what does work. This doesn’t leave clinicians with many options. Second, many evidence-based interventions (take JASPER from UCLA for kids with ASD), don’t have training programs for clinicians. Third, research articles don’t often provide enough information for clinicians to replicate the intervention.
I would like to offer an alternative to finding an EBP intervention, that may not exist or may not be working for your particular tiny person (in my case, which is EI). Single subject experimental design has been used in the field of special education for years. Clinicians (with training) can implement this type of methodology with individual people in order to test the extent to which a treatment is effective. In the absence of EBP, I say find a buddy with a similar issue and take some data — some hard-core, methodologically-sound data. We need more science that originates in field. MDs frequently publish clinical findings, yet it is rare for SLPs to do the same.
So we, as a field, need to change — we need more clinicians pursuing research degrees, we need to publish more treatment efficacy studies and we need to make evidence-based intervention materials more readily available to clinicians.
But I continue to be inspired by clinicians who despite these obstacles strive to find the best available evidence to support the needs of their clients, patients and their families.
Excellent point. I’d love to see more research done.
Mary Huston….Bravo! You are a sane voice in the wilderness of ignorance and greed. I’ve been an SLP since 1976 and am so glad I found your comments. Until now I was feeling like a dinosaur. I suggest that therapists go back and read the masters like Eisenson, Van Riper, West, Perkins, and the brilliant Mildred MccGinnis. You,in the collective sense, cannot “one-up” Bach, Beethoven, and Brahms.
Thanks for your comment Sandra.