Have we turned into pseudoscientists?

Posted in General, research by

This will not be a popular post.  I suspect it will anger a few, for that I’m sorry…A few will write to congratulate me…and a few will simply unfollow me. Regardless, I feel it has to be said. I hope that patience will be in abundance today.

It’s no secret that I love social media. I’ve been active with the #SLPeeps community on twitter since before the hashtag was created (that’s pre-2010 for those who don’t know). I’ve written about using social media to increase professional networking in the ASHA Leader Live several times. But, I’ll have to admit, I have a love-hate relationship with Facebook. I’ve been on several FB SLP groups (and left, and returned, and left, and returned). Today, I was resoundingly reminded of why and rather than leave the group, I’ve decided to vent here. 

pulling hairFirst, let me preface it by saying the FB group does offer some great information. I’m able to answer questions, ask questions, and generally share with like minded helpful individuals. For that, it is truly an invaluable resource.

Each time I’ve left, it’s not because there’s some big blow up…it was because I realized that my ideals of what an SLP is were sinking, that I was disillusioned with my fellow SLPs. Today, I realized why.

Speech-language pathology is a science field. As such, we have an obligation to maintain evidence based practice, do research, and READ the research that is done. What has been increasingly evident is that some (hopefully not many) SLPs have opted to throw the science out in favor of anecdotal evidence instead.

For instance, in a recent post, discussion was held on Retention of students. Ample evidence has shown that for the majority of students, retention is not helpful. It causes self-esteem issues, there are no long-term benefits, and it’s a life-long stain. Research has shown this. There are a few exceptions where retention is helpful – extremely immature social behavior, etc. This was discussed in the posts, but there were several SLPs who were absolutely adamant that the research is wrong and it IS helpful as long as it’s in Kindergarten or 1st (or as one person stated, not in Kindergarten but only in 1st).

We can add in discussions of PROMPT, Interactive Metronome, NSOME, FastForWord, Facilitated Communication, Rapid Prompting, etc (about the only thing we haven’t discussed is climate change!)… and the resounding theme is, I will listen to science when it supports what I want to do…but when it doesn’t, I’m going to say my personal belief trumps science.

An acquaintance of mine sent me this link “There’s a Good Reason Americans are Bad at Science.” In there it mentions how  “Our leaders and wanna-be leaders say that evolution is a myth, vaccines cause autism, and a snowball constitutes proof that climate change isn’t a problem.” I find we’re facing the same thing with the educated people in speech-language pathology. The article discusses how many individuals use statistics (science) “like a drunk uses a lamppost: for support, not illumination.”

Are we, as SLPs, guilty of using the very research that is supposed to drive our field, as a drunk uses a lamppost? Are we guilty of proving that we use science when it supports our goals but don’t believe it’s necessary when it doesn’t?

Don’t get me wrong, I’m very aware that there are exceptions to every rule – there are. Of course it is necessary to consider the client in a gestalt way and recognize that each individual IS an individual. But really, when did we start to say the research doesn’t matter? When did our personal belief begin to outweigh the other two thirds of Evidence Based Practice? Yes, personal EXPERIENCE has a role in EBP. Personal beliefs do not. There, I said it.

I’m saddened by the seemingly complete disregard of research (one SLP stated “all research is anecdotal” as a way to justify her statements). I’m deeply saddened that in our role as the experts in communication we are choosing to throw out the science in favor of the pseudoscience.  Parents and caregivers are entrusting us to take care of their loved ones, to treat them with the latest research and evidence based practices…and we are turning our backs on it in favor of something else.

I’m hesitant to ask for comments…but I think I’d like to see them. What are your thoughts? Am I being overly sensitive today? Are SLPs still scientists? What do we need to do to return to the research? Leave a note here…I’ll don my fireproof underwear.

Until then…Adventure on!

Mary

94 thoughts on “Have we turned into pseudoscientists?”

  1. Mary, you are once again right on! I do think the students coming out of graduate school now are much more scientifically based (I hope so, anyway; my students sure seem to be) than they used to be. I’m all for the art of speech-language pathology, but it must be mixed well with the science of SLP. We are really in the an interesting time…science is being devalued so often–watch what is happening in politics right now. Let’s keep up the good fight and help people to be good skeptics who find it to be valuable to question everything. Questioning is a good thing, not a bad thing. And we need to constantly be good skeptics so none of us fall into traps in thinking.

    • Thanks Greg. I appreciate the kind words. I agree, questioning is always good. Being a skeptic is good. Turning our back on the science is not. It’s interesting that a comment just after yours states that students AREN’T as well versed in research now as they were before. Interesting dichotomy.

    • Dr. Lof, I’m one of your former students and still fighting the good fight in my practice. Keeping skeptical and watching out for snake oil and charlatans. Thanks!

  2. Mary,
    thank you so much for writing this deeply insightful and highly relevant post! I applaud you! All the information contained in your post is observed by myself as well as multiple scientifically minded speech language pathologists on daily basis not just a variety of SLP-based social media forums but occasionally even on the ASHA forums as well. What you wrote will undoubtedly anger a number of individuals, such as the nature of this game. However, you should be incredibly proud of yourself because the content of your post is highly thought-provoking and I’m confident that it will influence numerous battle-hardened as well as newly-minted speech pathologists in profound ways.
    Thank you again for your continuous positive contributions to our profession

  3. Excellent post. I have learned something from my grad student intern: there is a lack of emphasis on research in the SLP grad program she attends (which also happens to be my alma mater). This was NOT the case when I attended. Part of the issue is the de-emphasis on conducting a master’s thesis. No one at the school in question does them anymore. Everyone did one when I was there. In fact, I know other students and recent grads from other universities-none of whom have conducted a thesis study. That lack of background alone contributes to the problem you describe.

    • Interesting thought. I graduated from the top SLP program in the country, where doctoral students wrote theses but masters students did not. We did have a rigorous exam, however, and had the opportunity to serve as research assistants (which I did, as well as a teaching assistant). We do seem to be living in an “anything-goes” time in our country, where outrageous statements earn attention and even votes. Perhaps the pendulum will once again swing to accountability. Thanks to Mary Huston for opening this topic!

  4. Mary-are you throwing out PROMPT, NSOMEs, Facilitated Communication, etc as techniques that you don’t believe in/have no scientific research/evidence to back up or as in techniques that are often argued about via online forums? I’m just curious and wanted to clarify (as someone who wants to take the PROMPT training in the near future).

    • At this time, it is my understanding that all of those (including PROMPT) fall into the pseudoscience category. Now, that doesn’t mean it doesn’t work for some, it means there is no research (that wasn’t created by someone with a vested interest) to support it.

      I too have wanted to be PROMPT trained, I can, theoretically, see where it could benefit some…but with the lack of evidence there’s no way to justify the expense at this time.

      Also, to be fair, I have not done a recent (within the last 6 months or so) search for evidence. There could be something new to support it. As far as NSOME, FC, and FFW, those have all be debunked for quite some time.

      • I really like how you just stressed no research from someone not involved/biased. WHO is finding and directing the research can be just as important as being peer reviewed!

        • Absolutely. It can be a bit harder to ferret out the financial stake but it’s critical. I firmly believe it’s one of the reasons ASHA has pushed for the disclosures before presenting. If there’s a financial interest – it should be disclosed. Heck, if there’s a non-financial interest it should be disclosed.

      • Hi Mary. As someone who tries to keep my feelers out, this is the first I’ve heard about PROMPT not being evidence based. I’m not Prompt trained, but have it on my list of training to look into as I’ve heard good things. I see that you spoke of writing about it at some point- have you been able to do that yet? If not, where would you suggest I start looking into this? Thanks! btw- your post it on point! I feel like half of my meetings lately have been defending research based practice, or talking about gaps that seem to be arising and increasing between grade level standards and what’s developmentally appropriate.

        • I haven’t written it yet , but I’m in the process of gathering information for several pseudoscience posts. There is a lot of research out there, but none, so far that I’ve read that have been written by someone without a financial gain tied to it. However, I haven’t fully read them all yet so I’m still hopeful. Now, just because it’s pseudoscience doesn’t mean it’s trash – it just means there’s it’s suspect and needs to be approached cautiously. It MAY be trash…it may be gods gift to SLPs…but without unbiased studies, we may never know.

  6. You are not being overly sensitive. A former professor of mine recently contacted me for continuing education seminar ideas and I suggested something about evaluating evidence and not buying into pseudoscience. I find much of the education I provide to parents is simply busting myths about speech/language development. I feel similarly about the FB groups. It’s not a replacement for staying current with research, reading texts/journals, and attending continuing ed. courses.

    Thank you for writing about this!

  7. Parents often ask for our input on different therapy techniques, and I always try to refer them to the research first and to talk to other parents who have tried these techniques to get their input as well. There are too many charlatans offering the latest, greatest outcomes at exorbitant prices to desperate families, with no research to support their claims.

    • Yes, I have been put in the postiton of having a private SLP recommend one of the pseudo-science programs, for one of my public school students.(Think multi-level-marketing). Needless to say, it was very expensive and it required extensive training. I tried to tell the parent and my SPED supervisor there was no indepedent evidence of efficacy, but I was over-ruled. I should have refused, but I need my job, so I voiced my dissatisfaction, plugged my nose, and went on with it.

  8. Hi Mary, no argument from me. We are lucky in British Columbia. UBC grads are really taught to critically appraise evidence, At the Centre I currently work in we have to look for evidence on a speaker or topic before applying for professional development funds or educational leave.
    On another note I was surprised to see the evidence for NS-OMEs is listed as strong on the Communication Trust What works website.

  9. You’ve stated my thoughts quite well & we haven’t even had a discussion. My concern is that there is not an abundance of clinical research. I understand the parameters that cause this. I would welcome an opportunity to work w/ a researcher looking for clinical application to theory.

    • I wish we had more opportunity to work with researchers myself Ruth. In talking with other school-based SLPs (although I’m sure others feel it too), there’s a definite lack of time and knowledge on how to get started with research. Having someone to walk through it and guide would be fabulous.

  11. Yes! You took the words right out of my mouth, Mary! In graduate school, we had an entire semester class devoted to learning how to dissect research studies to determine if they were actually good studies and if they seemed to test what they said they were testing. It was an eye-opening experience for me at the time realizing how statistical data can be murky and some scientists can use it to support their hypothesis without there being adequate data for that “support”. So from the very beginning of my career, I felt a bit skeptical when reading research articles because I wanted to know exactly how the research study was done, what types of data they used based on the type of experiment completed, and if that data actually showed any difference or support their hypothesis. What I find is the most difficult pill to swallow, is when there are numerous good research studies debunking myths and ineffective practices and still I see professionals “open” to the use of these techniques, strategies, “treatments”. Honestly when you want to be fair to parents, it becomes very difficult when another professional in your field is supporting something that has been debunked numerous times. It certainly becomes a sticky situation to be in as I try very hard to respect and collaborate with all professionals. We all can’t be researchers, bc we need clinicians. But we need clinicians with a good understanding of research and we need to take opinions out of the equation. Of course research changes over time. We find better more effective techniques and strategies to use, the more we learn about the brain and various disorders or disabilities. However, until then we cannot support the use of techniques or treatments that have so far been proven ineffective. So thank you for this post. It’s very helpful. It keeps us honest as a profession and helps us all become better SLPs.

    • Exactly. Not all research is created equal…but when we are faced with research that doesn’t support a chosen intervention we can’t just choose to ignore it. We need to really look at it and critically appraise it.

      It does become a sticky situation between being respectful of another SLP and being aware of the research. If a medical doctor did this, he’d be laughed out of hospital privileges and slapped with lawsuits right and left.

  12. Mary I always want to hear what you think, even when we disagree. On this we do agree. What we do has to be grounded in science in research. We do not serve our students and families when we chase each new bell and whistle (speech buddies). Keep fighting the good fight.

  13. Great post. I would also add that reading about real research second hand (e.g. summarized by someone else in the news, on a blog, whatever) is also risky and many fall into that trap.

    In undergrad we had to find a summary of new (peer reviewed) research anywhere in any media (ahem, the Internet was still very early so it was mostly newspapers and magazines – blogs didn’t exist yet haha).

    We had to read that person’s summary and reporting on the research, then find the actual article and discuss the innaccuracies. Someone raised their hand and said “what if there aren’t any innaccuracies?”. My (3rd year developmental psych) prof laughed and said “I guarantee that there is NO SUCH THING!”. She was right, of course. Great assignment I’ve carried with me since.

    • I am actually planning a writing a post summarizing someone else’s article and I’ve done a few posts like that in the past. Such posts are important because many notable scholars publish fantastic and valuable sources, which immediately fall into obscurity because a vast majority of SLPs don’t read them. I have seen time and time again of how a thoughtful review can really highlight main article points and really disseminate the work of the scholar and make it accessible to the ‘masses’, who are much more apt to read a blogger’s review on social media rather then search for a research post in an SLP journal. To ensure utmost accuracy of interpretation I provide a direct link to the article to offer the readers the opportunity to review it with a fine tooth comb.

      • Oh, I’m sure Tanya is not saying we shouldn’t disseminate research and post about it. She’s saying that after reading that post – don’t necessarily take it as written in stone. That sometimes the interpretation is off from what the author truly meant.

        I agree with you there are a lot of great research articles that get lost because many SLPs don’t read them. I for one love your posts on articles about CAPD and FASD.

        I love to read posts about research – and then read the research myself. But, similarly to citing research that you didn’t actually read – it’s not the best standard to read a blog post and accept that as the gold standard. It’s always best to followup what we’ve read by clicking the link and reading the article ourselves and understanding their research for ourselves.

  14. Thank you for posting. Your article is well thought out and thought-provoking. So many want to get on the proverbial bandwagon to try and help our clients but end up possibly hurting them instead. EBP is the best way to go. Unfortunately some SLPs in the schools do not choose to continue to do the research.

    • I don’t know that it’s just in the schools (although I will admit that’s where I’m seeing the most of it right now simply because that’s the group I belong to). But, I also see private practitioners eschewing the research in favor of training that put them a step beyond their peers in an effort to gain clientele. I don’t know if hospital based SLPs have a similar problem, but I’m suspecting they do. It’s almost like once we start earning a paycheck we figure the research doesn’t matter as much anymore.

  15. Mary I love this post so hard! I’m so tired of teachers asking me to “just give them” the exercises the child needs to “get better.”

    Honestly, if I had a wand don’t they think I’d have used it?

  18. You win the award for most thought provoking blog post of the year! This is really well thought out and I can tell you have been pondering this over for quite some time. I agree!

  19. Thanks Mary for saying what needed to be said: you are not the only one to have that approach repel experience with SLP related Facebook groups.
    At the risk of seeming like an advertisement, speechBITE.com is one way for clinicians can find treatment research. Free, evaluated and collated database of the SLP treatment research. Clinicians will still need to read the research though, all we do is make it easy to find.
    I will be as sharing this post with my students on Monday. Well done.

  20. I strongly agree. In fact that is why I pursued a clinical doctorate. The quality of work performed by many SLP’s is becoming poor and is diminishing the efforts of the SLP’s who use EBP. There are likely to be many individuals who disagree but the field needs to move towards a clinical doctorate requirement.

    • The argument was that we needed to move to a master’s because it was more rigorous and showed greater reliability… Now we require a master’s and the quality diminishes…so we need to increase it to a clinical doc? What happens when that becomes commonplace as well and the quality goes down?

      Note…I’m actually not against the idea – I’m currently looking at how/where to pursue my doc, but I’m not sure that requiring one would reap the benefits of better quality of work. I suspect complacency would again, rear it’s ugly head…probably more so because the workpool would be significantly reduced and caseloads would be even higher than they are now leaving no time for the desired research.

  21. I agree! I’m not sure about US prices but in Australia it makes me so mad when private SLPs recommend debunked interventions and charge very expensive fees to carry them out. 😡😡😡

  22. Thank you for writing this! I had this conversation with a couple of SLPs about this the other day. I wish there was more good quality impartial research out there. That way, hopefully, SLPs will be better informed and be able to make more appropriate clinical decisions. I find it’s usually with parents that I have to have these conversations (e.g. FastForWord). Parents come and say, “have you tried x?” or “why haven’t you tried y?!” I’m finding those can be my hardest battles. Some parents and SLPs want the miracle cure and often there isn’t one.

  23. I haven’t thoroughly read all the comments, so perhaps this point has been made, but part of the issue, I think, is that there aren’t enough hours in the day. I work in a school setting and I have negative 17 minutes left each day after seeing my kids and doing paperwork. Our scope of practice is so varied that I think it’s also a challenge to keep up on the research in every area (voice, dysphagia, dysarthria, language, articulation, phonology, etc). What I have found to work for me is to do research on a case by case basis. As I come across something new, something maybe more dire, then I’ll research it. When you consider the ease of getting online on any number of devices and searching for whatever the topic at hand is and the extreme number of responses that result it could be difficult for some to discern credible vs. questionable sources. You’ve brought up a worthy point though.

  24. Pardon my handle, as this is a pre-graduate school word press. I would also like to point out that we are in an era where making money off of ones ideas is more important than publishing them. So while I suspect that we could actually get some really good experimental small N data using the PROMPT method; that would mean ‘revealing’ the million dollar technique. If you give me well-written research articles, I can replicate your results and there is no need for “certification”. I’m just tossing this out there as another possibility.

  25. I am in total agreement! Thank you so much for having the courage to speak up! You rock!

  26. Excellent post! I so appreciate your professional integrity in writing this article. Cherry picking the art and science of clinical practice is a shaky platform whe we are dealing with real people with real lives. Your thoughts are greatly appreciated and I believe your assessment is right on.

  27. I always get excited, Mary, whenever you preface a blog with the words, “I am probably going to anger someone…” because it means a thought-provoking and usually spot-on assessment of some facet of our field. We need more forthrightness about situations such as this, as a reminder to everyone about what our code of ethics requires as well as just doing the right thing. Access to valid and reliable research is critical to us and any tips on where and how to use it will be greatly appreciated. One of the best courses I took in university was “Research Methods” (in my Psych minor, but the COMM DIS department had one, too) – it gave me the knowledge and skills to assess research to determine validity. Finding that research and having the time to fully consider it and apply it to our cases is another matter. I would LOVE to do research, but I have to make a living, and we were always told that SLPs with Master’s degrees made more than PhDs. It’s a shame because we were also told there is a HUGE need for all kinds of research to be done. Thanks for stepping out there and saying what needs to be said.

    • Thank you. Megan Roberts commented and linked to CLARC as a way for clinicians to pair with researchers to do more research out in the field. I think that’s one way to do it…Like you, I’m rather fond of my paychecks and need to work. But there’s nothing saying we have to be employed by a university and earning or sitting on a PhD to do research. We can do it now if we have the right resources. 🙂 I’m thinking a brush up on my psych stats and Research methods would be a good thing.

  28. Thank you! More professionals need to speak up about this issue. I’m accused of being non-collaborative, intolerant, etc. every time I point out that we should actually be making decisions based on science and not on how strongly we feel about an issue. It’s very irritating.

  29. Mary — these are exactly the conversations that our field needs to be having. I am so very excited to read this post. As a researcher and clinician, I am constantly frustrated by the gap between research and practice. Often we forget the immense responsibility we have to children and their families when it comes to making treatment recommendations. Without our guidance, families are more likely to make decisions based on a compelling anecdote posted on a blog than based on means and standard deviations reported in a research article. It is our (clinicians and researchers) job to make science just as compelling as pseudoscience. In the absence of an evidence-based intervention, we should consider collecting our own data to evaluate outcomes in a systematic way.

    I would also like encourage people to use ASHA’s CLARC to find researchers interested in partnering with clinicians (http://www.asha.org/academic/CLARC/). I know that I use it to partner with clinicians in the Chicagoland area.

    • There was a conversation on FB just yesterday about finding researchers willing to pair with clinicians to begin new research… (and for the record, although I know it’s necessary, particularly in this conversation, I really dislike the gap we have in the career between “researcher” and “clinician” as though it’s nearly impossible to be both.) Thank you SO much for this post and I will definitely pass the word.

    • I am glad you mentioned ASHA; it seems that some of our expensive dues could go toward researching these issues.

      • Elana, what do you feel ASHA could be doing differently about this? Do you mean that the clinicians/researchers who are willing to take the time to conduct the research and write should be focusing on certain problems or something else?

        I’m not sure that it’s ASHA’s job at all…honestly. They are already providing information that SLPs are choosing to ignore and they are providing research opportunities by pairing interested individuals through CLARC. It is up to the individual SLP to study, critically appraise, and follow through with the research that is provided.

        With that said, I *DO* wish that there were more research available. I wish more people were conducting it and I wish that it covered the information I need. Sadly that won’t happen until we have more people willing to take the time to do the research and submit it.

        • True… ASHA does gather research, for example, re: telepractice. But for some of these controversial approaches, I need to find out if ASHA is taking a stand for/against. Thanks again for opening the topic!

  30. Agreed!

    Just to play devil’s advocate: I came from a Science background and worked as an RA before jumping into a Masters of Speech Degree. I was disappointed at the lack of evidence-based teaching I received as part of my Masters 5+ years ago. When I did have a look at the evidence, I was surprised that the data was not as vigorous as working with mice or bacteria (funny that), especially with regards to the sample sizes. Whilst there is “best evidence”, I am sometimes discouraged at the research that I read and wonder if some of the “best evidence” is also pseudoscience. It has gotten better over the years, but slowly.

    I now work in private practice, and with regards to direct treatment with my clients, there is very little time put into place within the workplace to look for the best evidence-based treatment. I agree with Steph that private practitioners do sometimes recommend non-evidence based programs, perhaps due to the lack of quality control in the sector.

    Speechies are not the only group to ignore best evidence. See http://www.kevinmd.com/blog/2014/07/doctors-often-choose-ignore-evidence.html

  31. Thank you for this! I am going to print this out and bring to my IEP meetings! I have the opposite problem where the co Hort of SLPs I work with in my school setting base our therapy on science backed research. It’s the PARENTS who want us to do all these controversial therapies and then call their lawyer on us to fight us when we don’t agree or state that specific therapies are not backed by science. Super frustrating. Thank you for this article!

  33. I remember my grad school professor (who is also a prominent CD researcher) told us, “If you compare the SLP literature to the medical literature, we are still operating in the era of bleeding and leeches.” I have really, really tried to do my own review of the literature and I have found the process to be very time consuming and that the data has limited usefulness to me most of the time. I work with solely students of diverse cultural and linguistic background. I also work with adolescents and that literature is very limited as well. When I need evidence-based techniques, it is within the 30-60 minutes a day that I have time to “plan” for my sessions and frankly, that is hard to do.

  34. In the adult world, research is equal to homework in the kids’ world. It’s easier to “base” tx around our past experiences than it is to compare and contrast with what we think we should do and what we should indeed do with our clients due to the latest EBP. It’s the lazy way out. Terrific article! Thanks for the reminder that we are indeed life-changing scientists in the communication field.

  35. I wonder if the staus quo may be stable but a platform for exposing the ‘pseudoscientists’ amongst has allowed their voice to be heard more loudly? I have read a large number of posts that make me shake my head in disbelief. However, navigating the murky waters of social media and professionalism, I choose not to comment rather than engage in what will most likely be a unfruitful endeavor to open an already closed mind. Perhaps, many of us adopt this approach (leave, return, leave, return) and as such the ‘record’ shows a skewed ratio???

    Great post – thanks for sharing!

  36. Beautifully said. And as several have pointed out, SLPs are not alone in this problem. I recently read a post by an SLP stating that her experience carried more weight than the research. I found that frightening since that is exactly the thinking that led the facilitated communication movement to ruin so many lives in the 90s (and now it is back again ugh). When I train on EBP I try to tell educators that they are scientists, artists and detectives. They need to use the science of EBP but detect what the student benefits from and apply it Ina way that maximizes engagement, fits the student’s needs and has fidelity (that’s the art). Professional judgment comes into play but only with a base of science. Thanks fr writing this!
    Chris

  37. This is all very interesting. The term “evidence based” never came up in my training. It’s probably significant that my class was the last in my program to receive an MA….the very next year graduates received an MS. I was fortunate to have instructors who valued research, however and even got to be a research assistant my final year of graduate school. So my mind greets “evidence ” with excitement. I want to be effective and am all for anything that will improve my effectiveness. Keep up the good fight. There are many of us who aren’t wired to be primary researchers but who are happy to benefit from the labor of those who are!

Comments are closed.