Adventures with Research

Affectionately known as Adventures with CEUs

Alternately known as What do you mean it’s not EBP? They’re ASHA CEUs!

Every so often a situation arises where a person is slapped alongside the head with a clue by four. We have all had those “duh!” moments where we realize that a basic principle of life is not quite what we expected…if we’re lucky we also find out why.

There have been a couple of those slaps for me recently, and I thought I’d share them with you in the hopes that you can learn from my experiences. 

In the not too distant past one of the #SLPeeps on twitter was taken to task for having erroneous information on her webpage (no it wasn’t me!). Long story short, she has been in practice for years and has taken every CEU available for a particular therapy type program. She really bought into it, and as she said, the CEUs were all (or mostly) ASHA CEUs so they must be good. ASHA CEUs must be EBP…Right? WRONG! There was a big brouhaha and (to make an even longer story even shorter), she ended up changing her webpage, her stance on this particular subject, and even her practice.

On Thursday, I had the privilege of attending a conference (yes, it was a privilege because the weather cooperated and my work paid for it and I NEVER take either one of those for granted!). I live tweeted* the conference, and some of the responses I received in the afternoon were…interesting…frustrating…occasionally (I’m choosing to believe unintentionally) rude… and long-lasting (they continued until the next day).

You are probably thinking, Wow! This conference must have been really loaded with controversial topics to generate such a response. Unfortunately that wasn’t true (at least not on the surface)…but when I explored deeper (and thanks to the tweets I had the direction to do so) I realized there was a controversy here, and a lesson, but probably not one the presenter was expecting.

Now, I’m not going to get into the nitty-gritty of the conference. First, I haven’t had time to research it all to verify the information (although some I have) and second, my blog is not the place to call people out…it’s more a place to educate and encourage thought (that’s my theory and I’m sticking with it!). What I will say is that there was information presented that was used to support the presenter’s claim (and her $400 program). She had all the citations, she had the quotes, she had all her ducks in a row…or so I thought. This was not an area that I have strongly researched and I had to take it at face value (at least to start).

There was one particular tweet that set of a small flurry of responses…and then there was THE tweet. This tweet had a few researchers on twitter absolutely aghast. A virtual twitter storm began and I had people commenting on it for a good 16 hours after the conference. While on one hand I was a bit…annoyed (I was at a conference after all and not able to READ the responses if I was going to tweet the conference)…I was also reminded not to take things at face value.

One response made the cryptic statement that the research the presenter was citing to support her claims was incongruous with the claims. What? She’s citing claims that go against what she’s selling? That can’t be right (again, I couldn’t verify as I was tweeting).

A second tweet storm was sparked when I discussed a particular therapy model. Now, said therapy model is NOT one that I use and had only limited information about…but, as I was tweeting the conference, I didn’t take the time to analyze the information. I mean, seriously…I’m at a conference. It’s not like I can say – hold on…let me pull up that reference and see if it makes sense…or.. Wait. Which issue was that theory discussed because I think it was refuted later on. When you’re live tweeting – you listen with your ears, regurgitate with your fingers in 140 characters or less, and analyze later. In some cases – much later.

What I learned is; this particular therapy model and the research supposedly supporting that model was refuted by another researcher. So (just to make the muddy waters a tiny bit clearer) Researcher A made a claim that was supported by the research conducted in his/her clinic. Then Researcher B performed similar research and was able to point out the flaws in Researcher A’s research. Researcher A’s research was refuted.

Obviously there’s a lot going on behind the research doors and upper reaches of our profession that Susie SLP has NO idea about! Now, while it’s easy for me to be a bit tongue-in-cheek with this (simply to ease your reading), in all seriousness it’s an important issue. There’s research out there that’s faulty. Let me say it again. There’s PEER REVIEWED research out there that’s faulty. I’ll give you a few minutes to digest that information.

Ok. Now that I’ve shocked you all to no end. Let’s continue.

Now, I am not aware of all the mechanisms that go into a peer review. But, it’s important to remember that reviewers are human – and they are reviewing information presented at a particular time in a particular way. I am not criticizing the reviewers (and not just because I hope to be published some day). In all honesty, I do believe that peer review is the way to go – we just have to remember they are not omnipotent and they have to go on the information presented to them. After all they can’t reduplicate every research model for every submitted paper.

The whole purpose of this post is to remind us all that we ALL have the need and responsibility to be peer reviewers of a sort. When we read (which I strongly hope you are all doing) journal articles – we need to think of them critically. Does the research design make sense, does the discussion really make sense?

Even more importantly when we’re attending a conference – don’t just attend, take it all at face value and accept things blindly. Go home, read the research, and judge for yourself if the research supports the authors claims. If it doesn’t – contact the presenter.

Finally, and perhaps most importantly, know that just because something says “ASHA CEUs available” does not mean it’s ASHA approved…and it doesn’t mean it’s evidence based practice…and it doesn’t mean you should go out and start implementing right away.    In fact, if you look at ASHA’s criteria for CEUs it’s not hard to get approved. Note that no where does it say this must be research based, empirically proven, or anything of the sort.

One of the most important things we can teach our children is to think critically. To look at all aspects of a subject and really consider it. We need to do the same in our professional information.

We ALL need to do this – not just the researchers we all tend to hold up on pedestals.

I am just as responsible for determining whether information is EBP as the presenter is.  (Ouch! That burns a little bit doesn’t it?)  We need to remember that many of the presenters out there are making a living presenting and selling their product. If someone is making money off of it – we need to triple check it because they have financial gains. Which, as an aside, is why I believe ASHA is requiring those disclosures now!

So…please. Learn from the mistakes of the #SLPeep…learn from my experience live tweeting…and look…Really LOOK…at the research presented at those CEUs we’re all striving to get. Don’t go through the motions just to get your CCCs updated…don’t do it just because you have to… do it to better yourself, your profession, and to help your clients. Be an informed participant not a lemminglemming**.






I’d love to hear your thoughts here…were you aware that not all ASHA CEUs are vetted for EBP? Have you gone home and researched the information presented at conferences? Or do you (as I have in the past) simply attend, get the CEUs, and go about your business as usual? Drop me a line here and let me know.

Until then…Adventure on!

*live tweeting: while the presenter is speaking, I tweet the main ideas presented, cite the research when available, etc. It does not mean I condone, use, or verify the information. It is the equivalent of my taking notes in class…I just happen to do it so everyone and their pet duck can read it on twitter. After the conference I compile it into a ChirpStory so all the tweets are together in chronological order – and it is easy to go back through the “notes.”

**Lemming: A lemming is a small rodent that lives in Siberia. Urban myth has it that when they migrate they will all follow the leader off a cliff (if that’s where the leader goes). Apparently this isn’t common but has been known to happen to at least a few of them. They do migrate in huge packs – and blindly follow the leader. Much like many people will blindly follow their leaders (researchers) without thinking critically… (and I really hope I didn’t just tick anyone off…trust me, it wasn’t intentional).

5 thoughts on “Adventures with Research

  1. Do you know I don’t think I’ve ever seen a lemming, so thank you for that little trivia picture! I agree with you wholeheartedly here although I think the practical issue tends to be time (and maybe energy). When you finally get it together to pack, travel, organize your household so it runs during the few days you are gone, you want the conference to kind of take care of itself. It’s a mindset shift to think the lecture/conference is the starting point. That said, I had my annual physical a month ago and, dutiful patient that I am, I had my list of questions, including, “haven’t they increased the recommended levels of vitamin D.” Well, despite that “research” showing up in all the monthly women’s/fitness magazines it hasn’t been thoroughly vetted by the medical community. I’m sure this is an endless annoyance to physicians as they walk a fine line between being “too conservative” and hopping on every last bandwagon that pulls into town. Thanks for the post! Kim

  2. Excellent post as usual, Mary. We all need to be reminded or made aware that just because a session at a conference will give you ASHA CEUs doesn’t mean it is S&L Gospel. Speaking from experience as a presenter at the state level from KY, Ohio, and WV, I can attest to the fact that getting a session ASHA approved for CEUs required very little documentation. A bibliography nor research resources were asked for when applying.

    We need to remember that anyone can take “facts” (research) and use them to represent anything we want. I am always cautious of presenters who are pitching a product. I’m happy to say that Leah and I, as 2 Gals Speech Products, had a booth and sold our wares but our sessions were not about our products.

  3. A high school teacher at my former school said that one of the instances he felt most successful as a teacher was when his students started discussing Kony2012. The YouTube video stirred emotions and got everyone thinking about and discussing the terrible injustices. Some of them jumped on the proverbial bandwagon – but not right away.

    The next week’s worth of classes involved students reviewing the video from an inquiry stance. What fact were being cited and where did the authors get the data? What other political, economic, cultural, religious, and social factors might contribute to the travesty? What can history tell us about terrorists like Kony and how they can be stopped? How do people like Kong come to power? The questions kept coming.

    When we teach students to be critical thinkers, we empower them to find and digest research in a purposeful way. They are less likely to jump on bandwagons based purely on emotion. And, they will be the students who think very carefully before putting all their faith in one packaged program in any field.

  4. As someone who is starting to see how peer-reviewing works, I’ve learned that just because it’s peer reviewed doesn’t mean it’s good science or sound thinking.

  5. I have to share this with you because it’s timely: just yesterday I was in a meeting and cited some research. Someone pulled up the paper, and within 5 minutes, was criticizing the conclusions because the statistical analysis and proof of their claims was terrible. The other person was shocked that this was even published and I felt like such a heel; I should have known better! But because it was in a highly respected journal I didn’t really sit down and analyze the results section, but instead looked at the lit review and the discussion. Bad, BAD BAD.

    Don’t make my mistake! Look at the results, and make sure that there’s a statistical test for every claim they make (that is, don’t rely on single subject design’s “Visual inspection of the data” when looking at journal articles that aren’t specifically about SSD!)

    (And, after recovering enough to say something, we both agreed that it was surprising that this even got published in the first place; further proof that a journal is only as good as its editors and reviewers.)

Comments are closed.