It’s the second Tuesday of the month – which means it’s time for Blogging About Research. Today’s research will follow a similar format (I think) to last month; although, to be fair, I reserve the right to make [side excursions] as I often think out loud while typing. Don’t forget to read the compilation of all the article reviews from participating bloggers here.

Today’s research article comes from the Canadian Speech Language Hearing Association’s (CASLPA) journal the Canadian Journal of Speech-Language Pathology and Audiology (CJSLPA). I have to say, I love this journal.

The article that I chose for this month’s blogging is entitled: Inter-rater Reliability of Clinicians’ Ratings of Preschool Children Using the FOCUS: Focus on the Outcomes of Communication Under Six  (Disclaimer: Yes, I know it’s a research note not a true article…I thought it was an interesting read anyway)

Background: According to the article, approximately 6% of preschool children are identified as having a communication disorder and receive treatment. These disorders can be life-long and have significant impact on the child and their family. Some of these impacts include dropping out of school, failing to gain and/or maintain employment, and an increase of psychiatric disorders. However, to date, there have been no reliable measures available to determine if treatment is actually improving real-world outcomes of speech-language therapy.

Many SLPs use standardized tests not only to determine eligibility but to determine improvement as well. However, according to the authors, there are a few things inherently wrong with this concept. First, most standardized tests are not reliable enough or sensitive enough to measure improvement (even though they may be stellar at measuring delay). Second, most assessments measure a particular skill/attribute. For instance, voice assessments measure voice, vocabulary assessments measure vocabulary, etc. It’s also necessary to keep in mind, most SLPs measure improvement in the specific impairment, not in the functional conversation skills of the child. There are very few measures (if any) that truly measure the child in a gestalt way. [We need to remind ourselves to look at the whole child and not get lost focusing on the little things. The long-range goal for any child communication disorders or not, is to be a happy and functional adult.]

Unfortunately, using assessments that are not designed to measure progress may cause difficulty with treatment plans and clinical decision-making. Inappropriate use of assessments may also cause researchers and clinicians to make inappropriate comparisons between treatment methods ultimately leading to a greater negative impact on Evidence Based Practice.

Purpose:  The purpose of the study was to see if the Focus on the Outcomes of Communication Under Six (FOCUS), a broad-based outcome measurement tool was reliable, valid, and effective way to measure the communication growth in young children. 

Questions: What is the inter-rater reliability of clinicians using the FOCUS while rating change in children following speech-language interventions.

What they Did: The participants in the study included four SLPs and 13 preschool aged children (3;1 – 6;4). The SLPs rated each child using the Communication Function Classification System (CFCS) with a Likert type scale where 1 was best and 5 was lowest functioning.

Pairs of SLPs were asked to assess each child at two different points in time (once pre-intervention and once following a few months of intervention) using both interactions with the child and interviews with a parent. The SLPs completed the rating scales independently using the FOCUS.

Results: The inter-rater reliability (based on 11 cases as two were considered outliers) was determined to be acceptable with an intra-class correlation co-efficient (ICC) of .70. The change scores ranged from -18 to +116 with a mean of 57 and a standard deviation of 36.

Ultimately, it means that the study supported the test-retest and inter-rater reliabilities for using FOCUS as a means of determining the progress of children receiving speech-language therapy.

There are a few problems with the study. Thirteen, well eleven really, children is not a large participant size. Lack of appropriate training caused one SLP to improperly score the FOCUS without enough parent interview which skewed the results for the two outliers. The study also discusses the fact that their participants were fairly heterogeneous and the results may be different with a more homogenous group.

Real Life Applications: Using FOCUS may be an interesting way to really measure whether we, as SLPs, are being truly effective in our therapy. However, FOCUS is basically a parent report questionnaire and I’m… a bit leery of them.

My thoughts: As I mentioned, I’m leery (and for no real good reason, I admit it) of using parent report as a way to gauge effectiveness [or eligibility]. Don’t get me wrong – parents are an integral part of what we do and I recognize that. I encourage parents to be active participants and I want that input…But, parents don’t always see the little things that we might be working on. However, since FOCUS was designed to be used before and after, maybe it would truly be fair and accurate progress monitoring. Parents certainly know their children better than we ever could.

The article definitely made me think about what tools I use as progress monitoring…and… If I’m honest, it made me question whether or not I look at the child as a whole. I’d like to believe I do…I certainly try to…but, again if I’m honest, there are probably times that I don’t accomplish that goal.

I think one of my professional goals this year will be to look at the gestalt child. To really see everything about him or her and use that information to guide my clinical decisions. Hopefully, I will be able to convince the teachers in my school to do the same thing…To not just see the kid with the reading problem, the kid with the behavior problem, or the girl with no friends…but to see the child as a whole and to set the goals not with what will the child accomplish this year, but what can WE do to help them become a successful adult…to really look at the big picture.

What are your thoughts? I’d love to hear them…

Until next time…Adventure on!

Mary