Research Support: But Does It Work?

Blog #20 in the Phonology Means Nothing and Other Astounding and Very Practical Facts about Speech Sound Disorders Blog Series

For more information about this series, see the Phonology Means Nothing Series welcome page.

If your doctor recommends a procedure, you likely will ask what’s involved—how it’s performed, what is the recovery time, what are possible complications, and so on. Your doctor expects those questions, and they will likely give you answers even if you don’t ask, because you have a right to know the research support for decisions that affect you to make an informed decision.

Most clinicians recognize the importance of research support, both for their doctors and themselves. Evidence-based practice (EBP) embodies the clinician’s goal of providing expert care based on current research and in accordance with the wishes, values, and beliefs of those we serve. It is “the integration of best research evidence with clinical expertise and patient values” (Sackett, 2000), what Apel and Self (2003) call the marriage of research and clinical services.

Looking at Research

A useful way to look at research support is through a framework that organizes your thoughts. The Scottish Intercollegiate Guidelines Network (SIGN) is a well-respected system to evaluate the clinical support of a treatment approach (Harbour & Miller, 2001). The American Speech-Language-Hearing Association has adopted the framework, as have researchers in the field (Williams, 2010).

The SIGN System

SIGN (www.sign.ac.uk) divides types of research support into levels, the strongest research support being Level 1a. Level 1a is a well-designed meta-analysis of more than one randomized controlled trial. The next highest level includes well-designed randomized controlled studies (Level Ib). Next comes well-designed controlled studies without randomization (Level 11a), followed by well-designed quasi-experimental studies (Level 11b). Lower on the strength scale are well-designed nonexperimental studies, including correlation studies and case studies (Level III), followed by expert committee reports, consensus conference reports, and clinical experience of respected authorities.

SIGN and Speech

Baker and McLeod (2011a) reviewed treatment research for speech sound disorders and placed each study in its appropriate SIGN level. Their review comes from 134 treatment studies on speech sound disorders published from 1979 to 2009. See Baker and McLeod (2011a, 2011b) for further discussion, including a list of the 134 treatment studies.

They found:

• The studies represent 46 distinct successful treatment approaches and 7 successful approaches to selection of treatment sounds.
• Most studies (41.5%) were Level IIb (quasi-experimental studies), followed by Level III (nonexperimental case studies) (32.6%).
• The largest number of studies (27.2%) described one child, followed by 2 to 5 children (18.2%) and 6 to 10 children (18.2%).

Observations

Baker and McLeod (2011a) offer an excellent beginning place to discuss research support for treatment approaches for speech sound disorders. Here are three observations and an additional thought about their findings:

1. Number of Studies: 134

The number of treatment studies (134) means research support exists on which to base clinical decisions. It also suggests that a considerable number of people in our profession dedicate their time and efforts on behalf of children with speech sound disorders. The human hours needed to carry out a research study can be staggering. We are fortunate, I believe, to have colleagues who devote their careers to developing and testing intervention approaches. That said, 134 studies averaged over 30 years is less than 5 studies per year. This suggests our clinical area needs to continue to draw, train, and support researchers, hopefully in greater number in the future.

2. Number of Approaches: 46

The number of approaches, 46, indicates many ways exist to treat speech sound disorders successfully. The high number also suggests that at present, no speech treatment approach appears so superior that it has replaced all others. The Van Riper Approach (also called The Traditional Approach) perhaps is the closest the profession ever came to a single dominant method to treat speech sound disorders (Van Riper, 1978). That approach led the field in part because it was intuitive and practical. It also helped that Van Riper wrote extremely well. Additionally, it appeared early in the profession when obtaining a consensus among clinicians was easier since we served children less diverse and, also, faculty and students received training in fewer and more similar programs.

Importantly, the large number of successful approaches does not imply that “anything works.” Due to the nature of academic publishing, editors do not typically accept studies for publication that fail to report success. Journals require time, effort, and expense, and if a study either shows no successful results or simply replicates the results of a previous study, an editor is likely to ask why they should publish it. As a result, we seldom hear about treatment approaches that do not show success.

3. Level of Research Support: Quasi-Experimental and Case Studies

Most of the reviewed studies fall into the lower levels of research support, fully 75% being quasi-experimental studies (41.5%) or nonexperimental case studies (32.6%). These designs provide an excellent way to explore a clinical idea, a successful study suggesting an investigator is “on the right track” and serving as an impetus to perform additional investigations. Often, this level of research focuses on a small number of children, many times less than 10 and sometimes fewer than 5.

In research designs at these levels, the investigator knows the purpose of the project (in fact, in most cases for practical reasons, the investigator and the approach developer are the same person). Because intervention research is difficult and time-consuming, most investigators do not undertake it unless they are passionate about their treatment idea. This passion, commendable in its own way, can influence the study’s results, which raises the question: If someone with less passion for the approach did the study, would they achieve the same results?

An Additional Thought

An additional thought: Even when a treatment approach succeeds, it is notoriously difficult to determine why it succeeds. The temptation is to credit success to the theory or principle that underlies the treatment. To illustrate, if a phonological treatment succeeds, the temptation is to conclude that success was due to its good application of phonological principles. Though that may be true, it also could be that success resulted from factors unrelated to the theoretical framework of the study. This is because any treatment approach contains many components, which alone or in combination may contribute to treatment success. To illustrate, treatment activities, characteristics of the children, socioeconomic scale, time of day of treatment, and so on all may contribute to a successful approach. In a practical sense, this means you should accept that a treatment approach succeeded more quickly than you should accept the explanation for why it succeeded.

References

Apel, K., & Self, T. (2003, September 9). Evidence-based practice: The marriage of research and clinical services. ASHA Leader.

Baker, E., & McLeod, S. (2011a). Evidence-based practice for children with speech sound disorders: Part 1. Narrative review. Language, Speech, and Hearing Services in Schools, 42, 102–139.

Baker, E., & McLeod, S. (2011b). Evidence-based practice for children with speech sound disorders: Part 2. Application to clinical practice. Language, Speech, and Hearing Services in Schools, 42, 140–151.

Harbour, R., & Miller, J. (2001). A new system for grading recommendations in evidence-based guidelines. British Medical Journal, 11, 334–336.

Sackett, D. (2000).  Evidence-based medicine: How to practice and teach EBM (2nd ed.). Edinburgh, UK: Churchill Livingstone.

Van Riper, C. (1978). Speech correction: Principles and methods (6th ed.). Englewood Cliffs, NJ: Prentice-Hall.

Williams, L. (2010). Multiple oppositions intervention. In Williams, L. McLeod, S., & McCauley, R.J. (Eds.), Interventions for speech sound disorders in children (pp. 73–94). Baltimore, MD: Brookes Publishing.