Bridging the Research-Practice Gap with Systematic Methodological Pluralism

Entry for 27 November 2009:

For this year’s New Savoy Conference on Psychological Therapies in the NHS, Michael Barkham invited me to take part on a panel on evidence and the research-practice gap. As I thought about what I would say in my allotted 4 slides/10 min, I remembered a couple of lines from William Blake that I have loved since coming across them as an undergraduate in 1971. I looked the poem up and immediately realized how perfect the poem was for what I wanted to say. Here is the text version of my Powerpoint slides:

1. William Blake on Methodological Pluralism:

Now I a fourfold vision see,

And a fourfold vision is given to me:

'Tis fourfold in my supreme delight

And threefold in soft Beulah's night

And twofold always. May God us keep

From single vision and Newton's sleep!

(Letter to Thomas Butts, 1802)

2. Systematic methodological pluralism

a. Danger of “single vision”: Universal fallibility of knowledge practices

b. Points to need for multiple lines of evidence:

•To support practice

•To speak to different stakeholder groups

•To further science

c. Convergent operationism (Donald Campbell): Bringing multiple line of evidence to bear on a topic

3. Key Psychotherapy Research Methods Address Different Research Questions and Speak to Different Stakeholders

a. Randomized Clinical Trials:

•Research Questions: Test Causal models (internal validity)

•Stakeholders: Trialist Scientists (scientists who believe that RCTs are the one true way to evidencing a therapy; experimentalists)

b. Change process research:

Involves getting inside the “black blox” of therapy; including process-outcome, helpful factors, significant events and sequence analysis studies

•Research Questions: Develop causal models; Test causal models

•Stakeholders: SPR Scientists

c. Naturalistic effectiveness research:

Practice-based research (e.g., Michael Barkham and the CORE team)

•Research Questions: Establish generalizability to practice (external validity)

•Stakeholders: Policy-Makers/ Commissioners

d. Systematic case studies:

For example, Hermeneutic Single Case Efficacy Research (HSCED): looks at causality at the single case level

•Research Questions: Describe practice; Show what is possible; Test causal claims at an individual level

•Stakeholders: Practitioners

e. Qualitative/narrative research (first person accounts):

•Research Questions: Capture a sense of the lived experience of a therapy

•Stakeholders: Service Users

4. Blake’s Pluralist Epistemology Compared to Key Psychotherapy Research Methods

The Twofold Always:

1. Outer: Scientific-Objective-Observational: Randomized Clinical Trials; Naturalistic effectiveness research

2. Inner: Experiential-Subjective-Phenomenological: Qualitative/Narrative

The other forms of vision:

3. Unconscious: Interpretive-Tacit-Emergent : Change Process Research (getting inside the change process)

4. Transcendent: Spiritual-Immanent-Epiphanies: Systematic Case Studies (Most therapists are secular, so for them, the epiphanies occur with particular clients)

Commentary/further discussion:

1. Converging lines of evidence vs. range of convenience. In his presentation, Tony Roth, citing the American Psychologist George Kelly, raised the issue of the range of convenience of the different methods, suggesting some kinds of research may not be appropriate for addressing certain kinds of questions. This is certainly an implication of this framework. However, this turns out to be more complicated in actual practice: Thus, three different types of research all have a strong role to play in making causal inferences linking therapy to client change: (1) RCTs; (2) Systematic Case Studies (specifically, HSCED research; and (3) Change Process Research.

RCTs provide an operational method for justifying causal inference, but fail in themselves to satisfy the plausible explanation condition for causal inferences, and are subject to various ills ranging from differential attrition to researcher allegiance effects. Change Process research is actually a family of different types of methods that attempt to fill in the missing mediating processes linking therapy to outcome, or to test for the presence of hypothesized mediating processes. Systematic Case Studies (at least the HSCED variety) also seek to get inside the change process, but at the level of single cases.

Accumulating consistent evidence from these three different types of therapy research provides a set of converging operations that can provide a more solid basis for inferring psychotherapy efficacy, even though the questions they address differ. Beyond these, moreover, Naturalistic effectiveness research and Qualitative/narrative research can also play supportive roles in supporting causal efficacy claims for a psychotherapy: An efficacious therapy will show strong pre-post effects (especially persuasive with chronic problems) and will attract client narratives testifying to the experienced effectiveness of a psychotherapy. Each of these approaches provides evidence that bears on the broad question of effectiveness/efficacy, creating a network of linked evidence supporting the theory of a therapy’s effectiveness, what Cronbach and Meehl (1955) called a nomological net.

2. I got a real kick out of Roz Shafran, who teaches CBT at the University of Reading. In her presentation, she argued passionately that an excellent example of how to bridge the research-practice gap is work on key processes in Obsessive-Compulsive difficulties, including prolonged exposure and response prevention, thought-action fusion, contagion propagation, and the memory disrupting effects of repeated checking. These are all processes that were identified in practice, tested and refined through experimental laboratory research, and then fed back into practice as within-session therapeutic experiments. (These are analogous to EFT tasks.) During the question period, I said that I’d love to use change process research methods on this therapy, such as doing qualitative interviews. Roz responded, “Let’s work together!”, and I reached over and shook her hand; the audience applauded.

3. A few minutes later, someone in the audience, perhaps in an attempt to burst the panel’s bubble of pluralistic unanimity, asked the question that we’d all being avoiding: what we thought about NICE’s use of evidence. Although Tony ducked the question, Roz and I gave differing views, with her defending NICE and me criticizing its use of evidence. I’ve since learned that the situation with NICE is a bit more complicated, a topic that I intend to take up in a later blog entry. Shortly after that, the session ended, but the connection remains, and I look forward to further dialogue with Roz and others, such as Shirley Reynolds (who teaches at U of East Anglia and who I know from Sheffield days in the mid-1980’s).