Aspects of Causal Inference

Entry for 22 December 2006 (flying to California):

Much of what is emerging these days in my emotional and intellectual life comes in the early morning between sleep and waking, in the form of dreams and reveries. Two weeks ago I dreamed I had found a multidimensional framework for organizing causal inference, a key issue in psychotherapy outcome research. But I was thrust immediately into daily activities and was unable to write it down, and it faded from awareness. Then, a week ago, I was laying half-awake, and this dream came back to me. I lay there for an hour drifting in and out of consciousness, exploring and elaborating the framework, then, I got up and spent an hour making notes on the framework. I wrote the following entry as we flew to California today:

The starting point for this framework is the observation that different aspects of causal inference are typically confounded with each other in discussions about psychotherapy outcome, particularly in translating research into policy recommendations, such as when policy bodies try to making lists of empirical supported/evidence based psychotherapies. However, there are at least 3 different aspects of inference that have to be made and justified:
I. The type or level of causal inference being made (weak, strong, or deep)
II. The likelihood that a relationship is the case in a particular instance (surety)
III. The prevalence of the relationship in the population (breadth)

I. Type or level of causal inference:
A. Weak or change-related causal inference: Is this therapy related to client change? This simplifies to, Did the client change over the course of therapy? This concept parallel the following methodological formulations:
1. The first two conditions of causal inference in causal condition theories such as J.S. Mill; Haynes & O’Brien (2000), which state that necessary preconditions for making a causal inference are that there be some sort of association between cause and effect and that the supposed cause must precede the effect. In causal modelling, these are the conditions for a weak causal order.
2. Statistical Conclusion Validity in Cook & Campbell’s (1979) theory of design validity: Again, that there be a statistical correlation between cause and effect.
3. Change vs. nonchange explanations in Hermeneutic Single Case Efficacy Design (HSCED; Elliott, 2001, 2002): The first question in systematic single case studies of therapy: Did the client really did change over the course of therapy? (This establishes the existence of a correlation and an appropriate temporal order between cause and effect.)
4. In psychotherapy research, relevant ideas are: effectiveness research; reliable change; pre-post significance testing and effect size analysis; open clinical trials.

B. Strong or change-production causal inference: Does this therapy cause client change? In other words, is it responsible for observed client change? However, this question needs more unpacking, because we are not talking about mechanistic change. Clients are not billiard balls, and therapies (and therapists) are not pool sticks! In general, it is more precise to say that clients make use of therapy to change themselves, so a better version of this question might be: Does this therapy successfully provide this client(s) with useful opportunities to change themselves?
1. This inference involves a third condition of general causal inference: the ability to rule out reasonable alternative causes to one’s favored cause.
2. This corresponds to Cook & Campbell’s internal validity, that is, the ability to infer a causal relationship between cause and effect by ruling out plausible alternative causes.
3. Therapy vs. nontherapy explanations in HSCED, the second research question to be addressed in systematic single case studies: Did therapy make a substantial contribution to client change?
4. Relevant concepts in psychotherapy research: efficacy research, randomized clinical trials, no-treatment and alternative treatment control groups.

C. Deep or explanatory causal inference: Strong causal inference is empty, pointing to the existence of a causal relationship without specifying how the causal relationship works. So the question is, What is the nature of the process by which therapy brings about (or helps clients bring about) change? Or: What particular processes in this therapy (and preferably specified by the therapy’s theory) lead to change?
1. This inference corresponds to the 4th condition of general causal inference, that is, the existence of a plausible explanation for the relationship.
2. In Cook & Campbell, this condition is described as construct validity of causes, that is, what is the true nature of the cause, as opposed to its theorized nature?
3. In HSCED, deep or substantive inference is addressed by the third research question: What processes in therapy and outside it were responsible for the observed client change? Researchers then construct a narrative tying the different reasonable causal processes together.
4. In psychotherapy research research, the relevant concepts are: change process research, process-outcome research, dismantling and additive designs, and treatment adherence.

II. Inferential Probability (surety): Each of the three types or levels of inference requires a separate judgement or inference, and each has a separate probability associated with it: change-related, change-production, and explanatory inference. These probabilities can be quantified on a 0 to 100% scale, but useful phrases can be attached to different quantitative levels:
A. 10-50%: Possible: This means that the inference could be true, but is not likely.
B. 51- 80%: Likely: More probable than not. All things being equal, in the absence of countervailing evidence, where action is needed, this inference is worth making, that is, it is better than nothing.
C. 81 – 95%: Highly likely: A good basis for action in practical situations, but by no means certain.
D. 96-99%: Almost certain: This is the standard probability level in the social sciences. If you want to make an inference that is almost certain to be true, being wrong only 1 – 4 % of the time seems reasonable.

III. Population Probability Inference (breadth). Levels and probabilities of inference also apply across people/clients. In other words, for what proportion of people do these inferences hold? Alternatively, How far can we generalize our inferences? This corresponds to: (1) External validity in Cook & Campbell’s design theory, which refers to the extent to which causal inferences can be generalized to populations of interest. (2) Case precedence standards in legalistic research models such as HSCED. (3) In psychotherapy research, health services research, moderator research, replication, best practices standards are the corresponding concepts. Population probabilities can also be expressed quantitatively, from 0 to 100%, but different ranges of probability can be usefully distinguished:
A. Possible: Inferred successfully in 1 or 2 cases; possibly efficacious; based on a minimum precedent.
B. Probable: Inferred successfully in more than half (> 50%) of some reasonable number of cases (5? 9? 21?).
C. General: Inferred successfully in most (> 80%) cases.
D. Almost invariable: Inferred successfully all or almost all cases (> 95%).

Implications:

When would we be justified in saying that only therapy X should be offered for a particular client condition? I think that this would require a whole set of inferences: weak, strong, deep, sure and broad. That is, we would want to be almost certain (II.D) that almost all (III.D) clients changed (I.A), that their change was due to therapy (I.B.) and that we understood why (I.C.). Such a rigorous standard should be required in order to go against established bodies or knowledge supporting other approaches and also against considerations of client choice.

On the other hand, in busy practice settings where little is known and where action is required, it would be enough for us to infer that client change (I.A., weak causal inference) is likely (> 50%; II.B.) in at least half of clients (III.B.).

Finally, if the situation was desperate, as is often the case in the treatment of advanced stages of cancer, client change (I.A.) at reduced probabilities (up to 50%; II.A.) and for only one or two clients (III.A.) might suffice. This would not be ideal of course, but it is possible to imagine situations in which it would be the most ethical course of action.