Entry for August 20, 2007:
Offered by participants at “Doing Research on Humanistic Psychotherapy and Training: Practical Strategies for Practice-Based Research”, Workshop given at Humanistic Psychotherapy Conference, San Francisco, California, August 2007. Used by permission.
Topics identified by focusing on areas of curiosity. For an earlier list of research topics/curiosities, see Blog entry for 22 May 2007. My comments and suggestions are given in italics after each idea.
1. Take data from therapeutic alliance ratings and use them to work backwards to help consumers pick a therapist.
Robert’s commentary: Nice idea. Alliance ratings often show ceiling effects, so it would be important to use an instrument with good variability at the top end of the scale. Also, it might be a good idea to use ratings from multiple clients, i.e., to identify therapists who were generally seen by their clients as participating in high positive alliances with their clients.
2. Students in doctoral [i.e., postgraduate in the UK] programs in clinical psychology and counselling (psychology): What do they learn and what do they use? How much of what they do is from what they learned vs. what they brought to training. Try comparing first vs. third year students.
Robert: This is why pre-post designs are such a good idea for training research, because of the importance of prior training/skill levels. A cohort design (that is, comparing first vs. third year students) can be useful, but a more powerful design would be to track one or more groups of students longitudinally over training.
3. How do I measure clients’ intensified sense of their lives, existential presence, engagement, commitment?
Robert: This sounds like a good outcome variable for existential therapy research. I would start by collecting different descriptions of the desired state; I would also want to think about doing a qualitative study of this state to provide additional descriptions. Then, I would put these descriptions together into a draft questionnaire, get feedback from friends and experts, then test it out on 100 or so people. I would use the results of identify and get rid of bad items and to develop subscales.
4. Effects of shifts in client language: Is there a relationship between client therapy outcome and changes in client use of language (used to describe experiential states) from (for example) “I’m angry” to “It’s angry in here”.
Robert: This sounds like a really interesting topic. One way of getting starting and making this more manageable would be as part of a larger pre-post outcome study on the effects of focusing-oriented or other experiential/humanistic therapies. In such a study, one can then identify small subsets of high and low outcome clients in order to study their discourse practices for describing their experiential states.
5. [From the same person:] The effects of therapist use of language as in “You’re sensing”; “Something in you”; and “That place”. The hypothesized effects include (1) Changes in client language [see Idea 4]; (2) change from start to end within particular sessions, in terms of client ease, comfort, anxiety, and feeling less overwhelmed; (3) overall therapy outcome.
Robert: Ideas 4 & 5 are related and could be combined within the same larger research project. Client mood ratings could be collected at the beginning and end of sessions, using, for example, parts of Stiles’ Session Evaluation Questionnaire (SES), or building on the list you've given. It would be important that outcome measures include something that measures emotion regulation (Jeanne Watson’s group has developed one), or something that ties in to the change pointed to by the changes in client language.
6. Client nonverbal indicators of change: Does client change correlate with changes in client gestures, body movement, and especially gestures that differ from verbal content?
Robert: This is similar to idea 4, except that the variable is nonverbal (gestures) instead of verbal (discourse markers). Again, I would start with outcome and work backwards via high and low outcome clients.
7. How do clients tell their stories in therapy and how do their content and structure change over time? How can this inform the therapy process? How do changes in client stories relate to therapy outcome and process?
Robert: There is one study I’m familiar with from Edna Foa’s research group on this topic in the treatment of trauma (Amir, N., Stafford, J., Freshman, M. S. Foa, E. B. (1998). Relationship between trauma narratives and trauma pathology. Journal of Traumatic Stress, 11, 385-392.). This topic is also similar to ideas 4 and 6 in the interest in relating client discourse to therapy outcome. Because discourse analysis is difficult and time-consuming its generally a good idea to focus on high or low outcome clients (the working backwards strategy). In this case, a rather different line of research on the same topic could be done as task analytic research on client narrative retelling work. I have a graduate student at the University of Toledo, Emily Beighner, working on the latter angle.
8. Self-discrepancy as an outcome measure: What is the relationship of self-discrepancy ratings to anxiety and depression? What implications does this have for understanding the process of change in therapy?
Robert: A self-discrepancy measure compares different ways of rating oneself, typically, how I see myself with how I would like to be (real - ideal discrepancy) and takes decreases in discrepancy as a sign of positive change. This idea sounds like a pretty straight forward construct validation study. It would be interesting, however, to add another person-centred outcome variable, such as the Strathclyde Inventory, which is supposed to measure psychological flexibility vs. rigidity.
9. (A) Temperament and focusing in therapy: Do persons with certain temperaments, such as intuitive sensing, thinking, feeling enjoy Focusing more vs. less? (B) What client markers and guidelines do I use in order assist my own perceptual decision process re: C temperament in relation to whether I use Focusing or not?
Robert: A and B suggest fairly different lines of research. In any case, there several angles to this, ranging from client predictor research using something like the Myers-Briggs Type Indicator scale to predict outcome, to task analysis of Focusing events in therapy.
10. What are the key humanistic principles that support and make more effective other traditional interventions with survivors of sexual or physical emotion abuse?
Robert: This seems a bit tricky as a topic. Could it be that you’re asking about studying the humanistic elements of psychodynamic and cognitive-behavior therapy? One approach to that would be to adapt Jones and Ablon’s method of creating prototypes of different types of therapy (they did psychodynamic and CBT) in order to develop a profile for humanistic therapy. That would be a very interesting thing to do!
11. In the UK, what are the effects of recent government funding policies on counselling training?
Robert: This is policy research, but timely. You’d probably want to start with a survey of course directors, then see if you could back that up with some kind of quantitative trends, for example, declining applications.
12. How does change really happen from the client’s point of view? Including all the pieces, what is change all about? For example, much change is nonconscious, such as clients acting different in everyday life. How do we get at this?
Robert: Asking clients is a good starting point, as in the Change Inteview, but as noted may not yield a comprehensive account of changes, because nonconscious changes are likely to be left out. Asking other people has been proposed in the past as a way to get around this. This could include the therapist but significant others could also be interviewed. Another possibility is using in-session process (e.g., client experiencing level or repeating interpersonal themes) as an outcome measure. These possibilities are tricky and time-consuming, which is why they are not used much. The problem with indirect measures and measures of nonconscious changes is that they either correlate highly with direct/conscious measures, in which case they are redundant and not worth doing, or they don’t correlate with direct/conscious measures, in which case their validity is suspect. That means you really have to believe this is important to want to do this kind of research!
13. Are there tools or attitudes that clients develop to handle future crises by acting as a fully functioning person? What are the long term effects of developing such tools or attitudes?
Robert: The Strathclyde Inventory, developed by Freire, is an attempt to develop a measure of the fully functioning person. Another possibility would be to look at measures of coping style, of which there are many, for example, Carver’s COPE (http://www.psy.miami.edu/faculty/ccarver/sclCOPEf.html). Your research question, then, comes down to assessing the long-term predictive validity of this sort of measure, tested by following a group of people over time while giving them one or more measures of general functioning (problem distress, life satisfaction) and your measure of full-functioningness.
14. Client and therapist experiences of futility about therapeutic process.
Robert: This topic seems to me to be closely related to the research topics of therapeutic difficulties and alliance ruptures, an interesting, clinically-relevant topic about which there is a small scientific literature. For alliance ruptures, see the work of Safran and Muran; for therapeutic difficulties, see studies by Davis et al. (e.g., Davis, J.D., Elliott, R., Davis, M.L., Binns, M., Francis, V.M., Kelman, J., & Schroeder, T. (1987). Development of a taxonomy of therapist difficulties: Initial report. British Journal of Medical Psychology, 60, 109-119.) Measures of each of these exist, such as the Working Alliance Inventory 12-R and the Therapeutic Difficulties Scale.
As always, I really enjoyed hearing about all these different research ideas, and hope that at least some of them can be developed further, possibly along some of the lines suggested. Good researching!
Tuesday, August 21, 2007
Monday, August 13, 2007
Murray Creek, Summer 2007
Entry for 13 August 2007:
Return to Murray Creek. After a week in Toledo, we flew to San Francisco last Saturday, and drove up into the foothills of the Sierra Nevada, stopping for an hour or so in Pleasanton (a suburb in the East Bay Area) for a brief visit with Diane’s mom over lunch. We arrived in the late afternoon, following the winding Murray Creek road for the last three miles until we came to my mom’s place. There are two houses there, but after my dad’s death last year, my mom moved back to the upper house, even though it was still partly wrecked from the damage when a huge oak tree next to it uprooted last year and tore the sleeping porch off its foundation two weeks before my dad’s death. (My mom reports in a recent blog entry that when my dad saw the damage, he realized that the lung cancer was going to get him.)
Now, 17 months later, the repairs have been finished, and we were greeted by a renewed house, the old sleeping porch enlarged and opened up to create a light, airy space, with a brand new redwood deck just beyond, decorated with my grandmother’s cast iron patio furniture, which I recognized from my childhood 50 years ago.
Reconnecting with family. My mom, brothers Joseph and Conal and Conal’s partner Holly were there waiting for us, and we were very pleased to see them. Conal and I hadn’t spoken too much since our father’s death, when there had been a bit of tension between us over my being critical of Nonviolent Communication (one of those silly defensive academic petty things I sometimes fall into and now regret). However, last April he and Holly moved to Murray Creek and have been reconnecting with the family and the land. Then in May, there was a bad brush fire on their property (which is next to my mom’s), which was quite traumatic for them. Fortunately, their house survived without damage, and they have bounced back from the trauma and recovered from the fire-related PTSD; in fact, the fire appears to have even deepened their commitment to the valley. Meanwhile, my siblings and I pulled together to offer them emotional support. I am very grateful to them also for the support they have been able to provide to my mom.
My mom looked better than she had for a long while, getting around well with or without her walking sticks, and appears to have new energy and focus now that the house is fully repaired and renewed. After my dad died, it was really not clear what would happen next at Murray Creek. My mom took in a boarder, who has been living in the lower house, and my mom took to wandering among my siblings’ houses on a kind of hejira while her house was being repaired. Now, it seems, she is home again and to stay, and some kind of balance is being restored to the valley.
Climbing Mountains. So my brothers and I spent time catching up with each other. I heard about Joseph’s recent trip to Africa, and his and Ebru (my Turkish sister-in-law)’s trek up Mount Kllimanjaro, a 7-day journey from rain forest to glaciers that put Kenneth and my recent adventure climbing Wyndburgh Hill to shame. And we heard about Conal’s plans, his rediscovered love for the valley, how he and Holly are dealing with the aftermath of the fire, their plans for fixing up their house, and so on. Conal and Joseph plunged into the most obscure computer programming talk I have heard in a long time. It was wonderful! Then, yesterday, Kenneth and I got up and ran the Murray Creek Road, an annual ritual of ours, a demanding 10K course of steep up and down hills, first in the cool of the valley, then as we climbed the steep road out of the valley to the top of the ridge, warmer in the summer morning sun. At the end, we sprinted to make it in less than an hour.
Physical labor. Then, after breakfast, we tackled the large pile of chipped wood mulch in back of the house, left by the tree trimmers. Over the next two hours, as the day got progressively hotter, Joseph, Kenneth and I shifted a couple tons of wood chips, spreading them over the back garden, halfway up the hill to the water tank. Kenneth and I lasted about two hours at this back-breaking labor, so out of character for us that virtually the only place we engage in this level of physical labor is here, in Murray Creek. Finally, Joseph, who hadn’t been out running first thing in the morning, worked on for another half hour spreading the chips around, while Kenneth collapsed on his bed and I took a much needed shower.
Meteor Shower. There is something both deeply satisfying and cleansing about this level of physical labor, and so we lounged around for most of the rest of the day, talking among ourselves until it was time for Joseph to go back to San Francisco. Kenneth fell asleep about seven, I worked on my APA presentation for Friday. Finally, about 11 we went out to watch the Persiad meteor shower. One of my favorite things about Murray Creek is the stars. We went out onto my mom’s new deck, sat down on those wrought iron patio chairs, lay back, and watched the deep night sky. The Milky Way was emblazoned, north to south, across the sky. Watching for meteors is always a bit tedious, and some years it is totally fruitless, but this year we were lucky: Diane and I each saw three shooting stars in about twenty minutes. They weren’t the same three meteors, but it was enough to satisfy our desire for awe-inspiring but evanescent sights. We went inside and crashed.
Labyrinth Ritual. This morning, we got up and, after breakfast, walked down to the labyrinth, my mom moving right along with her trekking poles. She paused at the entrance to the labyrinth, and said, ”We have been saying a little something before entering the labyrinth; we said, “Go ahead.” She said, “O luminous spirits, who inhabit this place, watch over us here, and God bless you.” Then she paused, and said, “We usually say it three times.” We gestured, “Go ahead.” And she did.
Then she led the way into the labyrinth. Conal and Holly had recently run weed-eaters over the labyrinth, so it was better tended and neater than usual. The mention of luminous beings inhabiting the labyrinth made me think about all the other beings who inhabit it, mostly the gophers, the evidence of whose tunneling was everywhere along the way; I thought of dark beings and light ones; mundane and the numinous. Over the past year, my mom has come to understand the labyrinth as an entrance to the spirit world, a focus point for powers and energies or spirits to cross between our world and the other worlds posited in the shamanistic world view. I suppose the gopher holes could be thought of a symbolic entrances to the lower world.
My mom, me, Diane and Kenneth walked to the center of the labyrinth, faced the four cardinal directions, touched the Mother and Father Stones. There was a crystal resting on the Father Stone, very similar to the one that Kenneth and I had left on top of Wyndburgh Hill only two weeks before. I picked it up, looked at it, set it down. Then the four of us hugged, and we went back out.
Chinese Dam. After that, we walked up the road to Conal and Holly’s house, got the tour of their place and plans, then went down to the Chinese Dam. Sometime during the California Gold Rush (1848-), probably around 1863 (according to www.calgoldrush.com/part3/03asians.html, this was the peak of Chinese mining efforts), some of the 25,000 Chinese gold miners in the California Mother Lode at the time, got together and built a 3 –5 meter high dam across Murray Creek, just below where Conal and Holly’s house is today. The remains of this dam are on their property, and in Scotland would be considered to be an historical site of archeological significance and listed in the national register. The dam appears to have been a dry stone structure, somehow sealed on the upriver side. The dam would have created an artificial lake extending several hundred meters up the valley. Water was then diverted into along ditch on the south side of the valley for use in fluming gold ore. We don’t know exactly where the many tons of stone used to construct the dam came from; undoubtedly much of it came from the river bed. After the gold ran out, the dam was opened to let the river flow through unimpeded; much of the stone appears to have been carried downstream, on my parents’ property. We don’t really know any more than that about the dam or the people that built it!
We left shortly after our visit to the Chinese Dam, a too-short visit, but satisfying on multiple levels. Onward to San Francisco & APA!
Return to Murray Creek. After a week in Toledo, we flew to San Francisco last Saturday, and drove up into the foothills of the Sierra Nevada, stopping for an hour or so in Pleasanton (a suburb in the East Bay Area) for a brief visit with Diane’s mom over lunch. We arrived in the late afternoon, following the winding Murray Creek road for the last three miles until we came to my mom’s place. There are two houses there, but after my dad’s death last year, my mom moved back to the upper house, even though it was still partly wrecked from the damage when a huge oak tree next to it uprooted last year and tore the sleeping porch off its foundation two weeks before my dad’s death. (My mom reports in a recent blog entry that when my dad saw the damage, he realized that the lung cancer was going to get him.)
Now, 17 months later, the repairs have been finished, and we were greeted by a renewed house, the old sleeping porch enlarged and opened up to create a light, airy space, with a brand new redwood deck just beyond, decorated with my grandmother’s cast iron patio furniture, which I recognized from my childhood 50 years ago.
Reconnecting with family. My mom, brothers Joseph and Conal and Conal’s partner Holly were there waiting for us, and we were very pleased to see them. Conal and I hadn’t spoken too much since our father’s death, when there had been a bit of tension between us over my being critical of Nonviolent Communication (one of those silly defensive academic petty things I sometimes fall into and now regret). However, last April he and Holly moved to Murray Creek and have been reconnecting with the family and the land. Then in May, there was a bad brush fire on their property (which is next to my mom’s), which was quite traumatic for them. Fortunately, their house survived without damage, and they have bounced back from the trauma and recovered from the fire-related PTSD; in fact, the fire appears to have even deepened their commitment to the valley. Meanwhile, my siblings and I pulled together to offer them emotional support. I am very grateful to them also for the support they have been able to provide to my mom.
My mom looked better than she had for a long while, getting around well with or without her walking sticks, and appears to have new energy and focus now that the house is fully repaired and renewed. After my dad died, it was really not clear what would happen next at Murray Creek. My mom took in a boarder, who has been living in the lower house, and my mom took to wandering among my siblings’ houses on a kind of hejira while her house was being repaired. Now, it seems, she is home again and to stay, and some kind of balance is being restored to the valley.
Climbing Mountains. So my brothers and I spent time catching up with each other. I heard about Joseph’s recent trip to Africa, and his and Ebru (my Turkish sister-in-law)’s trek up Mount Kllimanjaro, a 7-day journey from rain forest to glaciers that put Kenneth and my recent adventure climbing Wyndburgh Hill to shame. And we heard about Conal’s plans, his rediscovered love for the valley, how he and Holly are dealing with the aftermath of the fire, their plans for fixing up their house, and so on. Conal and Joseph plunged into the most obscure computer programming talk I have heard in a long time. It was wonderful! Then, yesterday, Kenneth and I got up and ran the Murray Creek Road, an annual ritual of ours, a demanding 10K course of steep up and down hills, first in the cool of the valley, then as we climbed the steep road out of the valley to the top of the ridge, warmer in the summer morning sun. At the end, we sprinted to make it in less than an hour.
Physical labor. Then, after breakfast, we tackled the large pile of chipped wood mulch in back of the house, left by the tree trimmers. Over the next two hours, as the day got progressively hotter, Joseph, Kenneth and I shifted a couple tons of wood chips, spreading them over the back garden, halfway up the hill to the water tank. Kenneth and I lasted about two hours at this back-breaking labor, so out of character for us that virtually the only place we engage in this level of physical labor is here, in Murray Creek. Finally, Joseph, who hadn’t been out running first thing in the morning, worked on for another half hour spreading the chips around, while Kenneth collapsed on his bed and I took a much needed shower.
Meteor Shower. There is something both deeply satisfying and cleansing about this level of physical labor, and so we lounged around for most of the rest of the day, talking among ourselves until it was time for Joseph to go back to San Francisco. Kenneth fell asleep about seven, I worked on my APA presentation for Friday. Finally, about 11 we went out to watch the Persiad meteor shower. One of my favorite things about Murray Creek is the stars. We went out onto my mom’s new deck, sat down on those wrought iron patio chairs, lay back, and watched the deep night sky. The Milky Way was emblazoned, north to south, across the sky. Watching for meteors is always a bit tedious, and some years it is totally fruitless, but this year we were lucky: Diane and I each saw three shooting stars in about twenty minutes. They weren’t the same three meteors, but it was enough to satisfy our desire for awe-inspiring but evanescent sights. We went inside and crashed.
Labyrinth Ritual. This morning, we got up and, after breakfast, walked down to the labyrinth, my mom moving right along with her trekking poles. She paused at the entrance to the labyrinth, and said, ”We have been saying a little something before entering the labyrinth; we said, “Go ahead.” She said, “O luminous spirits, who inhabit this place, watch over us here, and God bless you.” Then she paused, and said, “We usually say it three times.” We gestured, “Go ahead.” And she did.
Then she led the way into the labyrinth. Conal and Holly had recently run weed-eaters over the labyrinth, so it was better tended and neater than usual. The mention of luminous beings inhabiting the labyrinth made me think about all the other beings who inhabit it, mostly the gophers, the evidence of whose tunneling was everywhere along the way; I thought of dark beings and light ones; mundane and the numinous. Over the past year, my mom has come to understand the labyrinth as an entrance to the spirit world, a focus point for powers and energies or spirits to cross between our world and the other worlds posited in the shamanistic world view. I suppose the gopher holes could be thought of a symbolic entrances to the lower world.
My mom, me, Diane and Kenneth walked to the center of the labyrinth, faced the four cardinal directions, touched the Mother and Father Stones. There was a crystal resting on the Father Stone, very similar to the one that Kenneth and I had left on top of Wyndburgh Hill only two weeks before. I picked it up, looked at it, set it down. Then the four of us hugged, and we went back out.
Chinese Dam. After that, we walked up the road to Conal and Holly’s house, got the tour of their place and plans, then went down to the Chinese Dam. Sometime during the California Gold Rush (1848-), probably around 1863 (according to www.calgoldrush.com/part3/03asians.html, this was the peak of Chinese mining efforts), some of the 25,000 Chinese gold miners in the California Mother Lode at the time, got together and built a 3 –5 meter high dam across Murray Creek, just below where Conal and Holly’s house is today. The remains of this dam are on their property, and in Scotland would be considered to be an historical site of archeological significance and listed in the national register. The dam appears to have been a dry stone structure, somehow sealed on the upriver side. The dam would have created an artificial lake extending several hundred meters up the valley. Water was then diverted into along ditch on the south side of the valley for use in fluming gold ore. We don’t know exactly where the many tons of stone used to construct the dam came from; undoubtedly much of it came from the river bed. After the gold ran out, the dam was opened to let the river flow through unimpeded; much of the stone appears to have been carried downstream, on my parents’ property. We don’t really know any more than that about the dam or the people that built it!
We left shortly after our visit to the Chinese Dam, a too-short visit, but satisfying on multiple levels. Onward to San Francisco & APA!
Sunday, August 12, 2007
Weighting Studies in Meta-Analysis
Entry for 11 August 2007:
[This is one of my technical research entries, but it also shows how my mind works.]
Beth and I are working away on the meta-analysis of Person-Centred and Experiential therapy outcome research. So far, most of what Beth has been up to is re-analyzing the data I previously analyzed, in order to check on my calculations and ratings. This is so that we will be able to calculate inter-rater reliability, which considered to be the standard for meta-analysis. Of course, doing this with someone else means that various things that I have been doing my own way for years (through the previous generations of the analysis) now come into question and have to be discussed and clarified. This is a good process but sometimes challenging.
For example, should we use weighting in estimating effect sizes? That is, should we give some studies more weight than others in calculating the overall effect sizes? I learned in grad school that weighting rarely improves measurement, and so I have tended to distrust such approaches. I have been using weighting by sample size as one of the two ways of estimating overall effects; this appears to be the most common form of weighting, but does raise questions about significant testing, because is creates statistical nonindependence problems. (That’s because when you weight by sample size you the unit for significance testing becomes the client rather than the study, and clients within studies tend to share variance for various reasons, including having the same therapist, being in the same therapy and filling out the same instruments at the same times, etc.)
There are complicated methods for correcting sample size estimates, but these are cumbersome. In some ways, also, they seem to me to miss the point, which is that the true unit in meta-analysis is the study. The original idea of meta-analysis, as developed by Gene Glass, was that meta-analysis works with populations of studies. This makes sense to me because it is clear that there are wild differences between studies in clients, therapists, treatment, methods and effects. But of course, some of the studies in the meta-analysis we are doing have a sample size of five (our lower limit), while one or two of the larger German studies in our sample have samples of 1400. Given this, both weighting all studies the same (“unit weighting”) nor weighting by sample size seem problematic.
This is probably why Bruce Wampold recommends weighing by the inverse of the error: Smaller samples tend to have less reliable error estimates (that is, the error estimates have more error), which also creates bias in effect estimates (for this reason, I’ve been using a separate correction for small sample bias). Using the inverse of the error (1 divided by the standard deviation or standard error of the mean), gives more weight to studies with small error. Such studies tend to be more tightly designed and to sample clients more narrowly, which would give more weight to controlled studies and less weight to naturalistic studies.
Hmm… whatever you do, there is always problem. If you believe that there is a single True Effect Size, then you will want to weight by inverse error or sample size, in the belief that doing this will lead you closer to Truth. If you are more of a relativist, a methodological pluralist, or even a critical realist, then you will be suspicious of the search for singular Truth, but will be more interested in looking at things from various angles and looking at the points of convergence and divergence. Thus, it makes sense to me to estimate overall effect sizes in at least two different ways: Unit weighting (treating all studies as equal) and sample-size weighting. When I have done this in the past, it has sometimes produced virtually identical results, and at other times has produced slightly different results (a difference of .13 sd is the largest I’ve found so far), indicating (a) that weighting didn’t make much difference; and (b) that the estimates are fairly robust. Relying on single weighting solutions doesn’t provide this kind of information!
[This is one of my technical research entries, but it also shows how my mind works.]
Beth and I are working away on the meta-analysis of Person-Centred and Experiential therapy outcome research. So far, most of what Beth has been up to is re-analyzing the data I previously analyzed, in order to check on my calculations and ratings. This is so that we will be able to calculate inter-rater reliability, which considered to be the standard for meta-analysis. Of course, doing this with someone else means that various things that I have been doing my own way for years (through the previous generations of the analysis) now come into question and have to be discussed and clarified. This is a good process but sometimes challenging.
For example, should we use weighting in estimating effect sizes? That is, should we give some studies more weight than others in calculating the overall effect sizes? I learned in grad school that weighting rarely improves measurement, and so I have tended to distrust such approaches. I have been using weighting by sample size as one of the two ways of estimating overall effects; this appears to be the most common form of weighting, but does raise questions about significant testing, because is creates statistical nonindependence problems. (That’s because when you weight by sample size you the unit for significance testing becomes the client rather than the study, and clients within studies tend to share variance for various reasons, including having the same therapist, being in the same therapy and filling out the same instruments at the same times, etc.)
There are complicated methods for correcting sample size estimates, but these are cumbersome. In some ways, also, they seem to me to miss the point, which is that the true unit in meta-analysis is the study. The original idea of meta-analysis, as developed by Gene Glass, was that meta-analysis works with populations of studies. This makes sense to me because it is clear that there are wild differences between studies in clients, therapists, treatment, methods and effects. But of course, some of the studies in the meta-analysis we are doing have a sample size of five (our lower limit), while one or two of the larger German studies in our sample have samples of 1400. Given this, both weighting all studies the same (“unit weighting”) nor weighting by sample size seem problematic.
This is probably why Bruce Wampold recommends weighing by the inverse of the error: Smaller samples tend to have less reliable error estimates (that is, the error estimates have more error), which also creates bias in effect estimates (for this reason, I’ve been using a separate correction for small sample bias). Using the inverse of the error (1 divided by the standard deviation or standard error of the mean), gives more weight to studies with small error. Such studies tend to be more tightly designed and to sample clients more narrowly, which would give more weight to controlled studies and less weight to naturalistic studies.
Hmm… whatever you do, there is always problem. If you believe that there is a single True Effect Size, then you will want to weight by inverse error or sample size, in the belief that doing this will lead you closer to Truth. If you are more of a relativist, a methodological pluralist, or even a critical realist, then you will be suspicious of the search for singular Truth, but will be more interested in looking at things from various angles and looking at the points of convergence and divergence. Thus, it makes sense to me to estimate overall effect sizes in at least two different ways: Unit weighting (treating all studies as equal) and sample-size weighting. When I have done this in the past, it has sometimes produced virtually identical results, and at other times has produced slightly different results (a difference of .13 sd is the largest I’ve found so far), indicating (a) that weighting didn’t make much difference; and (b) that the estimates are fairly robust. Relying on single weighting solutions doesn’t provide this kind of information!
Friday, August 10, 2007
Ohio vs. Scotland
Entry for 9 August 2007:
We’ve returned to the US for three weeks of business (home repairs), conferences (Humanistic Therapy and American Psychological Association meetings in San Francisco), work (Rogers chapter, catching up on email, and getting ready for the conferences), and maybe a bit of vacation.
After the cool Scottish summer, it was a bit of a shock to be reminded of what a hot, humid Ohio summer feels like. We have had the house closed up with the air-conditioning running the entire time we’ve been here. So much for sitting on the front porch reading and working in the summer cool breeze! I tried this a couple of times, but my computer and I both got too hot.
Every time we come home we are confronted by the subtle and not-so-subtle differences between Scotland and Ohio. Not-so-subtle and more superficial includes, the heat and driving on the opposite side of the road (both Diane and I have had to watch ourselves carefully this time). But there are many revealing differences:
Scotland: Many things are an effort: driving (blocked and one-way streets); parking (in Hyndland); understanding (many) people; getting to the downtown (by train, never by car, the traffic patterns are too confusing for us).
Ohio: It seems designed to be easy, convenient: the streets are wide, laid out in a grid pattern, so there are million ways to get from one place to another; parking is plentiful; we follow the speech of the locals without effort; to get to the downtown, we can hop on the freeway that runs a half-block from our house, and be downtown in 10 minutes.
In Hyndland,our neighborhood in Glasgow, we live in a flat in a lovely 100-year-old red sandstone tenement building; we have no garden of our own, and the flat is full of architectural details like stained glass, bay window, decorated tile fireplace, recessed cupboards, coved ceiling.
Toledo, Ohio: We live in a lovely but small detached wood frame house about 80 years old. (We would not feel comfortable living in a US “tenement,” which is the name of a stigmatized building style found primarily in poor neighborhoods.) Here, we have a small garden, which because we aren’t here very much, is somewhat overrun with weeds (although cherry tomatoes did get planted this year and have started to produce tasty fruit). Compared to most US houses, ours is full of architectural details (it was the perpendicular or tudor arches in various places that we fell in love with after having spent a year in England in the mid 1980’s), but these seem few and plain compared to our place in Glasgow.
Most fundamentally, as Diane and I concluded after a long talk this evening, in Ohio we feel grounded in the place, the culture and the people. This feels comfortable and familiar; even the dangers and inconveniences are familar, like a broken piece of favorite furniture.
In Scotland, by contrast, even after a year, we are still just beginning to feel at home and grounded in the place. Misunderstandings are more common, and we generally feel we have to be careful not to say something silly or annoying. We don't really have a feel for much of people's ordinary lives and their courses.
To give a flavor of the some of the cultural differences, consider this incident:\
We’ve returned to the US for three weeks of business (home repairs), conferences (Humanistic Therapy and American Psychological Association meetings in San Francisco), work (Rogers chapter, catching up on email, and getting ready for the conferences), and maybe a bit of vacation.
After the cool Scottish summer, it was a bit of a shock to be reminded of what a hot, humid Ohio summer feels like. We have had the house closed up with the air-conditioning running the entire time we’ve been here. So much for sitting on the front porch reading and working in the summer cool breeze! I tried this a couple of times, but my computer and I both got too hot.
Every time we come home we are confronted by the subtle and not-so-subtle differences between Scotland and Ohio. Not-so-subtle and more superficial includes, the heat and driving on the opposite side of the road (both Diane and I have had to watch ourselves carefully this time). But there are many revealing differences:
Scotland: Many things are an effort: driving (blocked and one-way streets); parking (in Hyndland); understanding (many) people; getting to the downtown (by train, never by car, the traffic patterns are too confusing for us).
Ohio: It seems designed to be easy, convenient: the streets are wide, laid out in a grid pattern, so there are million ways to get from one place to another; parking is plentiful; we follow the speech of the locals without effort; to get to the downtown, we can hop on the freeway that runs a half-block from our house, and be downtown in 10 minutes.
In Hyndland,our neighborhood in Glasgow, we live in a flat in a lovely 100-year-old red sandstone tenement building; we have no garden of our own, and the flat is full of architectural details like stained glass, bay window, decorated tile fireplace, recessed cupboards, coved ceiling.
Toledo, Ohio: We live in a lovely but small detached wood frame house about 80 years old. (We would not feel comfortable living in a US “tenement,” which is the name of a stigmatized building style found primarily in poor neighborhoods.) Here, we have a small garden, which because we aren’t here very much, is somewhat overrun with weeds (although cherry tomatoes did get planted this year and have started to produce tasty fruit). Compared to most US houses, ours is full of architectural details (it was the perpendicular or tudor arches in various places that we fell in love with after having spent a year in England in the mid 1980’s), but these seem few and plain compared to our place in Glasgow.
Most fundamentally, as Diane and I concluded after a long talk this evening, in Ohio we feel grounded in the place, the culture and the people. This feels comfortable and familiar; even the dangers and inconveniences are familar, like a broken piece of favorite furniture.
In Scotland, by contrast, even after a year, we are still just beginning to feel at home and grounded in the place. Misunderstandings are more common, and we generally feel we have to be careful not to say something silly or annoying. We don't really have a feel for much of people's ordinary lives and their courses.
To give a flavor of the some of the cultural differences, consider this incident:\
I was out mowing my front lawn the other night. It was still very hut and muggy, and I was pushing my old-fashioned push-mower over the bumpy, ratty patch between the sidewalk and the street.Fortunately, at this point, his mom, who was getting into her SUV, called him to her, and he went off with her, leaving me somewhat bemused, to finish mowing my lawn. “Toto,” I said to myself, “I don’t think we are in Scotland anymore!”
A small African-American boy of about 5 walks slowly by, stops and turns to me, saying, “What’s that?”
I reply, “You’ve never seen one of these before?”
He says, “No… only in cartoons!”
“Well,” I say, “it’s an old fashioned lawn mower.” Since I was a college professor and find it too difficult to stop educating people, even my 5-year-old neighbor, I go on, “In the old days, everyone used these to mow their lawns…. or they used sheep.”
“What’s that?” he wants to know.
“You know, it has wool all over it and it goes ‘baaaa’?
”That’s a goat!” he tells me (apparently my sheep imitation was not as good as I thought it was.
Monday, August 06, 2007
Last Saturday Adventure of the Summer: Scottish Crannog Centre
Entry for 5 August (in transit to USA):
When Kenneth started his volunteer work for the Glasgow City Museums, he spent his first day sorting slides in their Resource Centre in the southern suburbs of the city. Several of the 600 slides he sorted that day were of “crannogs”. What, he asked Katinka Stentoft, his mentor, is a crannog? Crannogs, she replied, were ancient forts or dwellings built in Scottish lochs.
We were all intrigued by this ancient form of builty culture with the strange name. I was particularly interested because I remembered reading with fascination about the lake dwellers of prehistoric Switzerland when I was a child. What would it be like to live in a wooden town built over the waters of an lake?
When we mentioned crannogs to our friend Margaret MacKay from St. Mary’s (she’s head of Celtic Studies at U. of Edinburgh), she told us there was a reconstructed one in Tayside. So, when it came to planning our last Saturday Adventure of the summer – and Kenneth’s last excursion before he went back to the US, I proposed a crannog visit. Beth and her daughter Ana were just back from the person-centred conference in New York City, so I invited them along as well, and they readily agreed to join us.
We picked them up a little after 9 on Saturday morning. We made a wrong turn leaving Glasgow and ended up heading toward Carlisle, but soon were back on track and on our way to Tayside, Loch Tay, to be exact. It took us a bit more than two hours to get to the Scottish Crannog Centre. The reconstructed crannog sits at the end of a short wooden causeway, a round house built on log pilings, its tall thatched roof sitting jauntily over of the choppy waters of the wind-swept loch. It was a strange and inspiring sight.
The crannog at the Scottish Crannog Centre is a reconstruction of the 2500-year old Oakbank iron age crannog across Loch Tay, which has been extensively excavated and studied. That, and a fair amount of inference, guesswork and experimentation. Our guide, a young computer science graduate named Ewan, was a real crannog person, having been involved with the reconstruction since he was 11. He took us out along the wooden causeway, made of logs laid next to each other, which made a bumpy walk, especially for Ana, who was still using one crutch after her recent knee injury.
It was dark inside the crannog. The floor was covered with a thick layer of bracken, making it very soft to walk on, and, according to our guide, nice for sleeping also. We were startled to discover that there was no hole at the top of the roof; Ewan explained that the smoke dissipates through the thatch, and that leaving a hole in the roof would cause the fire to burn too quickly, putting out sparks and burning the place down. There was also a loft; possibly there was originally a second floor in the same level as the loft. There are partitions made of woven willow branches, some of which were used to house the more vulnerable livestock.
We spent some time experiencing this reconstructed ancient structure, before going back out into the bright sunshine to watch demonstrations of wood carving using various versions of a lathe, methods for making holes in stones (e.g., for fishing weights), grinding grain (spelt was big), and using tinder to start a fire.
All in all we all loved this site; it was one of our best Saturday Adventures. We spent the rest of the day winding our way back through the Highlands, passing alongside of one wonderful loch after another, until we finally made it back home in time to start packing up for our trip the USA the next day. Crannogs were now longer an interesting abstraction to us, but instead a lived experience.
When Kenneth started his volunteer work for the Glasgow City Museums, he spent his first day sorting slides in their Resource Centre in the southern suburbs of the city. Several of the 600 slides he sorted that day were of “crannogs”. What, he asked Katinka Stentoft, his mentor, is a crannog? Crannogs, she replied, were ancient forts or dwellings built in Scottish lochs.
We were all intrigued by this ancient form of builty culture with the strange name. I was particularly interested because I remembered reading with fascination about the lake dwellers of prehistoric Switzerland when I was a child. What would it be like to live in a wooden town built over the waters of an lake?
When we mentioned crannogs to our friend Margaret MacKay from St. Mary’s (she’s head of Celtic Studies at U. of Edinburgh), she told us there was a reconstructed one in Tayside. So, when it came to planning our last Saturday Adventure of the summer – and Kenneth’s last excursion before he went back to the US, I proposed a crannog visit. Beth and her daughter Ana were just back from the person-centred conference in New York City, so I invited them along as well, and they readily agreed to join us.
We picked them up a little after 9 on Saturday morning. We made a wrong turn leaving Glasgow and ended up heading toward Carlisle, but soon were back on track and on our way to Tayside, Loch Tay, to be exact. It took us a bit more than two hours to get to the Scottish Crannog Centre. The reconstructed crannog sits at the end of a short wooden causeway, a round house built on log pilings, its tall thatched roof sitting jauntily over of the choppy waters of the wind-swept loch. It was a strange and inspiring sight.
The crannog at the Scottish Crannog Centre is a reconstruction of the 2500-year old Oakbank iron age crannog across Loch Tay, which has been extensively excavated and studied. That, and a fair amount of inference, guesswork and experimentation. Our guide, a young computer science graduate named Ewan, was a real crannog person, having been involved with the reconstruction since he was 11. He took us out along the wooden causeway, made of logs laid next to each other, which made a bumpy walk, especially for Ana, who was still using one crutch after her recent knee injury.
It was dark inside the crannog. The floor was covered with a thick layer of bracken, making it very soft to walk on, and, according to our guide, nice for sleeping also. We were startled to discover that there was no hole at the top of the roof; Ewan explained that the smoke dissipates through the thatch, and that leaving a hole in the roof would cause the fire to burn too quickly, putting out sparks and burning the place down. There was also a loft; possibly there was originally a second floor in the same level as the loft. There are partitions made of woven willow branches, some of which were used to house the more vulnerable livestock.
We spent some time experiencing this reconstructed ancient structure, before going back out into the bright sunshine to watch demonstrations of wood carving using various versions of a lathe, methods for making holes in stones (e.g., for fishing weights), grinding grain (spelt was big), and using tinder to start a fire.
All in all we all loved this site; it was one of our best Saturday Adventures. We spent the rest of the day winding our way back through the Highlands, passing alongside of one wonderful loch after another, until we finally made it back home in time to start packing up for our trip the USA the next day. Crannogs were now longer an interesting abstraction to us, but instead a lived experience.
Friday, August 03, 2007
Rogers and Sarbin
Entry for 3 August 2007:
Barry Farber (an old SPR friend who teaches at Columbia University) and I are writing an chapter on Carl Rogers for a book on important psychotherapy researchers. To help with this, I managed to get a hold of an advance copy of Howard Kirschenbaum’s forthcoming The Life and Work of Carl Rogers. I’m supposed to be skimming it for relevant bits on his career as a psychotherapy researcher, but it is a fascinating read, and I keep getting distracted by interesting bits.
This morning I ran across a curious synchronist bit in the chapter on Rogers’ Ohio years. In 1940, a junior faculty member at the University of Minnesota invited Rogers to come give a lecture on counselling and psychotherapy. This turned out to an historically important lecture, because it was here that Rogers began to lay out his critique of directive approaches to counselling and to describe a more growth-oriented alternative. In order to support his point, he quoted a passage from a recent article, in which the author described a not-very-successful attempt to get an undergraduate student to change his major.
What really caught me eye, however, was the identity of both the junior faculty member and the counselor in the quoted passage: It was Ted Sarbin, my undergraduate mentor from the University of California at Santa Cruz!
Kirschenbaum notes that Rogers reported having been mortified to learn that the counselor in the passage was in the audience and was fact Ted Sarbin. However, Kirschenbaum also interviewed Sarbin , who said that he was impressed by Rogers' tact in mentioning him by name! My sense of Ted Sarbin is that he would not have taken it personally and would have been amused rather than offended by Rogers’ using him as an example of bad practice. In my experience, that was the kind of person he was. When I took his course, “The Social Psychology of Deviant Conduct” at UC Santa Cruz in 1971 or so, he told us many interesting and unusual stories of clients he saw, not all of whom had good outcomes.
I am fascinated by this conjunction between two key important figures in my development as a psychologist. It’s ironic that I am writing about Carl Rogers, whom I only met once (casually, at APA in 1980), whereas I knew Ted Sarbin fairly well. But it made me think again about Ted Sarbin and the role he played supporting my intellectual development: His sparking my interest in metaphor; his critique of traditional mental health concepts and practices, different but parallel to Rogers’; the strength of his loyalty to his students, which has come to mean more to me over the years as I have appreciated how unusual it is; and his intellectually adventurous spirit and courage.
Ted Sarbin died in 2005, at the age of 94. Three years before he died, he contacted me to invite me to my first “Sarbin dinner.” This dinner was a 40-year tradition that he held for his students and friends at APA, during which he gave out the Role Theorist of the Year Award, the remit for which had broadened over the years to include contributions to narrative and critical psychology as well.
I had not seen him in years, but there was something he wanted to tell me and the others there: That for years he had felt badly about not having properly credited my thesis research in a book chapter he had written a couple of years after I graduated. For my part, at the time I had been a bit bothered by the incident, which I soon came to understand as one of those stupid, embarrassing things that sometimes happens in producing a paper; but I had long since moved on. Nevertheless, I was immensely moved and touched by his public confession. After that, I attended two more Sarbin dinners at APA. In the summer of 2005, he contacted me twice to make sure I was going to be there, and then at that years’ APA conference in Washington, DC, knowing that he was dying of cancer, he gave out two of his awards, the last of which he gave to me. He died ten days later. The award plaque he gave me is one of my most treasured mementos.
All of this came back to me this morning as I read about the long ago encounter of these two key figures in 20th century American psychology, who are also two of my own most important influences and role models. Ted Sarbin never became a person-centred therapist – he followed no school, although he is the founder of narrative psychology - but he and Carl Rogers shared much in common, and I think I can say that he always supported my growth tendency and inspired me to follow it, while treating me with acceptance, empathy and integrity. It is one of my main goals to try to live up to his example.
Barry Farber (an old SPR friend who teaches at Columbia University) and I are writing an chapter on Carl Rogers for a book on important psychotherapy researchers. To help with this, I managed to get a hold of an advance copy of Howard Kirschenbaum’s forthcoming The Life and Work of Carl Rogers. I’m supposed to be skimming it for relevant bits on his career as a psychotherapy researcher, but it is a fascinating read, and I keep getting distracted by interesting bits.
This morning I ran across a curious synchronist bit in the chapter on Rogers’ Ohio years. In 1940, a junior faculty member at the University of Minnesota invited Rogers to come give a lecture on counselling and psychotherapy. This turned out to an historically important lecture, because it was here that Rogers began to lay out his critique of directive approaches to counselling and to describe a more growth-oriented alternative. In order to support his point, he quoted a passage from a recent article, in which the author described a not-very-successful attempt to get an undergraduate student to change his major.
What really caught me eye, however, was the identity of both the junior faculty member and the counselor in the quoted passage: It was Ted Sarbin, my undergraduate mentor from the University of California at Santa Cruz!
Kirschenbaum notes that Rogers reported having been mortified to learn that the counselor in the passage was in the audience and was fact Ted Sarbin. However, Kirschenbaum also interviewed Sarbin , who said that he was impressed by Rogers' tact in mentioning him by name! My sense of Ted Sarbin is that he would not have taken it personally and would have been amused rather than offended by Rogers’ using him as an example of bad practice. In my experience, that was the kind of person he was. When I took his course, “The Social Psychology of Deviant Conduct” at UC Santa Cruz in 1971 or so, he told us many interesting and unusual stories of clients he saw, not all of whom had good outcomes.
I am fascinated by this conjunction between two key important figures in my development as a psychologist. It’s ironic that I am writing about Carl Rogers, whom I only met once (casually, at APA in 1980), whereas I knew Ted Sarbin fairly well. But it made me think again about Ted Sarbin and the role he played supporting my intellectual development: His sparking my interest in metaphor; his critique of traditional mental health concepts and practices, different but parallel to Rogers’; the strength of his loyalty to his students, which has come to mean more to me over the years as I have appreciated how unusual it is; and his intellectually adventurous spirit and courage.
Ted Sarbin died in 2005, at the age of 94. Three years before he died, he contacted me to invite me to my first “Sarbin dinner.” This dinner was a 40-year tradition that he held for his students and friends at APA, during which he gave out the Role Theorist of the Year Award, the remit for which had broadened over the years to include contributions to narrative and critical psychology as well.
I had not seen him in years, but there was something he wanted to tell me and the others there: That for years he had felt badly about not having properly credited my thesis research in a book chapter he had written a couple of years after I graduated. For my part, at the time I had been a bit bothered by the incident, which I soon came to understand as one of those stupid, embarrassing things that sometimes happens in producing a paper; but I had long since moved on. Nevertheless, I was immensely moved and touched by his public confession. After that, I attended two more Sarbin dinners at APA. In the summer of 2005, he contacted me twice to make sure I was going to be there, and then at that years’ APA conference in Washington, DC, knowing that he was dying of cancer, he gave out two of his awards, the last of which he gave to me. He died ten days later. The award plaque he gave me is one of my most treasured mementos.
All of this came back to me this morning as I read about the long ago encounter of these two key figures in 20th century American psychology, who are also two of my own most important influences and role models. Ted Sarbin never became a person-centred therapist – he followed no school, although he is the founder of narrative psychology - but he and Carl Rogers shared much in common, and I think I can say that he always supported my growth tendency and inspired me to follow it, while treating me with acceptance, empathy and integrity. It is one of my main goals to try to live up to his example.
Wednesday, August 01, 2007
Lunasa, Celtic Harvest Festival
Entry for 1 Aug 2007:
Today is Lunasa, Lugnasad, or Lammas, according the French version of the Wikipedia, the Celtic Harvest festival, the “gathering of Lug” (Lug is the king of the Gods in the Celtic pantheon). It was a time of truce, and for the equitable distribution of goods and wealth, and for the arrangement of marriages. It marks the end of Summer and the beginning of Autumn in the Celtic calendar. The may be some connection to ancient corn goddesses, who are still honored in the form of corn dollies, or little figures made of woven wheat straw (which Diane and I collect). Harvest festivals continue in the US today in the form of county fairs, typically held in July or August.
As my mom always notes when I phone her up to wish her well on one of these celtic seasonal celebrations (like Imbolc, Beltaine & Samain), this is also (around) the cross quarter day, that is, a day mid-was between a solstice and an equinox. Cross quarter days are the tipping point into the next season of the year. From now, the days will begin to shorten more rapidly. But for the moment, summer hovers, the days are lovely (generally, it is Scotland, after all), and there is a bit of time before Normal Life begins again.
Today is Lunasa, Lugnasad, or Lammas, according the French version of the Wikipedia, the Celtic Harvest festival, the “gathering of Lug” (Lug is the king of the Gods in the Celtic pantheon). It was a time of truce, and for the equitable distribution of goods and wealth, and for the arrangement of marriages. It marks the end of Summer and the beginning of Autumn in the Celtic calendar. The may be some connection to ancient corn goddesses, who are still honored in the form of corn dollies, or little figures made of woven wheat straw (which Diane and I collect). Harvest festivals continue in the US today in the form of county fairs, typically held in July or August.
As my mom always notes when I phone her up to wish her well on one of these celtic seasonal celebrations (like Imbolc, Beltaine & Samain), this is also (around) the cross quarter day, that is, a day mid-was between a solstice and an equinox. Cross quarter days are the tipping point into the next season of the year. From now, the days will begin to shorten more rapidly. But for the moment, summer hovers, the days are lovely (generally, it is Scotland, after all), and there is a bit of time before Normal Life begins again.
Subscribe to:
Posts (Atom)