The therapists using AI to make therapy better
Repper anticipates some reluctance. “This technology represents a challenge for therapists,” he says. “It’s as if they’ve got someone else in the room for the first time, transcribing everything they say.” To start with, Trent PTS is using Lyssn’s software only with trainees, who expect to be monitored. When those therapists qualify, Repper thinks, they may accept the monitoring because they are used to it. More experienced therapists may need to be convinced of its benefits.
The point is not to use the technology as a stick but as support, says Imel, who used to be a therapist himself. He thinks many will welcome the extra information. “It’s hard to be on your own with your clients,” he says. “When all you do is sit in a private room with another person for 20 or 30 hours a week, without getting feedback from colleagues, it can be really tough to improve.”
Freer agrees. At Ieso, therapists discuss the AI-generated feedback with their supervisors. The idea is to let therapists take control of their professional development, showing them what they’re good at—things that other therapists can learn from—and not so good at—things that they might want to work on.
For example, in a paper published in JAMA Psychiatry in 2019, Ieso researchers described a deep-learning NLP model that was trained to categorize utterances from therapists in more than 90,000 hours of CBT sessions with around 14,000 clients. The algorithm learned to discern whether different phrases and short sections of conversation were instances of specific types of CBT-based conversation—such as checking the client’s mood, setting and reviewing homework (where clients practice skills learned in a session), discussing methods of change, planning for the future, and so on—or talk not related to CBT, such as general chat.
The researchers showed that higher ratios of CBT talk correlate with better recovery rates, as measured by standard self-reported metrics used across the UK. They claim that their results provide validation for CBT as a treatment. CBT is widely considered effective already, but this study is one of the first large-scale experiments to back up that common assumption.
In a paper published this year, the Ieso team looked at clients’ utterances instead of therapists’. They found that more of what they call “change-talk active” responses (those that suggest a desire to change, such as “I don’t want to live like this anymore”) and “change-talk exploration” (evidence that the client is reflecting on ways to change) were associated with greater odds of reliable improvement and engagement. Not seeing these types of statements could be a warning sign that the current course of therapy is not working. In practice, it could also be possible to study session transcripts for clues to what therapists say to elicit such behavior, and train other therapists to do the same.
https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2748757
Quantifying the Association Between Psychotherapy Content and Clinical Outcomes Using Deep Learning
Question What aspects of psychotherapy content are significantly associated with clinical outcomes?
Findings In this quality improvement study, a deep learning model was trained to automatically categorize therapist utterances from approximately 90 000 hours of internet-enabled cognitive behavior therapy (CBT). Increased quantities of CBT change methods were positively associated with reliable improvement in patient symptoms, and the quantity of nontherapy-related content showed a negative association.
Meaning The findings support the key principles underlying CBT as a treatment and demonstrate that applying deep learning to large clinical data sets can provide valuable insights into the effectiveness of psychotherapy.
https://www.tandfonline.com/doi/epub/10.1080/10503307.2020.1788740
Understanding the relationship between patient language and outcomes in internet-enabled cognitive behavioural therapy: A deep learning approach to automatic coding of session transcripts
Objective: Understanding patient responses to psychotherapy is important in developing effective interventions. However, coding patient language is a resource-intensive exercise and difficult to perform at scale. Our aim was to develop a deep learning model to automatically identify patient utterances during text-based internet-enabled Cognitive Behavioural Therapy and to determine the association between utterances and clinical outcomes.
Method: Using 340 manually annotated transcripts we trained a deep learning model to categorize patient utterances into one or more of five categories. The model was used to automatically code patient utterances from our entire data set of transcripts (∼34,000 patients), and logistic regression analyses used to determine the association between both reliable improvement and engagement, and patient responses.
Results: Our model reached human-level agreement on three of the five patient categories. Regression analyses revealed that increased counter change-talk (movement away from change) was associated with lower odds of both reliable improvement and engagement, while increased change-talk (movement towards change or self-exploration) was associated with increased odds of improvement and engagement.
Conclusions: Deep learning provides an effective means of automatically coding patient utterances at scale. This approach enables the development of a data-driven understanding of the relationship between therapist and patient during therapy.