ArticlePDF Available

Reliability and Criterion-Related Validity of the California Psychotherapy Alliance Scales-Patient Version

Authors:

Abstract

Aspects of the reliability and criterion-related validity of the patient version of the California Psychotherapy Alliance Scales (CALPAS-P) were examined. The sample consisted of 147 patients consulting in private practice. Ss completed the 24-item CALPAS-P designed to assess 4 alliance dimensions. They also filled out questionnaires on symptomatology, intimacy problems, social desirability, and satisfaction. Coefficients of internal consistency varied from .43 to .73 for the 4 scales and reached .83 for the total CALPAS-P. Correlations among CALPAS-P scales ranged from .37 to .62. Patients', therapists', and treatments' characteristics were not related to CALPAS-P scales. Selected CALPAS-P scales were related to symptomatology and intimacy problems, whereas all scales were associated with satisfaction in therapy. No association was found between CALPAS-P scales and estimates of social desirability. Confirmatory factor analysis would help support the theoretically based dimensionality of the CALPAS-P. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... This insight gains importance when considering Flückiger et al.'s (2013) meta-analysis, which revealed an attenuated alliance-outcome correlation for ethnic minority versus majority clients, possibly co-determined by unaddressed cultural factors. Similarly alluding to the importance of ethnic-cultural conversations in psychotherapy with ethnic minority clients is its potential to strengthen therapy commitment, defined as a client's willingness to go through the complete process of therapy even at difficult moments (Gaston, 1991). A lack of commitment and early dropout are more prevalent in ethnic minority clients compared to majority clients, and can be attributed to the failure of treatment to meet the unique needs of ethnic minority clients (Owen et al., 2012). ...
... Therapy commitment (Sample 1-Wave 2 & Sample 2). The California Psychotherapy Alliance Scales-Patient Commitment subscale was used to measure the participants' commitment to show continued engagement to the therapeutic process, irrespective of monetary and time-related sacrifices and potential difficulties that occur (Gaston, 1991). An example item is: "Did you feel that even if you might have moments of doubt, confusion or mistrust, that overall therapy is worthwhile?" ...
... M = 4.97, SD = 1.03). Internal consistency rates were consistent with prior studies (Delsignore et al., 2014;Gaston, 1991). ...
Article
Objective: Addressing ethnic-cultural topics during the process of psychotherapy, i.e. broaching, is considered highly important for ethnic minority clients who consult mental health care services. Surprisingly little is known, however, about clients’ perception of a therapist’s broaching qualities, and how clients’ mental construction of broaching translates into behavioural broaching acts a therapist may display. Method: Based on previous work and nine in-depth interviews with ethnic minority clients, a client-rated measure of therapists’ broaching behaviour was developed and psychometrically evaluated in two samples. Sample 1 (N = 252 UK ethnic minority clients) was used to empirically delineate the factor structure of an initial item set. Participants were then resolicited to complete a revised item pool. Results: The empirical structure resulted in a final 25-item broaching instrument with five subscales probing into therapists’ broaching behaviour. This Broaching Assessment Scale (BrAS) was validated in Sample 2 (N = 239 US ethnic minority clients). Strict measurement invariance of the factor structure was observed across the two samples and distinctive correlational patterns with therapeutic process measures were found. Conclusion: The BrAS provides new insights on how sensitivity to ethnic-cultural topics can be targeted along its concrete features, and is a promising tool for conceptualizing culturally sensitive mental healthcare assessment.
... The HLS-14 was compared with the scale developed, and Pearson correlation analysis was performed on the measured data to test the criterion-related validity of this scale. If the correlation coefficient is ≥ 0.5 (p < 0.05), it is considered to have a good calibration [50]. The correlation between the total scores of the HLS-14 and the ML Scale was analyzed, and if the results showed that the total score of the Health Literacy Scale is significantly positively correlated with the total score of the medication literacy Scale, it indicated that the medication literacy scale had good criterion-related validity. ...
Article
Full-text available
Background Irrational drug use has become a global problem threatening human health. As future health professionals, medical college students’ medication literacy (ML) is critical. Their mastery of medication-related knowledge will directly affect public medication safety in the future. Methods The initial scale was developed through a literature review and was modified through expert consultation and student interviews to form the initial scale with 14 items. In 2020, a questionnaire survey was conducted among students in a medical college for item analysis, reliability test, validity test and other analyses. Results The total Cronbach’s α coefficient was 0.826 and split-half reliability was 0.852. The Cronbach’s α coefficients for functional medication literacy, communicative medication literacy, and critical medication literacy were 0.901, 0.858, and 0.851, respectively. The item-level content validity index (I-CVI) ranged from 0.833 to 1.000 (≥ 0.78). Factor analysis of 14 items showed that KMO = 0. 852(> 0.7) and Bartlett’s spherical test p < 0.001, indicating that the data are very suitable for factor analysis. Three principal axis factors were extracted by principal component analysis, and the total variance interpretation rate was 69.031% (> 40%). The confirmatory factor analysis identified a three-factor model and showed goodness of fit indices for the scale: The χ²/df = 2.623, The Goodness of Fit Index (GFI) = 0.905, The Comparative Fit Index (CFI) = 0.950, Normed Fit Index (NFI) = 0.922, Tucker-Lewis index (TLI) = 0.938, and The Root Mean Square Error of Approximation (RMSEA) = 0.078. Conclusion A new scale for evaluating the medication literacy of Chinese medical college students was preliminarily developed, demonstrating good reliability and validity. Itcan be used as a preliminary measurement tool for assessing medical students’ medication literacy. However, due to the limitations of this study, the practical application of the scale needs to be further examined in a larger sample and should be refined in future studies.
... A preliminary review of the literature found several measuring instruments focusing on this construct: (a) the Vanderbilt Psychotherapy Process Scale and the Vanderbilt Therapeutic, which assesses positive and negative aspects of the therapist's and patient's behaviour and attitudes that impact on the relationship and the success of the therapy (Suh et al. 1989); (b) California Psychotherapy Alliance Scale, which assesses the involvement of both parties in the therapeutic process, process and also has a version to be used in the context of group therapy (Gaston 1991); (c) Helping Alliance Questionnaire, which allows the patient to assess the relevance of the therapist and the therapy (Luborsky et al. 1996); (d) Agnew Relationship Measure, a self-report instrument that assesses the patient-therapist therapeutic alliance (Agnew-Davies et al. 1998); (e) Relationship Inventory, developed by Barrett-Lennard, based on Carl Rogers' person-centred approach (Barrett-Lennard 1986); and (f) Working Alliance Inventory Greenberg 1986, 1989) and Working Alliance Inventory-Short Revised (Munder et al. 2010), which are among the most widely used instruments for assessing the quality of the therapeutic alliance in research, following Bordin's model of the therapeutic alliance. ...
Article
Full-text available
The therapeutic relationship plays a crucial role in nursing care for people with mental illness. Adopting a systemic and person-centred approach that considers the individual experiences and needs of the person is paramount. However, no instruments were found in the literature designed to evaluate the nurse-patient therapeutic relationship from the perspective of a person with mental illness. This study aimed to develop and evaluate the psychometric properties of an instrument for assessing the quality of the nurse-patient therapeutic relationship from the patient's perspective. An e-Delphi study was conducted to develop the assessment tool, and a psychometric study was carried out to examine its psychometric properties. The sample comprised 240 adults with mental illness. Internal consistency was assessed using Cronbach's alpha and the Omega coefficient. The final structure of the assessment instrument included 24 items distributed across two factors, explaining 64.2% of the variance. Cronbach's alpha was 0.94, and the Omega coefficient was 0.96. The intraclass correlation coefficient was 0.84 (95% CI: 0.66-0.92). The therapeutic relationship assessment scale (TRAS-Patient) shows good psychometric properties. This is a relevant tool for assessing the quality of the nurse-patient therapeutic relationship from the patient's perspective, thus promoting a patient-centred approach and responding to the patient's needs. Mental health nurses can access a tool for evaluating the nurse-patient therapeutic relationship centred on disciplinary knowledge. This enables patient involvement in care, enhanced care and person-centred practice.
... These dimensions of therapeutic alliance emerged from the work of Freud (1937;1958), Rogers (1957), Bordin (1979) and Sterba (1934) on the subject. Previous studies using the CALPAS reported Cronbach alphas between 0.43 and 0.77 for the subscales and between 0.83 and 0.89 for the total score (Delsignore et al., 2014;Gaston, 1991). ...
... The measure utilizes an intensity scale of 1 to 7, where the higher the reported score, the greater the therapeutic alliance. The scale is reported to have internal consistency reliability of ⍺=0.84 and sufficient criterionrelated validity [52]. ...
Article
Full-text available
Background Law enforcement frequently responds to substance abuse and mental health crises. Crisis stabilization units (CSUs) operate as a public-receiving facility to provide short-term stabilization services for individuals experiencing these crises and offer law enforcement an important alternative to arrest. However, there is limited understanding about how and when law enforcement decides to use CSUs. There is also the challenge of retaining individuals in treatment after CSU stabilization to prevent future crises and persistent engagement with police. This study will respond to these gaps by exploring CSU procedures and examining the feasibility and acceptability of a technology-assisted mobile aftercare intervention designed for individuals brought to a CSU by law enforcement. Methods This study will consist of three aims. Aim 1 will include qualitative interviews with law enforcement and CSU-affiliated mental health staff (n=80) regarding CSU utilization and collaboration logistics between the groups. Findings from Aim 1 will be synthesized for the development of an implementation guide of our intervention, mobile, and technology-assisted aftercare, designed for individuals brought to a CSU by law enforcement, during Aim 2. During Aim 2, intervention services will be pilot-tested for 6 months through a small sample (n=24), randomized control trial (RCT). Control participants will receive standard services available for individuals discharging from a CSU. Treatment participants will receive the mobile aftercare intervention. Qualitative and quantitative data will be collected at 2 weeks, 3 months, and 6 months post-recruitment for all study participants. Aims 1 and 2 will inform the design of a multi-site RCT to compare CSUs with and without mobile and technology-assisted aftercare (Aim 3). Discussion The study will offer decision making and procedural insight into law enforcement use of CSUs as an alternative to jail and provide opportunities to inform that process. This research will provide outcome trends for those who go through standard CSU services compared to those who receive mobile and technology-assisted aftercare services. The current study will inform a larger RCT efficacy study of CSUs with and without technology-assisted aftercare services. Trial registration This study was registered on ClinicalTrials.gov (reference #NCT04899934) on May 25, 2021.
Article
Full-text available
Background The therapeutic alliance (TA) is a robust and pantheoretical predictor of treatment outcome in Face-to-Face- (F2F-) and Online-psychotherapy (Online-PT). Many authors have proposed several conceptualizations of TA, which are oftentimes operationalized. The resulting diversity of conceptualizations and measures is presented in this review. Methods We performed a three-parted literature search for self-report-instruments of TA in individual, voluntary F2F-PT with adults (1. utilization of past reviews, 2. systematic literature search yielding 5,205 articles, 3. reference lists). Analogously, we conducted a systematic literature search for instruments of TA in the Online-setting (yielding 200 articles). Additionally, we analyzed the content of the instruments qualitatively. Results A current overview of 48 instruments for measuring TA (46 for F2F-PT, 2 for Online-PT) including their conceptual backgrounds, characteristics and main content aspects is presented. Most instruments (n = 24) operationalize one or more theoretical conceptualizations of TA. Other instruments are adaptation/syntheses of existing measures (n = 14), based on literature searches (n = 3) or on an empirical survey (n = 3) and two instruments provide no conceptual background information. The content of the instruments mainly focused on the following aspects: 1. Self-disclosure and authenticity; 2. Agreement; 3. Active participation, motivation and compliance; 4. Trust and secure attachment; and 5. Considering needs/abilities/wishes of the patient. Additionally, a narrative review of various approaches to conceptualize TA is presented and linked to respective corresponding instruments. Discussion The broad variety of conceptualizations and measures of TA makes coherent research on TA difficult. There are conceptual challenges such as the role of attachment style in TA that remain to be clarified. The current conceptualizations and measures do not incorporate the practical experience and expertise of psychotherapists and patients sufficiently. A metatheoretical conceptualization and measure of TA based on an empirical survey of psychotherapists and patients could address these issues.
Article
Full-text available
Introduction COVID-19 restrictions created barriers to “business as usual” in healthcare but also opened the door to innovation driven by necessity. This manuscript (1) describes how ADVANCE, an in-person group perpetrator program to reduce intimate partner violence (IPV) against female (ex)partners by men in substance use treatment, was adapted for digitally-supported delivery (ADVANCE-D), and (2) explores the feasibility and acceptability of delivering ADVANCE-D to men receiving substance use treatment. Methods Firstly, the person-based approach and mHealth development framework were used to iteratively adapt ADVANCE for digitally-supported delivery including conceptualization, formative research, and pre-testing. Then, a non-randomized feasibility study was conducted to assess male participants’ eligibility, recruitment, and attendance rates and uptake of support offered to their (ex)partners. Exploratory analyses on reductions in IPV perpetration (assessed using the Abusive Behavior Inventory; ABI) and victimization (using the revised ABI; ABI-R) at the end of the program were performed. Longitudinal qualitative interviews with participants, their (ex)partners, and staff provided an understanding of the program’s implementation, acceptability, and outcomes. Results The adapted ADVANCE-D program includes one goal-setting session, seven online groups, 12 self-directed website sessions, and 12 coaching calls. ADVANCE-D includes enhanced risk management and support for (ex)partners. Forty-five participants who had perpetrated IPV in the past 12 months were recruited, forty of whom were offered ADVANCE-D, attending 11.4 (SD 9.1) sessions on average. Twenty-one (ex)partners were recruited, 13 of whom accepted specialist support. Reductions in some IPV perpetration and victimization outcome measures were reported by the 25 participants and 11 (ex)partners interviewed pre and post-program, respectively. Twenty-two participants, 11 (ex)partners, 12 facilitators, and 7 integrated support service workers were interviewed at least once about their experiences of participation. Overall, the program content was well-received. Some participants and facilitators believed digital sessions offered increased accessibility. Conclusion The digitally-supported delivery of ADVANCE-D was feasible and acceptable. Remote delivery has applicability post-pandemic, providing greater flexibility and access. Given the small sample size and study design, we do not know if reductions in IPV were due to ADVANCE-D, time, participant factors, or chance. More research is needed before conclusions can be made about the efficacy of ADVANCE-D.
Article
Full-text available
The article traces the development of the concept of the therapeutic working alliance from its psychodynamic origins to current pantheoretical formulations. Research on the alliance is reviewed under four headings: the relation between a positive alliance and success in therapy, the path of the alliance over time, the examination of variables that predispose individuals to develop a strong alliance, and the exploration of the in-therapy factors that influence the development of a positive alliance. Important areas for further research are also noted.
Article
Full-text available
Background Therapeutic alliance has been found to play an influential role in predicting outcomes for adults and adolescents in psychotherapy. However, thus far, the information concerning the impact of therapeutic alliance on outcomes for adolescents in treatment for substance misuse has not yet been critically synthesised. Methods In accordance with PRISMA guidelines, the current review aimed to systematically collate published research investigating the association between alliance and outcomes for adolescents undergoing substance misuse treatment. Database searching produced 1083 records, with 16 studies meeting eligibility criteria. Results Twelve out of the 16 studies (75%) reported significant alliance–outcome relationships, whereby higher alliance ratings predicted better treatment outcomes, as well as improved engagement and retention in treatment. In addition, the review explored the conditions whereby alliances better predict outcomes, with reference to the alliance rater, the timing of the alliance rating and comorbid diagnoses. These results, however, largely remain inconclusive. Conclusions The evidence as it stands demonstrates the importance of the therapeutic alliance in predicting outcomes for adolescents in substance misuse treatments. The implications of the review's findings and recommendations for future research are discussed.
Article
Full-text available
Although the trend toward psychotherapy integration and eclecticism has gathered momentum, research on eclectic approaches to psychotherapy has lagged behind. To address this gap, the National Institute of Mental Health (NIMH) held a 2-day workshop in March 1986 on psychotherapy integration. Fourteen psychotherapy researchers were invited to consider the key issues associated with psychotherapy integration in order to advise the NIMH on guidelines for launching a program to stimulate relevant empirical research. Recommendations were developed to initiate such a research program that encompassed (a) conceptual clarification, (b) efficacy studies of systematic eclectic therapies, (c) the role of traditional psychotherapy process research, and (d) the training of integrative or eclectic therapists.
Article
Full-text available
The Working Alliance Inventory (WAI) was completed after the 1st psychotherapy session by 84 university counseling center clients and 15 therapists rating their work with 123 clients. The factor structure of these responses was examined using confirmatory factor analysis. A model with 1 general factor, a model with 3 specific factors, and a bilevel model of the factor structure were examined. The bilevel factor structure, with a General Alliance factor as its primary factor and 3 secondary specific factors, fit the data best. The items most indicative of the 3 specific factors were selected to form a 12-item short form of the WAI.
Article
Full-text available
Structural equation modeling (multivariate analysis with latent variables, also called causal modeling or covariance structure analysis) is a valuable methodological tool for use in counseling psychology research. Essentially the broad framework that subsumes many well-known procedures (e.g., multiple linear regression, factor analysis, path analysis), structural equation modeling allows for analysis of causal patterns among unobserved variables represented by multiple measures. It permits testing of causal hypotheses and theory, examination of psychometric adequacy, and enhancement of the explanatory power of correlational data that characterize counseling psychology research. I present and illustrate structural equation modeling, followed by a discussion of (a) issues and problems related to the use of this methodology, (b) possible applications of structural equation modeling to counseling psychology research, and (c) resources for those wanting further study. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Reviews definitions of the alliance (ALI) concept and proposes 4 relatively independent ALI dimensions that are complementary and compatible. These include (1) the therapeutic ALI, (2) the working ALI, (3) the therapist's empathic understanding and involvement, and (4) the patient–therapist agreement on the goals and tasks of treatment. There is evidence to support the predictive validity of the ALI in psychotherapy, with observations available across a variety of psychotherapy approaches, symptomatic disorders in outpatient samples, and sources of information (patient, therapist, clinical judges). A lack of association between ALI and outcome was, however, observed in some studies. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Present stages of development and preliminary validation of a self-report instrument for measuring the quality of alliance, the Working Alliance Inventory (WAI). The measure is based on Bordin's (1980) pantheoretical, tripartite (bonds, goals, and tasks) conceptualizaton of the alliance. Results from 3 studies were used to investigate the instrument's reliability and validity and the relations among the WAI scales. Data suggest that the WAI has adequate reliability. The instrument is reliably correlated with a variety of counselor and client self-reported outcome measures. Nontrivial relations were also observed between the WAI and other relationship indicators. Results are interpreted as preliminary support for the validity of the instrument. Although the results obtained in the reviewed studies are encouraging, the high correlations between the 3 subscales of the inventory bring into question the distinctness of the alliance components. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The California Therapeutic Alliance Rating System, CALTARS, was used by judges to rate four therapy sessions for each of 52 subjects treated in brief dynamic psychotherapy for pathological grief. A principal-components analysis provided support for the theoretically proposed domains of alliance. The interrater reliability at the treatment level was satisfactory for the five component-based scales. As hypothesized, scores reflecting positive contribution to the alliance were positively related to educational attainment, motivation for psychotherapy, pretreatment interpersonal functioning, patient experiencing in therapy, symptom improvement, and gains in interpersonal functioning but negatively associated with stressful life events and greater therapist emphasis on addressing patient resistances. As further hypothesized, scores reflecting negative contribution to the alliance were positively associated with greater therapist emphasis on addressing resistances. Alliance ratings were largely independent of pretreatment symptomatology. These findings build toward the construct validity of the CALTARS. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Describes a new instrument, the Inventory of Interpersonal Problems (IIP), which measures distress arising from interpersonal sources. The IIP meets the need for an easily administered self-report inventory that describes the types of interpersonal problems that people experience and the level of distress associated with them before, during, and after psychotherapy. In Study 1, psychometric data are presented for 103 patients who were tested at the beginning and end of a waiting period before they began brief dynamic psychotherapy. On both occasions, a factor analysis yielded the same six subscales; these scales showed high internal consistency and high test–retest reliability. Study 2 demonstrated the instrument's sensitivity to clinical change. In this study, a subset of patients was tested before, during, and after 20 sessions of psychotherapy. Their improvement on the IIP agreed well with all other measures of their improvement, including those generated by the therapist and by an independent evaluator. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
The Therapeutic Bond Scales assess the quality of the therapeutic relationship from the patient's perspective. The therapeutic bond is composed of 3 aspects: working alliance, empathic resonance, and mutual affirmation. Scales were developed to measure these aspects and the therapeutic bond as a whole. The correlations between these scales and 2 measures of outcome (session quality assessed by the patient and termination outcome evaluated by nonparticipant raters) were examined. All scales were significantly correlated with session quality. Therapeutic bond was significantly correlated with termination outcome in both a linear and a curvilinear fashion, suggesting that, at least in the initial phase of therapy, the therapeutic bond can be too high as well as too low. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
This two-part article suggests ingredients in the therapy relationship that are common to all interventions. It then examines similarities and differences in how the relationship works within the three dominant approaches to therapy. The overall aim of the article is to restimulate research and theory on the relationship. The first part defines the relationship and proposes three components to all therapeutic relationships: a working alliance, a transference configuration, and a real relationship. Five propositions are offered about the operation of each component within and across theoretical orientations. The second part examines how views of the relationship in perspectives broadly labeled psychoanalytic, humanistic, and learning vary according to three theoretical dimensions: the centrality, real-unreal, and means-end dimensions. Central research findings are reviewed for each theoretical perspective, the current state of research is examined for each, and suggestions are offered for future directions.
Article
Responds to C. J. Gelso and J. A. Carter's (see record 1986-09708-001) article on the client–therapist relationship in counseling and psychotherapy, particularly their suggestion that learning-oriented approaches (including cognitive and behavioral responses) have been notably silent on the importance of relationships in therapy. It is argued that cognitive-behavior therapists do not generally deny the existence or even potential importance of transference phenomena but that they would disagree on the automatic importance of the phenomena. (7 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)