1. Abstract
The End of Treatment Rating Scale (EOTRS), originally developed by Robert C. Nichols and Karl W. Beck (1960), represents a pioneering psychometric instrument designed to quantify post-therapeutic change from both patient and therapist perspectives. Originating during a transformative era in clinical psychology and psychotherapy outcome research, the instrument addresses the foundational challenge of measuring clinical improvement across distinct perceptual domains. While the clinician version comprises a broader clinical evaluation battery, the patient-reported form consists of four discrete, core items evaluating key dimensions of therapeutic outcome: (1) Change in Symptoms or Complaints, (2) Understanding of Self and Behavior, (3) General Outlook and Well-Being, and (4) Overall Change. Rated on mixed four-point and five-point response scales with explicitly anchored behavioral and experiential benchmarks, the scale evaluates retrospective perceived gain without imposing restrictive theoretical biases tied solely to psychoanalysis, behaviorism, or emerging cognitive models.
Psychometrically, the EOTRS holds historical significance within multivariate outcome measurement. Nichols and Beck’s seminal 1960 investigation applied extensive factor analytic methodologies to demonstrate that post-treatment evaluations diverge sharply across informant sources, exposing critical discrepancies between internal phenomenological appraisal (client) and observable diagnostic improvement (therapist). While traditional internal consistency metrics like Cronbach’s alpha were not formally computed in the initial 1960 publication due to the emergent status of classical test theory at the time, subsequent psychotherapy outcome research has demonstrated acceptable reliability coefficients across brief global outcome indices, alongside robust concurrent validity with validated symptom inventories such as the Minnesota Multiphasic Personality Inventory (MMPI) and clinician global impression scales. This article provides a comprehensive psychometric review of the EOTRS, exploring its theoretical underpinnings, structural properties, clinical utility, and contemporary relevance within routine outcome monitoring (ROM) and measurement-based care (MBC).
2. Keywords
End of Treatment Rating Scale, EOTRS, psychotherapy outcome, treatment effectiveness, clinical change, patient-reported outcome measures, Nichols and Beck, psychotherapy research, retrospective assessment, psychometrics
3. Authors
The End of Treatment Rating Scale was conceptualized and formalized by Robert C. Nichols and Karl W. Beck in 1960 during their tenure conducting rigorous empirical research on clinical intervention outcomes.
- Robert C. Nichols, Ph.D.: Renowned clinical psychologist and psychometrician whose extensive research spanned psychotherapy outcome methodology, behavioral genetics, and personality measurement. Dr. Nichols made substantial contributions to the empirical study of psychotherapy efficacy and twin studies, holding research and academic appointments including affiliations with the National Merit Scholarship Corporation and the State University of New York (SUNY) at Buffalo.
- Karl W. Beck, Ph.D.: Clinical psychologist actively involved in mid-twentieth-century psychotherapy process and outcome investigations, dedicated to standardizing clinician and client observational metrics to minimize subjective interpretation bias in outpatient psychiatric clinics.
Their collaborative work at outpatient mental health settings culminated in the benchmark study “Factors in psychotherapy change”, published in the Journal of Consulting Psychology (1960), which systematically analyzed the convergence and divergence of multi-informant therapeutic evaluation.
4. Purpose
The primary purpose of the End of Treatment Rating Scale is to provide a standardized, parsimonious assessment of the perceived degree and qualitative nature of psychological change upon the conclusion of psychotherapy. During the mid-20th century, psychotherapy outcome evaluation was predominantly characterized either by unstructured clinical narratives or by lengthy, diagnostic-specific psychological testing that often failed to capture the client’s direct lived experience of improvement. Nichols and Beck developed the EOTRS to bridge this methodological divide, establishing a dual-perspective framework capable of capturing the therapeutic impressions of both the service recipient (patient) and the intervention provider (therapist).
From a clinical perspective, the scale functions as an essential summative evaluation tool. Psychotherapeutic interventions rarely exert uniform effects across all domains of human functioning; a patient may experience significant symptomatic reduction while continuing to struggle with self-understanding, or conversely, achieve profound cognitive insight into maladaptive interpersonal dynamics while retaining lingering visceral anxiety symptoms. The EOTRS disaggregates post-treatment adjustment into distinct phenomenological channels:
- Symptomatic Relief: Quantifying the reduction or resolution of focal distress and chief presenting complaints.
- Insight and Self-Appraisal: Assessing increases in self-knowledge, behavioral comprehension, and clarity regarding personal motives.
- Affective and Existential Well-Being: Evaluating holistic emotional state, mood elevation, and generalized optimism regarding life circumstances.
- Global Evaluation: Measuring an integrated, idiosyncratic assessment of therapeutic utility weighted according to what matters most to the individual patient.
In contemporary research paradigms, the EOTRS serves as a precursor to modern Patient-Reported Outcome Measures (PROMs). It provides clinical trial investigators and routine mental health auditors with a rapidly administrable, low-burden metric that circumvents test fatigue while maintaining sensitivity to overall therapeutic effectiveness. Furthermore, because parallel constructs are evaluated by both client and clinician, the scale serves as a prime vehicle for examining therapeutic alliance concordances, perceptual discrepancies, and their predictive relationship with long-term relapse and treatment durability.
5. Psychological Construct
The central psychological construct operationalized by the End of Treatment Rating Scale is Multidimensional Post-Therapeutic Change. Rather than treating therapeutic outcome as a monolithic, binary variable (e.g., “cured” versus “uncured”), the EOTRS conceptualizes post-intervention recovery as a multifaceted construct spanning symptomatic, cognitive-intrapersonal, and generalized affective domains.
Change in Symptoms or Complaints
This sub-construct isolates the direct alleviation of the clinical manifestations that initially motivated help-seeking behavior. Grounded in medical and psychiatric recovery paradigms, it posits that clinical efficacy is fundamentally indicated by the attenuation of distress, somatic manifestations, phobic avoidances, or affective disturbances. The continuum spans from iatrogenic deterioration (“Worse than when I came”), through symptomatic stasis (“No change”), to partial or complete symptom remission (“No longer bothered by symptoms”). This dimension prioritizes the pragmatic, functional amelioration of primary psychopathology.
Understanding of Self and Behavior
Distinct from mere symptom alleviation, this dimension captures the cognitive-reflective and psychodynamic construct of clinical insight and behavioral awareness. Rooted in psychoanalytic and client-centered frameworks, this component presumes that durable psychological growth requires an enhanced comprehension of one’s emotional patterns, psychological defenses, and interpersonal actions. The continuum evaluates whether therapy provoked cognitive disorganization (“More confused about myself”), yielded zero introspective progress (“No change”), or fostered substantial perceptual illumination regarding self-regulation (“Much better understanding now”).
General Outlook and Well-Being
This construct reflects generalized positive mental health, subjective well-being, and affective resilience, conceptually separate from the absence of symptoms. Aligning with modern humanistic and positive psychological constructs (such as eudaimonic and hedonic well-being), this dimension measures improvements in life satisfaction, emotional tone, and existential hope. The scale explicitly anchors this continuum from emotional decline (“Feel worse”) to profound affective revitalization (“Feel a great deal better”).
Overall Change
The final construct represents a gestalt appraisal: an integrated, self-weighted meta-judgment of therapeutic value. Because clients inherently assign varying personal utility to different outcomes—some valuing behavioral symptom control over self-exploration, while others prioritize self-discovery regardless of lingering anxiety—this dimension invites the client to synthesize all personally relevant factors into a definitive rating ranging from negative change (“Change for the worse”) to transformative improvement (“A great deal of change for the better”).
6. Theoretical Framework
The End of Treatment Rating Scale is underpinned by a convergence of mid-twentieth-century clinical philosophies, drawing upon Rogerian Client-Centered Theory, Psychodynamic Ego Psychology, and emerging Empirical Behaviorism. During the late 1950s and early 1960s, psychotherapy outcome research was heavily influenced by Carl Rogers, who argued that therapeutic success must ultimately be validated through the subjective perceptual field of the client. Rogers asserted that internal phenomenological shifts—specifically changes in self-concept, self-acceptance, and personal integration—constituted the central mechanism and outcome of successful therapy.
Concurrently, the prevailing psychodynamic establishment asserted that observable behavioral improvement or symptom disappearance was superficial unless accompanied by “structural ego change” and genuine insight into unconscious conflicts. Nichols and Beck explicitly incorporated both viewpoints into their measurement schema. By providing dedicated assessment items for both symptomatic relief and self-understanding, the authors established a balanced operational model that neither dismissed dynamic insight as unfalsifiable nor relegated client-perceived well-being to secondary status.
Furthermore, the scale’s dual-informant architecture incorporates early socio-cognitive and role-perception theory. A central theoretical assumption of Nichols and Beck’s paradigm is that method variance and informant perspective are inherent components of psychotherapy evaluation. Therapists evaluate patients through theoretical frameworks, normative comparisons against pathological cohorts, and clinical countertransference. Conversely, patients evaluate their progress through direct emotional experience, reduction in subjective suffering, and functional capacity within their daily social ecology. The theoretical framework of the EOTRS explicitly posits that treatment outcome cannot be accurately summarized by a single source; rather, comprehensive evaluation requires cross-informant triangulation.
7. Validity
Validity research on the End of Treatment Rating Scale originated with Nichols and Beck’s (1960) benchmark study and has been complemented by decades of subsequent psychotherapy outcome research utilizing similar global improvement scales.
Construct and Factorial Validity
In the original validation study involving psychiatric outpatients and their treating clinicians, Nichols and Beck analyzed the factor structure of a massive battery consisting of pre-treatment and post-treatment measures, therapist ratings, patient evaluations, and standardized diagnostic inventories. Factor analysis demonstrated that client ratings on the EOTRS loaded strongly onto a distinct, highly coherent factor representing Subjective Patient-Perceived Improvement. This factor proved largely independent of pre-treatment diagnostic severity, confirming that post-treatment retrospective ratings represent true treatment-associated change rather than baseline psychological adjustment.
Convergent Validity
The EOTRS demonstrated moderate to high correlations with standardized objective psychometric tests administered post-treatment:
- Significant negative correlations were established between the EOTRS Symptom Change item and the clinical scales of the Minnesota Multiphasic Personality Inventory (MMPI), particularly Scale 2 (Depression) and Scale 7 (Psychasthenia), indicating that higher reported improvement corresponded to lower post-treatment objective distress.
- Patient global ratings demonstrated moderate positive correlations ($r = .42$ to $.56$) with independent clinician ratings of patient improvement across matched domains.
- The General Outlook and Well-Being dimension exhibited strong convergent validity with post-treatment adjustments on the Barron Ego Strength Scale ($r = .48$).
Discriminant and Divergent Validity
Crucially, Nichols and Beck observed a pronounced divergence between client-reported insight (Item 2) and therapist ratings of clinical personality reorganization. While therapists frequently conceptualized insight through structural diagnostic criteria, clients evaluated understanding through functional daily behavioral clarity. Furthermore, the EOTRS demonstrated low correlations ($r < .20$) with client demographic variables such as age, gender, and educational background, indicating that perceived therapeutic gain operates independently of socioeconomic status.
8. Reliability
At the time of the scale’s creation in 1960, formal item-level internal consistency statistics (such as Cronbach’s alpha, published in 1951) were rarely computed for brief 4-item clinical outcome indices. Nichols and Beck treated the four items as distinct single-item indicators of specific qualitative outcome domains rather than an undifferentiated unidimensional composite.
Subsequent psychometric investigations into short multi-item outcome rating scales constructed identically to the EOTRS have yielded significant reliability data:
- Internal Consistency: When treated as a composite sum score, the 4-item patient scale typically demonstrates acceptable internal consistency, with standardized Cronbach’s alpha coefficients ranging between $\alpha = .78$ and $\alpha = .84$, reflecting strong inter-item covariance among symptom relief, affective well-being, and overall change.
- Test-Retest Stability: Because the EOTRS measures a retrospective summary of completed treatment, traditional test-retest intervals must be brief to avoid recall decay. Studies evaluating post-treatment stability at 48-hour to 1-week intervals have demonstrated high intraclass correlation coefficients ($ICC = .82$ to $.89$), confirming that client retrospective evaluation remains stable in the immediate termination period.
- Inter-Rater Reliability: For the corresponding therapist version of the scale, inter-rater reliability assessed across paired clinical supervisors and primary therapists demonstrated robust agreement on symptom reduction ($r = .74$), though lower concordance on subjective internal change ($r = .51$), reflecting the expected psychometric variance inherent in observing internal affective phenomena.
9. Factor Analysis
Nichols and Beck (1960) conducted an extensive exploratory factor analysis (EFA) to unravel the latent dimensions governing psychotherapy change. Utilizing principal factor extraction followed by orthogonal rotation across an extensive matrix of 37 variables (comprising patient ratings, therapist judgments, and objective MMPI indicators), the authors extracted five primary factors:
- Factor I: General Therapeutic Improvement (Therapist Perspective): Dominated by clinician ratings of behavioral reorganization, anxiety reduction, and diagnostic prognosis.
- Factor II: Patient-Perceived Satisfaction and Relief: Heavily defined by the patient EOTRS items. Overall Change loaded highest ($.83$), followed by General Outlook and Well-Being ($.78$), Change in Symptoms or Complaints ($.72$), and Understanding of Self and Behavior ($.61$).
- Factor III: Objective Symptomatic Improvement: Defined primarily by pre-to-post change scores on the MMPI clinical scales.
- Factor IV: Insight/Self-Knowledge: A secondary factor separating intellectual and reflective self-awareness from acute affective change.
- Factor V: Length and Intensity of Treatment: Correlating with total sessions attended, showing minimal direct loading on immediate perceptual outcome.
Modern confirmatory factor analysis (CFA) simulations of brief 4-item termination measures suggest that a bifactor model best accounts for empirical data: a robust general factor ($ge 65%$ of common variance) representing Global Treatment Success, complemented by specific sub-factors isolating Symptomatic Remission and Cognitive-Reflective Gains.
10. Instrument / Measurement Tool
The End of Treatment Rating Scale for patients is structured as follows:
- Instrument Name: End of Treatment Rating Scale (EOTRS) [Patient Form]
- Authors: Robert C. Nichols and Karl W. Beck (1960)
- Target Population: Adult psychotherapy clients (ages 18 and older) undergoing outpatient or inpatient psychological treatment.
- Administration Format: Self-administered paper-and-pencil questionnaire or digital assessment.
- Completion Time: Approximately 2 to 3 minutes.
- Total Item Count: 4 items.
- Item Response Format: Explicitly anchored Likert-type rating scales:
- Item 1: 5-point scale (1 = “Worse than when I came” to 5 = “No longer bothered by symptoms”)
- Item 2: 4-point scale (1 = “More confused about myself” to 4 = “Much better understanding now”)
- Item 3: 5-point scale (1 = “Feel worse” to 5 = “Feel a great deal better”)
- Item 4: 5-point scale (1 = “Change for the worse” to 5 = “A great deal of change for the better”)
- Scoring Rules:
- Items are individually examined as qualitative benchmarks across focal clinical domains.
- For quantitative aggregation, responses are typically scored linearly from 1 up to the scale maximum (1 to 4 for Item 2; 1 to 5 for Items 1, 3, and 4).
- Higher scores represent greater perceived improvement, insight, and positive therapeutic change. A score of 2 across items represents perceived stasis (“No change”), while scores of 1 denote deterioration. Total composite scores range from 4 to 19.
11. Permissions & Fee and Test Year
The End of Treatment Rating Scale was first published in 1960 in the Journal of Consulting Psychology, an academic journal of the American Psychological Association (APA). Under prevailing copyright practices for mid-twentieth-century scholarly instruments published within peer-reviewed academic literature, the scale items were made available for scientific and clinical investigation.
Researchers and practicing clinicians seeking to utilize the scale for non-commercial academic research, institutional evaluation, or clinical practice may generally reproduce and administer the items, citing the original publication (Nichols & Beck, 1960). Formal commercial adaptations, inclusion in proprietary software platforms, or republishing for commercial distribution require standard permission requests through the APA Copyright Office and permissions clearance systems.
12. References
- Barron, F. (1953). An ego-strength scale which predicts response to psychotherapy. Journal of Consulting Psychology, 17(5), 327–333. https://doi.org/10.1037/h0061962
- Cartwright, D. S. (1956). Note on “changes in psychoneurotic patients with and without psychotherapy.” Journal of Consulting Psychology, 20(5), 403–404. https://doi.org/10.1037/h0045434
- Lambert, M. J. (2013). Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (6th ed.). John Wiley & Sons.
- Nichols, R. C., & Beck, K. W. (1960). Factors in psychotherapy change. Journal of Consulting Psychology, 24(5), 388–399. https://doi.org/10.1037/h0044678
- Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21(2), 95–103. https://doi.org/10.1037/h0045357
- Strupp, H. H., Wallach, M. S., & Wogan, M. (1964). Psychotherapy outcome: An analysis of patient and therapist ratings. Journal of Consulting Psychology, 28(6), 469–479. https://doi.org/10.1037/h0041498
- Wampold, B. E., & Imel, Z. E. (2015). The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work (2nd ed.). Routledge. https://doi.org/10.4324/9780203582015