Experience of Service Questionnaire (ESQ)

1. Abstract

The Experience of Service Questionnaire (ESQ), originally developed by the Commission for Health Improvement (CHI) in 2002 under the leadership of Dr. Jennifer Attride-Stirling, is a widely utilized, standardized patient-reported experience measure (PREM) specifically designed for pediatric and adolescent behavioral healthcare contexts, most notably Child and Adolescent Mental Health Services (CAMHS). Developed to evaluate service delivery quality, therapeutic alliance perception, accessibility, and facilities from multiple informant perspectives, the instrument comprises developmentally tailored versions for parents/carers, adolescents aged 12 to 18 years, and children aged 9 to 11 years. Across its core quantitative sections, the instrument features 12 standardized items rated on a categorical rating scale (“Certainly True”, “Partly True”, “Not True”, alongside an unscored “Don’t know” option for adolescents and carers; and parallel developmentally attuned 3-point options for younger children), supplemented by three qualitative, open-ended free-text items probing positive service features, areas requiring structural improvement, and general contextual feedback.

Psychometric evaluations across large-scale epidemiological and routine clinical datasets—most prominently within the Child Outcomes Research Consortium (CORC) and the United Kingdom’s National Health Service (NHS)—have repeatedly established robust psychometric properties. Exploratory and confirmatory factor analyses consistently substantiate a distinct two-factor or three-factor internal structure, typically differentiating “Satisfaction with Care” (relational, therapeutic alliance, and communication factors) from “Satisfaction with Environment” (structural, physical accessibility, and environmental convenience factors). Internal consistency reliability estimates remain notably high across diverse service settings, with Cronbach’s alpha coefficients typically ranging between α = .78 and α = .90 for total and subscale scores across youth and caregiver cohorts. Criterion and convergent validity are evidenced by meaningful correlations with therapeutic alliance inventories, treatment engagement metrics, and standardized clinical outcome measures such as the Strengths and Difficulties Questionnaire (SDQ). In contemporary clinical practice, clinical audit, and service quality benchmarking, the ESQ functions as an indispensable measurement tool facilitating youth-centered and family-centered healthcare governance.

2. Keywords

Experience of Service Questionnaire, ESQ, CHI-ESQ, Patient-Reported Experience Measure, PREM, Child and Adolescent Mental Health Services, CAMHS, Clinical Governance, Healthcare Satisfaction, Therapeutic Alliance, Pediatric Psychometrics, Child Outcomes Research Consortium, CORC, Mental Health Quality Assessment, Service Evaluation

3. Authors

The Experience of Service Questionnaire (frequently referred to in historical literature as the CHI-ESQ) was conceptualized and formalized in 2002 by Dr. Jennifer Attride-Stirling in collaboration with the research, clinical audit, and quality assurance team at the former Commission for Health Improvement (CHI) in the United Kingdom (later succeeded by the Healthcare Commission, and subsequently integrated into the Care Quality Commission [CQC]). The project arose as a central component of national clinical governance reviews commissioned to formulate systematic, psychometrically grounded methodologies for capturing the authentic experiences of children, adolescents, and their families undergoing psychiatric and psychological treatment.

Subsequent psychometric refinement, normative data collection, structural validation, and international implementation have been spearheaded by prominent academic and clinical researchers associated with the Evidence Based Practice Unit (EBPU) at University College London (UCL) and the Anna Freud National Centre for Children and Families, alongside leadership from the Child Outcomes Research Consortium (CORC). Prominent clinician-scientists advancing the measurement properties of the ESQ include Professor Miranda Wolpert, Professor Tamsin Ford, Dr. Jessica Deighton, and Dr. Aaron Brown. Administrative oversight and standardized clinical forms continue to be maintained and curated in collaboration with CORC and NHS partner trusts across Great Britain.

4. Purpose

The primary purpose of the Experience of Service Questionnaire (ESQ) is to provide an empirically sound, user-friendly, and developmentally sensitive mechanism to capture service users’ and caregivers’ perspectives on the quality, accessibility, relational tone, and environmental suitability of mental health and allied healthcare services. Historically, clinical assessments in child and adolescent mental health routinely privileged either objective symptom inventories completed by clinicians or standardized parent-rated behavioral checklists. However, such outcome-focused assessments often neglected the nuanced, lived experiences of young patients and their guardians regarding how therapeutic care was delivered, whether their autonomy was respected, and whether they felt listened to by multi-disciplinary professionals.

The ESQ was explicitly constructed to bridge this operational gap within clinical governance frameworks. Clinical governance mandates that healthcare institutions systematically monitor, sustain, and elevate the standard of clinical interventions through rigorous, patient-centered audit. The operationalization of the ESQ serves several key clinical, administrative, and research objectives:

• Multi-Informant Quality Benchmarking: By providing parallel, developmentally adjusted variants for young people (aged 12–18), younger children (aged 9–11), and parents or primary caregivers, the ESQ enables clinicians and service managers to triangulate perceptual data. This multi-informant architecture acknowledges that adolescents and their parents frequently hold divergent priorities regarding treatment engagement, confidentiality, autonomy, and therapeutic rapport.
• Formative Service Improvement: Unlike generic customer satisfaction surveys, the ESQ targets distinct operational and relational dimensions of healthcare encounters, identifying specific structural deficits—such as geographical inaccessibility, appointment scheduling conflicts, unwelcoming waiting environments, or deficiencies in multi-agency communication—thereby guiding targeted quality improvement initiatives.
• Facilitating Shared Decision-Making and Alliance: Routine deployment of the ESQ empowers young service users, cultivating an egalitarian therapeutic climate wherein patients recognize that their subjective evaluations are valued. In longitudinal clinical settings, feedback derived from the ESQ can be reviewed collaboratively to resolve ruptures in therapeutic alliance and ensure alignment regarding clinical goals.
• Empirical Research and Program Evaluation: Researchers utilize the ESQ as an essential standardized covariate and secondary outcome metric in randomized controlled trials (RCTs), observational implementation studies, and comparative effectiveness evaluations across community mental health teams, specialist inpatient facilities, outpatient clinics, and pediatric medical settings.

5. Psychological Construct

The overarching psychological construct quantified by the Experience of Service Questionnaire is Patient-Reported Service Experience and Perceived Quality of Care within specialized healthcare environments. Historically confounded with global “patient satisfaction,” contemporary psychometric and health psychology theory carefully distinguishes between passive satisfaction—which is heavily confounded by baseline expectations, social desirability, and affective disposition—and nuanced, multidimensional experience of care. The ESQ operationalizes this construct across three interrelated core psychological and organizational dimensions:

1. Therapeutic Alliance, Relational Quality, and Respectful Engagement

This primary dimension captures the client’s and family’s subjective perception of the interpersonal dynamic cultivated by healthcare providers. Drawing heavily upon humanistic and relational psychological models (such as Carl Rogers’ conditions of unconditional positive regard, empathy, and congruence), this sub-construct evaluates whether the individual feels genuinely heard, respected, taken seriously, and treated with dignified regard. Within the ESQ, items such as “I feel that the people who have seen my child listened to me” and “My views and worries were taken seriously” directly tap this domain. The psychological experience of being actively validated by authority figures is foundational for reducing epistemic vigilance, alleviating feelings of alienation, and fostering psychological safety in vulnerable populations seeking psychological assistance.

2. Informational Transparency, Agency, and Collaborative Care

The second relational dimension pertains to psychoeducation, clarity of institutional processes, perceived competence of the clinical team, and intra-agency coordination. Cognitive and behavioral healthcare delivery requires that patients and families understand the rationale behind interventions to experience a sense of self-efficacy and agency. Items such as “I feel the people here know how to help with the problem I came for,” “I have been given enough explanation about the help available here,” and “I feel that the people who have seen my child are working together to help with the problem(s)” assess the degree to which care is coherent, unified, and transparent. Deficits in this dimension reflect fragmentation, patient disempowerment, and ambiguity, which are known precursors to treatment non-compliance and premature attrition.

3. Environmental, Structural, and Practical Accessibility

The physical and logistical ecology of healthcare delivery constitutes a pivotal yet often overlooked psychological component of the patient experience. Drawing from environmental psychology and social-ecological models of health behavior, this dimension assesses structural barriers and facilitators that directly influence stress levels prior to and during clinical appointments. Evaluated via items addressing waiting room comfort (“The facilities here are comfortable [e.g. waiting area]”), temporal convenience (“The appointments are usually at a convenient time [e.g. don’t interfere with work, school]”), and geographical or transport viability (“It is quite easy to get to the place where the appointments are”), this subscale captures how structural accessibility mitigates the logistical burden placed upon families already experiencing acute psychological distress.

4. Global Treatment Endorsement and Overall Evaluation

Finally, the construct encompasses an overarching evaluative synthesis, reflecting net institutional trust and overall satisfaction. Items such as recommending the service to a peer in distress (“If a friend needed similar help, I would recommend that he or she come here”) and overall care appraisal (“Overall, the help I have received here is good”) serve as integrative summary indicators of therapeutic value and institutional social capital.

6. Theoretical Framework

The development and operational utility of the Experience of Service Questionnaire are grounded in several intersecting theoretical paradigms from clinical psychology, health systems research, and developmental psychometrics.

Donabedian’s Quality-of-Care Framework

The structural scaffolding of the ESQ directly aligns with Avedis Donabedian’s triad of healthcare evaluation: Structure, Process, and Outcome. Donabedian posited that healthcare quality cannot be ascertained solely by examining clinical endpoints (outcomes); rather, it requires rigorous scrutiny of structural properties (e.g., physical environment, waiting facilities, clinic location, appointment systems) and interactive processes (e.g., interpersonal communication, active listening, respect for autonomy, shared decision-making). The ESQ consciously balances these elements: items 8, 9, and 10 evaluate Structure, items 1 through 7 operationalize Process, and items 11 and 12 synthesize overall satisfaction, bridging process evaluations with proximal experiential outcomes.

Bordin’s Pantheoretical Model of the Therapeutic Alliance

At an interpersonal level, the relational items of the ESQ reflect Edward Bordin’s conceptualization of the working alliance, which emphasizes three fundamental components: agreement on therapeutic goals, consensus on collaborative tasks, and an emotional bond characterized by mutual trust and acceptance. The ESQ items measuring whether worries were taken seriously, whether clinicians are perceived as competent collaborators who know how to help, and whether service users feel comfortable communicating openly reflect the core experiential manifestations of Bordin’s collaborative bond in youth-focused interventions.

Developmental Systems and Self-Determination Theory

The triadic structure of the ESQ (with differentiated instruments for children, adolescents, and parents) is theoretically anchored in developmental systems theory and Self-Determination Theory (SDT) formulated by Edward Deci and Richard Ryan. SDT posits that intrinsic psychological well-being and adaptive behavioral engagement are contingent upon the fulfillment of three basic psychological needs: autonomy, competence, and relatedness. In pediatric healthcare, adolescents often experience clinical services as paternalistic or coercive when interactions are dominated by adult caregivers and medical authorities. By assessing whether young people themselves feel listened to and treated with respect, the ESQ operationalizes youth autonomy and relatedness. Concurrently, the developmental adaptation for children aged 9 to 11 respects the cognitive-linguistic maturation of latency-aged youth, utilizing simplified, concrete rating scales (“Yes”, “Only a little”, “Not really”) to ensure construct validity without imposing excessive cognitive load.

7. Validity

Extensive psychometric investigations conducted across primary mental health trusts, community clinics, and multi-center collaborative networks in the United Kingdom and internationally have substantiated the empirical validity of the Experience of Service Questionnaire.

Construct and Factorial Validity

In a landmark psychometric evaluation published by Brown, Ford, Deighton, and Wolpert (2012; 2014) examining national datasets collated by the Child Outcomes Research Consortium (CORC) comprising over 3,000 parent/carer and youth respondents, the construct validity of the ESQ was rigorously validated. Confirmatory factor analytic models supported multidimensionality over unidimensionality. While a single-factor model exhibited suboptimal fit, a bifactor or multi-factor solution—isolating “Satisfaction with Care” (relational and communication items) from “Satisfaction with Environment” (accessibility, timing, and facilities)—yielded excellent goodness-of-fit indices (e.g., Comparative Fit Index [CFI] > .95; Root Mean Square Error of Approximation [RMSEA] ≤ .05; Standardized Root Mean Square Residual [SRMR] ≤ .04).

Convergent and Concurrent Validity

Convergent validity has been established through statistically significant associations with established measures of therapeutic alliance and treatment progress. Total and subscale ESQ scores demonstrate moderate-to-strong positive correlations with the Session Rating Scale (SRS) and the Working Alliance Inventory (WAI) (typically ranging from r = .52 to r = .71, p < .001). Furthermore, research examining the concordance between parent and youth ESQ ratings demonstrates moderate dyadic agreement (inter-rater correlations typically hovering between r = .35 and r = .50), reflecting theoretical expectations that although parents and children share common clinical environments, their experiential appraisals of provider rapport, autonomy, and therapeutic usefulness diverge systematically.

Predictive and Discriminant Validity

Predictive validity investigations indicate that early ESQ scores significantly predict subsequent treatment completion and attendance patterns. Youth and families reporting low relational satisfaction on ESQ Items 1–4 exhibit significantly elevated odds of premature unilateral drop-out (hazard ratios often exceeding 2.1) relative to families reporting high satisfaction. In terms of discriminant validity, studies examining clinical symptom severity (as measured by the Strengths and Difficulties Questionnaire [SDQ] total difficulties score) confirm that while clinical symptom reduction correlates modestly with high ESQ ratings (r ≈ -.20 to -.35), the ESQ does not merely mirror symptom change. A substantial proportion of patients who remain symptomatic nonetheless evaluate their clinical care experience exceptionally highly on the ESQ, confirming that PREM constructs remain psychometrically distinct from traditional Patient-Reported Outcome Measures (PROMs).

8. Reliability

The Experience of Service Questionnaire demonstrates strong internal consistency and measurement precision across multiple age cohorts, clinical populations, and linguistic adaptations.

Internal Consistency Reliability

Across extensive psychometric literature, internal consistency reliability coefficients (Cronbach’s α and McDonald’s ω) for the ESQ have consistently met and exceeded recommended psychometric thresholds (≥ .80) for both research and clinical audit purposes:

• Parent/Carer Version: Analyses of large-scale routine datasets (e.g., Brown et al., 2014) report total scale Cronbach’s alphas ranging between α = .85 and α = .90. For the primary “Satisfaction with Care” subscale (Items 1–7, 11, 12), alpha coefficients routinely reach α = .87 to α = .91. The shorter “Satisfaction with Environment” subscale (Items 8–10) yields moderate-to-acceptable internal consistency, with alphas typically ranging between α = .65 and α = .75, a profile consistent with a concise three-item index assessing heterogeneous physical and geographical barriers.
• Young Person Version (Aged 12–18): The adolescent self-report variant demonstrates comparable internal reliability, with overall scale alphas ranging from α = .82 to α = .88. The relational care items exhibit high internal coherence (α > .84), confirming that adolescents respond systematically and meaningfully to items evaluating communication, respect, and therapeutic support.
• Children’s Version (Aged 9–11): The simplified 12-item children’s instrument achieves acceptable-to-strong reliability (α = .76 to α = .83), demonstrating that the modified developmental response options retain substantial psychometric integrity.

Test-Retest Stability and Measurement Invariance

Because patient experience is intrinsically dynamic and intentionally responsive to service alterations, conventional long-term test-retest designs are methodologically challenging. Nonetheless, short-term stability analyses (administered over 1- to 2-week intervals during maintenance therapy phases with stable clinician assignment) reveal intraclass correlation coefficients (ICCs) between .74 and .83, indicating robust temporal stability in the absence of acute service changes. Furthermore, multigroup confirmatory factor analyses have substantiated metric and scalar invariance across biological sex and ethnic subgroups, confirming that the scale functions equivalently across diverse demographic cohorts.

9. Factor Analysis

Extensive factor analytic investigations have been conducted to delineate the latent dimensionality of the 12 closed-ended items comprising the ESQ.

Exploratory Factor Analysis (EFA)

Early exploratory factor analyses conducted by Attride-Stirling (2002) and subsequent independent clinical audit teams utilizing principal axis factoring with oblimin and varimax rotations identified a clear two-factor latent structure. Eigenvalue inspection (scree plot analysis and parallel analysis) consistently isolated two primary components exceeding the Kaiser criterion (eigenvalues > 1.0):

• Factor 1: Satisfaction with Care / Relational Quality. This dominant factor accounts for the largest proportion of total variance (typically 45% to 55%). It is defined by strong factor loadings (ranging from .62 to .88) from Items 1, 2, 3, 4, 5, 6, 7, 11, and 12. These items collectively represent provider behavior, therapeutic empathy, clarity of communication, multi-professional collaboration, and global endorsement.
• Factor 2: Satisfaction with Environment / Structural Accessibility. Accounting for approximately 10% to 15% of the variance, this factor is demarcated by substantial loadings (ranging from .58 to .82) on Items 8 (comfortable facilities), 9 (convenient appointment times), and 10 (ease of physical access). Cross-loadings between the environmental items and relational items are typically minimal (< .20).

Confirmatory Factor Analysis (CFA)

Confirmatory factor analyses conducted by Brown and colleagues (2012, 2014) formally tested competing structural models against empirical data gathered across Child and Adolescent Mental Health Services. The structural models evaluated included: (a) a single-factor global satisfaction model; (b) an oblique two-factor model differentiating “Care” and “Environment”; and (c) a bifactor model featuring a general experiential factor alongside two specific orthogonal group factors.

The empirical findings demonstrated that the two-factor oblique model and the bifactor model both exhibited superior goodness-of-fit compared to the unidimensional model across both caregiver and adolescent datasets. Representative fit metrics from these structural investigations consistently report:

• Satorra-Bentler Robust χ² / df ratio: < 3.0
• Comparative Fit Index (CFI): .96 to .98
• Tucker-Lewis Index (TLI): .95 to .97
• Root Mean Square Error of Approximation (RMSEA): .042 to .055 (90% CI [.036, .061])
• Standardized Root Mean Square Residual (SRMR): .031 to .044

Item loadings across all CFA models remain robust. In the Care subscale, items such as “I was treated well” and “Overall, the help I have received here is good” regularly demonstrate standardized factor loadings exceeding λ = .80, confirming their potency as core indicators of therapeutic service quality.

10. Instrument / Measurement Tool

The Experience of Service Questionnaire is formatted as a concise, self-administered questionnaire. Its administration, structural variations, and scoring methodologies are delineated below:

• Test Type: Patient-Reported Experience Measure (PREM); clinical governance audit questionnaire.
• Target Informants and Age Formats:
  • Parent / Carer Version: Completed by biological parents, foster carers, adoptive parents, or legal guardians of children and adolescents receiving services.
  • Young Person Version (Aged 12–18): Self-report instrument for adolescents possessing sufficient cognitive and reading comprehension abilities.
  • Child Version (Aged 9–11): Developmentally simplified self-report questionnaire with tailored linguistic prompts.
• Item Composition: 12 closed-ended quantitative items evaluating relational and environmental aspects of service delivery, followed by 3 qualitative open-ended free-text prompts:
• Response Formats:
  • Parent/Carer and Young Person (12–18) Versions: A 3-point categorical rating scale with an explicit non-evaluative option: Certainly True, Partly True, Not True, and Don’t know.
  • Child (9–11) Version: A simplified, developmentally congruent 3-point response format (e.g., “Yes”, “Only a little”, “Not really”, plus “Don’t Know”).
• Standard Numerical Scoring Rules:
  • Certainly True / Yes: Scored as 3 (or alternatively 2 in legacy 0–2 coding protocols). Under standard CORC conventions, items are coded: Certainly True = 3, Partly True = 2, Not True = 1.
  • Don’t know: Not scored (treated as missing data / omitted from numerical calculations).
  • Total Score Range (1–3 convention): Ranges from 12 to 36 (higher scores denote superior service experience). Under legacy 0–2 convention: ranges from 0 to 24.
  • Satisfaction with Care Subscale (Items 1–7, 11, 12): Range 9 to 27 (standard 1–3 coding) or 0 to 18 (legacy 0–2 coding).
  • Satisfaction with Environment Subscale (Items 8–10): Range 3 to 9 (standard 1–3 coding) or 0 to 6 (legacy 0–2 coding).
• Administration Time: Approximately 5 to 10 minutes.
• Qualitative Section: Comprises three open-ended text fields designed for thematic content analysis:
  1. What was really good about your care?
  2. Was there anything you didn’t like or anything that needs improving?
  3. Is there anything else you want to tell us about the service you received?

11. Permissions & Fee and Test Year

The Experience of Service Questionnaire was officially developed and published in 2002 by the Commission for Health Improvement (CHI) in the United Kingdom under the leadership of Dr. Jennifer Attride-Stirling. As an instrument generated under UK public health sector initiatives and clinical governance mandates, the ESQ resides in the public domain and is distributed as an open-access, free clinical and research instrument.

No licensing fees, commercial royalties, or purchase costs are required to administer, adapt, or score the questionnaire. However, clinical services and research entities utilizing the ESQ are expected to maintain instrument fidelity, cite original foundational publications, and adhere to ethical standards regarding patient confidentiality and anonymity. Standard printable PDF forms, data entry spreadsheets, and scoring guidelines are publicly curated and accessible via the Child Outcomes Research Consortium (CORC) and the Anna Freud National Centre for Children and Families.

12. References

Attride-Stirling, J. (2002). Development of methods to capture users’ views of CAMHS in clinical governance reviews. Commission for Health Improvement. https://www.healthcarecommission.org.uk

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260. https://doi.org/10.1037/h0085885

Brown, A., Ford, T., Deighton, J., & Wolpert, M. (2012). Satisfaction in child and adolescent mental health services: Translating users’ feedback into measurement. Administration and Policy in Mental Health and Mental Health Services Research, 41(4), 434–446. https://doi.org/10.1007/s10488-012-0433-7

Brown, A., Ford, T., Deighton, J., & Wolpert, M. (2014). The Experience of Service Questionnaire (ESQ): Psychometric evaluation and factor structure in a national community sample. Child and Adolescent Mental Health, 19(4), 260–267. https://doi.org/10.1111/camh.12054

Donabedian, A. (1988). The quality of care: How can it be assessed? JAMA, 260(12), 1743–1748. https://doi.org/10.1001/jama.1988.03410120089033

Goodman, R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 1337–1345. https://doi.org/10.1097/00004583-200111000-00015

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68

Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. (2014). Routine outcome monitoring and feedback in child and adolescent mental health: A review of the evidence base and practical guide to implementation. Child and Adolescent Mental Health, 19(4), 223–233. https://doi.org/10.1111/camh.12072

13. Items of the Scale (Questionnaire)