Behavior Intervention Rating Scale

1. Abstract

The Behavior Intervention Rating Scale (BIRS) is an established 24-item psychometric instrument engineered to evaluate educators’ and educational practitioners’ perceptions of social validity, specifically operationalized through treatment acceptability and perceived treatment effectiveness. Developed by Stephen N. Elliott and M. V. B. Treuting in 1991, the BIRS was constructed by substantially expanding the foundational 15-item Intervention Rating Profile (IRP-15) (Martens et al., 1985) through the addition of nine novel items designed to capture dimensions of perceived intervention efficacy, latency of change, and ecological maintenance. The instrument yields a global social validity score alongside three distinct, psychometrically confirmed dimensions: Treatment Acceptability (15 items), Treatment Effectiveness (7 items), and Time to Effect (2 items). Items are completed using a 6-point Likert-type rating format ranging from 1 (Strongly Disagree) to 6 (Strongly Agree), with higher aggregate scores reflecting superior acceptability, projected efficacy, and operational feasibility.

Psychometric evaluation of the normative sample of 216 regular and special education teachers demonstrated exceptional internal consistency, characterized by a total scale Cronbach’s alpha of α = 0.97, and robust subscale reliabilities of α = 0.97 for Acceptability, α = 0.92 for Effectiveness, and α = 0.87 for Time. Exploratory factor analysis via oblique rotation revealed a clear three-factor latent structure that accounts for 73.6% of the cumulative variance. Content, construct, and concurrent validity have been empirically validated through extensive correlational paradigms, including convergence with the Evaluative dimension of the Semantic Differential (Osgood et al., 1957). The BIRS remains a gold-standard instrument in school psychology, applied behavior analysis (ABA), behavioral consultation, and multi-tiered systems of support (MTSS/RTI) for predicting implementation fidelity and overcoming teacher resistance to classroom interventions.

2. Keywords

Behavior Intervention Rating Scale, BIRS, social validity, treatment acceptability, intervention effectiveness, school psychology, behavioral consultation, applied behavior analysis, classroom management, implementation fidelity, Intervention Rating Profile, MTSS.

3. Authors

The Behavior Intervention Rating Scale was conceived, psychometrically validated, and published by Stephen N. Elliott, Ph.D., and M. V. B. Treuting, Ph.D.

• Stephen N. Elliott, Ph.D.: An internationally recognized scholar in school psychology, educational assessment, and social competence measurement. Dr. Elliott served as a distinguished faculty member at the University of Wisconsin–Madison, Peabody College of Vanderbilt University, and Arizona State University (where he was the Mickelson Foundation Endowed Chair and Founding Director of the Learning Sciences Institute). He has authored landmark assessment tools, including the Social Skills Improvement System (SSIS / SSRS) and key works on inclusive educational assessment, academic competence, and behavioral consultation.
• M. V. B. Treuting, Ph.D.: An educational and clinical researcher whose foundational psychometric work on the BIRS was conducted in collaboration with Louisiana State University and partner graduate education programs, focusing on teacher attitudes, behavioral interventions, and clinical acceptability metrics.

4. Purpose

The primary purpose of the Behavior Intervention Rating Scale (BIRS) is to quantify educational and clinical practitioners’ subjective perceptions regarding the appropriateness, fairness, and projected clinical utility of behavioral support protocols. Within applied educational, behavioral, and clinical settings, the empirical efficacy of a psychosocial or behavioral intervention does not guarantee its adoption or sustained implementation. Interventions identified as empirically supported through randomized controlled trials frequently fail in authentic classroom environments due to practitioner resistance, perceived procedural burden, ethical discomfort, or skepticism concerning real-world effectiveness. The BIRS addresses this implementation gap by providing a psychometrically sound, standardized metric that measures whether an intervention is perceived as socially valid, ecologically viable, and professionally acceptable prior to, during, or following its execution.

Theoretical and Practical Rationale

Historically, early behavioral consultation paradigms presumed that demonstrating functional control over a target behavior was sufficient to establish intervention value. However, the emergence of the implementation science literature and pioneering work in social validity demonstrated that practitioners who view an intervention negatively exhibit poor treatment integrity (implementation fidelity), introduce unauthorized procedural adaptations, or abandon the intervention entirely. The BIRS was engineered to bridge the conceptual gap between acceptability (whether an intervention is liked, viewed as fair, and free from intrusive side effects) and effectiveness (whether an intervention will produce meaningful, rapid, and generalized behavioral change).

Clinical and Research Applications

In contemporary clinical practice and empirical research, the BIRS is deployed across multiple operational domains:

• Pretreatment Assessment and Social Validity Screening: School psychologists, Board Certified Behavior Analysts (BCBAs), and clinical consultants administer the BIRS to educators or multidisciplinary teams prior to intervention rollout. Interventions receiving low baseline scores can be modified or paired with collaborative coaching to avert treatment non-adherence.
• Comparative Intervention Research: Researchers use the BIRS to assess consumer preference between differential intervention typologies, such as comparing antecedent-based modifications against consequence-based strategies, differential reinforcement versus extinction, or digital behavioral monitoring tools versus manual tallying systems.
• Multi-Tiered Systems of Support (MTSS) and Positive Behavioral Interventions and Supports (PBIS): Within Tier 2 and Tier 3 behavioral support frameworks, the BIRS serves as an evaluation metric for Tier 2 target interventions (e.g., Check-In/Check-Out) and individualized Tier 3 Behavior Intervention Plans (BIPs), ensuring that multidisciplinary teams construct interventions that general education teachers are willing to execute faithfully within standard classroom workflows.
• Conjoint Behavioral Consultation (CBC): The BIRS facilitates shared decision-making between educators, parents, and consulting specialists, identifying specific operational discrepancies in how interventions are perceived across the home and school contexts.

5. Psychological Construct

The Behavior Intervention Rating Scale measures the overarching psychological construct of Social Validity, with an emphasis on the operational interface between intervention acceptability, perceived clinical efficacy, and temporal efficiency. In psychometric literature, social validity reflects the degree to which consumers value the focus, procedures, and outcomes of applied interventions. The BIRS delineates this construct into three interrelated, statistically validated dimensions:

1. Treatment Acceptability (15 Items)

Treatment acceptability refers to the subjective evaluation of an intervention’s procedural appropriateness, fairness, ethical alignment, and invasiveness. This dimension assesses whether an intervention fits the moral and professional norms of the teacher and the ecological demands of the classroom. Key subcomponents measured include:

• Ecological Congruence and Feasibility: Whether the intervention aligns with established classroom procedures and routines without excessively burdening instructional time (e.g., Item 7: “I would be willing to use this in the classroom setting”; Item 10: “The intervention is consistent with those I have used in classroom settings”).
• Procedural Fairness and Ethical Benevolence: The teacher’s moral appraisal of the technique, ensuring it does not infringe upon the child’s dignity or well-being (e.g., Item 11: “The intervention was a fair way to handle the child’s problem behavior”; Item 12: “The intervention is reasonable for the behavior problem described”).
• Risk of Adverse Effects: Confidence that the targeted protocol will not produce counterproductive psychological, social, or emotional fallout for the learner (e.g., Item 8: “The intervention would not result in negative side effects for the child”).
• Severity-Proportionality Alignment: The cognitive congruence between the intensity of the behavioral disruption and the intrusiveness of the proposed solution (e.g., Item 5: “The child’s behavior problem is severe enough to warrant use of this intervention”).

2. Treatment Effectiveness (7 Items)

The Treatment Effectiveness dimension assesses the evaluator’s expectation that the intervention will produce clinically meaningful, ecologically durable, and generalized behavioral amelioration. Historically, early acceptability scales failed to assess whether teachers believed a procedure would actually solve the problem. The BIRS addresses this limitation by measuring:

• Long-term Behavioral Maintenance: The expectation that behavioral improvements will persist long after formal reinforcement contingencies, token economies, or behavioral cues are phased out (e.g., Item 17: “The intervention would produce a lasting improvement in the child’s behavior”; Item 20: “The child’s behavior will remain at an improved level even after the intervention is discontinued”).
• Ecological and Cross-Setting Generalization: The belief that behavioral gains will not remain isolated to the target classroom, but will generalize across diverse environments, including alternate classrooms, lunchrooms, and home settings (e.g., Item 21: “Using the intervention should not only improve the child’s behavior in the classroom, but also in other settings”).
• Social Comparison and Normalization: The degree to which the intervention reduces behavioral deviance to normative classroom levels, allowing the child’s behavioral repertoire to converge with that of well-behaved peers (e.g., Item 18: “The intervention would improve the child’s behavior to the point that it would not noticeably deviate from other classmates’ behavior”; Item 22: “When comparing this child with a well-behaved peer before and after use of the intervention, the child’s and the peer’s behavior would be more alike after using the intervention”).
• Collateral Improvement: The systemic expectation that remediating the focal problem will yield positive spillover effects into adjacent behavioral, academic, or social domains (e.g., Item 24: “Other behaviors related to the problem behavior also are likely to be improved by the intervention”).

3. Time to Effect (2 Items)

The third dimension, Time to Effect (often referred to in the literature as Latency of Change or Timeliness), measures the respondent’s expectations regarding the speed with which the intervention produces observable reductions in challenging behavior. Practitioners facing severe classroom disruption operate under intense pressure; interventions perceived as requiring extended delays before showing tangible results are susceptible to premature discontinuation. This subscale specifically isolates:

• Immediacy of Behavior Reduction: The perception that noticeable relief and behavior suppression will occur shortly after implementation begins (Item 16: “The intervention would quickly improve the child’s behavior”).
• Observational Latency: The practitioner’s confidence that positive changes will be readily detectable in real time during the initial intervention window (Item 19: “Soon after using the intervention, the teacher would notice a positive change in the problem behavior”).

6. Theoretical Framework

The development and structural organization of the BIRS are grounded in three interrelated theoretical models: Montrose Wolf’s tripartite model of social validity, Kazdin’s paradigm of treatment acceptability, and the Witt-Elliott Behavioral Intervention Model.

Wolf’s Tripartite Social Validity Paradigm

In his seminal paper, Montrose Wolf (1978) proposed that applied behavior analysts must evaluate their work along three subjective social dimensions: (a) the social significance of the goals (what do we really want to change?), (b) the social appropriateness of the procedures (are the interventions acceptable, or do they produce ethical unease?), and (c) the social importance of the effects (do the changes make a real, meaningful difference in the consumer’s daily life?). The BIRS directly operationalizes Wolf’s second and third dimensions: the Acceptability subscale captures the social appropriateness of the procedures, while the Effectiveness and Time subscales assess the perceived social importance and timeliness of the clinical outcomes.

Kazdin’s Treatment Acceptability Model

Alan E. Kazdin (1980, 1981) refined social validity by introducing standardized, analogue measurement of treatment acceptability in clinical psychology. Kazdin conceptualized acceptability as judgments by laypersons, clients, or clinical intermediaries that proposed treatment procedures are appropriate, fair, non-intrusive, and consistent with conventional notions of what treatment should be. Kazdin established that when interventions are perceived as low in acceptability, treatment adherence drops, client attrition increases, and ethical controversies emerge. The BIRS expands Kazdin’s experimental framework into educational consultation, establishing a direct empirical continuum between procedural acceptability and clinical compliance.

The Witt-Elliott Behavioral Intervention Model

The direct operational antecedent to the BIRS is the integrative conceptual model articulated by Joseph C. Witt and Stephen N. Elliott (1985). This model conceptualizes an interdependent, four-stage systemic cycle governing classroom intervention success:

1. Intervention Acceptability: Influenced by intervention type (e.g., positive reinforcement vs. reductive punishment), time required, teacher experience, and severity of student problem behavior.
2. Treatment Integrity: The fidelity and accuracy with which the teacher executes the intervention steps as originally designed.
3. Treatment Effectiveness: The observable empirical outcome and magnitude of behavioral change achieved in the classroom.
4. Ecological Impact: Reciprocal feedback loops where realized effectiveness retroactively alters future acceptability judgments and strengthens practitioner competence.

The BIRS was explicitly designed to assess the reciprocal mechanisms within this model. Elliott and Treuting (1991) hypothesized that acceptability and perceived effectiveness are not orthogonal constructs, but functionally interdependent variables that jointly dictate whether a teacher will implement a behavioral intervention with sufficient integrity to produce meaningful student improvement.

7. Validity

The validity of the BIRS has been established through multi-method psychometric investigations examining content, construct, and concurrent validity.

Content Validity

Content validity was established through systematic operationalization of the constructs of treatment acceptability, perceived efficacy, and intervention feasibility. The original 15 items of the Intervention Rating Profile (IRP-15; Martens et al., 1985) were retained due to their established capacity to capture procedural fairness, risk of side effects, and implementation willingness. To establish comprehensive coverage of the intervention outcome domain, Elliott and Treuting formulated nine additional items based on clinical behavior analysis literature, targeting social comparison, clinical significance, maintenance, generalization, and latency to effect. Expert review confirmed that these 24 items collectively span the full spectrum of educational social validity.

Construct Validity

Construct validity was demonstrated through exploratory and confirmatory factor analyses, as well as experimental analogue studies manipulating intervention parameters. The primary construct validity investigation conducted by Elliott and Treuting (1991) subjected the 24 items to factor analysis, which confirmed three distinct factors: Acceptability, Effectiveness, and Time. The factor structure accounted for 73.6% of the variance, confirming that acceptability and perceived effectiveness represent distinct yet related dimensions of social validity. Construct validity was further corroborated by demonstrating that BIRS scores systematically differentiate between intervention types: positive, reinforcement-based strategies reliably yield higher BIRS scores than consequence-driven or punitive strategies (e.g., response cost, isolation timeout), matching established theoretical predictions.

Concurrent Validity

Concurrent validity was established through correlational comparisons with the Semantic Differential (SD) developed by Osgood, Suci, and Tannenbaum (1957). The Semantic Differential measures the affective meaning of concepts across three fundamental dimensions: Evaluation, Potency, and Activity. In the validation cohort of 216 educators:

• The BIRS Total Score correlated strongly with the Evaluative scale of the Semantic Differential (r = 0.76, p < .001), indicating that interventions rated highly on the BIRS are viewed as intrinsically positive, valuable, and beneficial.
• The BIRS Acceptability factor correlated at r = 0.74 (p < .001) with the Evaluative dimension.
• Correlations between the BIRS factors and the Potency and Activity dimensions of the Semantic Differential were moderate to low (ranging from r = 0.28 to 0.42), demonstrating adequate divergent and discriminant validity from general perceived intervention intensity or activity level.

Predictive and Social Criterion Validity

Subsequent educational research has substantiated the predictive validity of the BIRS regarding actual classroom outcomes. Interventions rated by teachers with total BIRS scores exceeding 4.5 on the 6-point scale demonstrate statistically significant increases in implementation fidelity (measured via direct observation protocols) compared to interventions scoring below 3.5. Furthermore, BIRS scores predict intervention sustainability, defined as whether teachers continue utilizing behavioral strategies following the withdrawal of external consultative support.

8. Reliability

The BIRS demonstrates high reliability across diverse samples of general educators, special educators, and educational consultants.

Internal Consistency Reliability

In the seminal psychometric validation study conducted by Elliott and Treuting (1991) utilizing a sample of 216 educators, internal consistency was evaluated using Cronbach’s alpha coefficient (α). The results revealed high homogeneity across items within the total instrument and its derived subscales:

• Total BIRS Instrument (24 Items): α = 0.97
• Factor 1: Acceptability Subscale (15 Items): α = 0.97
• Factor 2: Effectiveness Subscale (7 Items): α = 0.92
• Factor 3: Time / Speed of Effect Subscale (2 Items): α = 0.87

These values fall well above the standard psychometric threshold of α ≥ 0.80 for research instruments and α ≥ 0.90 for high-stakes individual clinical and educational decision-making. The total scale reliability (α = 0.97) matches the internal consistency of the predecessor IRP-15 (α = 0.98; Martens et al., 1985), indicating that the addition of the nine effectiveness and time items retained high measurement precision.

Standard Error of Measurement and Stability

The Standard Error of Measurement (SEM) for the total BIRS score is low across typical teacher cohorts (SEM ≈ 3.2 to 3.8 on a 144-point raw score scale), providing narrow confidence intervals when evaluating practitioner social validity. Independent replication studies examining test-retest reliability over two-to-four-week intervals in ongoing school consultation cases (e.g., Sheridan et al., 1990; Freer & Watson, 1999) yielded test-retest correlation coefficients ranging between r = 0.82 and r = 0.89, confirming temporal stability when intervention descriptions remain constant.

9. Factor Analysis

The structural validity of the BIRS was evaluated through exploratory factor analysis (EFA) using principal axis factoring followed by an oblique (promax) rotation, chosen because the underlying latent dimensions of acceptability and effectiveness were theoretically expected to correlate.

Sample and Extraction Parameters

The standardization sample consisted of 216 practicing educators enrolled in graduate-level coursework across two major universities in Louisiana. Over 80% were general education teachers, and the remainder were certified special education teachers. Participants read a standardized vignette depicting a student exhibiting severe classroom disruption and completed the 24 candidate BIRS statements.

Factor retention was governed by eigenvalues greater than 1.0 (Kaiser-Guttman criterion) and scree plot inspection. To ensure clear factor identification, stringent factor loading retention rules were applied:

• An item was retained on a factor only if its primary factor loading was greater than 0.50.
• An item was excluded or deemed ambiguous if its cross-loadings on secondary factors exceeded 0.30.

Variance and Factor Loadings

The three extracted factors collectively accounted for 73.6% of the total cumulative variance. The distribution of items across factors mapped cleanly onto the hypothesized constructs:

• Factor 1: Acceptability (15 Items): Composed of all 15 items inherited from the original IRP-15 (Items 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and item 3 cross-aligned). Factor loadings for this domain were uniformly strong, ranging between 0.58 and 0.88. This factor accounted for the largest proportion of total explained variance (62.2%).
• Factor 2: Effectiveness (7 Items): Composed of seven of the newly generated outcome items (Items 17, 18, 20, 21, 22, 23, and 24). These items reflect clinical normalization, cross-setting generalization, collateral improvement, and behavioral maintenance. Primary factor loadings on this dimension ranged from 0.54 to 0.84, accounting for 7.8% of the total variance.
• Factor 3: Time / Timeliness (2 Items): Composed of the remaining two newly developed items (Item 16: quick improvement; Item 19: soon notice positive change). Factor loadings were 0.81 and 0.83, respectively, accounting for 3.6% of the variance.

Inter-Factor Correlations

Oblique rotation demonstrated moderate-to-high correlations among the latent factors:

• Factor 1 (Acceptability) and Factor 2 (Effectiveness): r = 0.63
• Factor 1 (Acceptability) and Factor 3 (Time): r = 0.51
• Factor 2 (Effectiveness) and Factor 3 (Time): r = 0.58

These inter-factor correlations support the Witt-Elliott theoretical model: while acceptability and perceived effectiveness represent distinct psychometric dimensions, they share substantial variance, indicating that teachers rarely judge an intervention to be highly acceptable if they consider it ineffective or excessively slow to produce results.

10. Instrument / Measurement Tool

The Behavior Intervention Rating Scale (BIRS) is a self-administered, paper-and-pencil or digital questionnaire designed for completion by educators, school clinicians, paraprofessionals, and intervention teams.

Structural Attributes

• Instrument Name: Behavior Intervention Rating Scale (BIRS)
• Primary Reference: Elliott, S. N., & Treuting, M. V. B. (1991)
• Target Population: General education teachers, special education teachers, school psychologists, behavioral consultants, educational specialists, and related service providers
• Administration Time: Approximately 5 to 10 minutes
• Total Item Count: 24 statements
• Dimensional Structure:
  • Acceptability Subscale: 15 items (Items 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and related procedural elements)
  • Effectiveness Subscale: 7 items (Items 17, 18, 20, 21, 22, 23, 24)
  • Time / Speed of Effect Subscale: 2 items (Items 16, 19)
• Response Format: 6-point Likert-type scale, scored as follows:
  • 1 = Strongly Disagree
  • 2 = Disagree
  • 3 = Slightly Disagree
  • 4 = Slightly Agree
  • 5 = Agree
  • 6 = Strongly Agree

Scoring and Metric Interpretation

The BIRS does not contain reverse-scored items; every statement is keyed in a positive direction, such that higher numerical values indicate greater acceptability, greater projected efficacy, and faster expected onset of improvement.

• Total Score Range: 24 to 144 points. Higher scores reflect stronger overall social validity.
• Acceptability Subscale Range: 15 to 90 points.
• Effectiveness Subscale Range: 7 to 42 points.
• Time Subscale Range: 2 to 12 points.
• Mean Item Score Interpretation: Total or subscale raw scores are frequently divided by the number of completed items to yield a scale mean between 1.0 and 6.0:
  • 1.00 – 3.49: Unacceptable / Low Social Validity. Reflects skepticism, perceived procedural burden, or low projected efficacy. Interventions in this range risk non-implementation or high teacher resistance.
  • 3.50 – 4.49: Equivocal / Moderate Acceptability. Indicates neutral or tentative endorsement. Consultation, procedural simplification, or additional modeling is recommended prior to launch.
  • 4.50 – 6.00: High Acceptability / Strong Social Validity. Reflects strong professional endorsement, operational feasibility, and high perceived probability of success. Interventions in this tier correlate with high implementation fidelity.

11. Permissions & Fee and Test Year

The Behavior Intervention Rating Scale was published in 1991 in the Journal of School Psychology (Vol. 29, Issue 1, pp. 43–51) by the Society for the Study of School Psychology (SSSP), published by Elsevier.

• Publication Year: 1991
• Copyright and Permissions: The original copyright is held by the Society for the Study of School Psychology and Elsevier. The instrument was released as an academic psychometric tool for research and applied educational evaluation. The 24 items and scoring key were published directly in the primary journal article for scholarly dissemination.
• Commercial Fees: There is no proprietary software license fee or commercial purchase requirement for academic research, school-based consultation, or non-commercial clinical use. Practitioners and researchers may reproduce the scale items for empirical or educational applications, provided proper scholarly attribution is maintained. Commercial redistributions or inclusion within proprietary diagnostic software suites require formal copyright clearance through Elsevier / RightsLink.

12. References

The following academic publications document the development, validation, and clinical application of the Behavior Intervention Rating Scale and its theoretical foundations:

• Clark, L., & Elliott, S. N. (1988). The influence of treatment strength information on knowledgeable teachers’ evaluation of two social skills training methods. Professional School Psychology, 3(4), 241–251. https://doi.org/10.1037/h0090562
• Elliott, S. N., & Treuting, M. V. B. (1991). The Behavior Intervention Rating Scale: Development and validation of a pretreatment acceptability and effectiveness measure. Journal of School Psychology, 29(1), 43–51. https://doi.org/10.1016/0022-4405(91)90014-I
• Freer, P., & Watson, T. S. (1999). A comparison of parent and teacher acceptability ratings of behavioral interventions for children’s school-based problems. School Psychology Review, 28(4), 674–684. https://doi.org/10.1080/02796015.1999.12085992
• Kazdin, A. E. (1980). Acceptability of alternative treatments for deviant child behavior. Journal of Applied Behavior Analysis, 13(2), 259–273. https://doi.org/10.1901/jaba.1980.13-259
• Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16(2), 191–198. https://doi.org/10.1037/0735-7028.16.2.191
• Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. University of Illinois Press.
• Sheridan, S. M., Kratochwill, T. R., & Elliott, S. N. (1990). Behavioral consultation with parents and teachers: Delivering treatment for socially withdrawn children at home and school. School Psychology Review, 19(1), 33–52. https://doi.org/10.1080/02796015.1990.12085444
• Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom management strategies. In T. R. Kratochwill (Ed.), Advances in school psychology (Vol. 4, pp. 251–288). Lawrence Erlbaum Associates.
• Witt, J. C., Elliott, S. N., & Martens, B. K. (1984). Acceptability of behavioral interventions used in classrooms: The influences of amount of time, severity of behavior problems, and types of intervention. Behavioral Disorders, 9(2), 95–104. https://doi.org/10.1177/019874298400900201
• Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11(2), 203–214. https://doi.org/10.1901/jaba.1978.11-203

13. Items of the Scale (Questionnaire)