Direct Observation Instrument (BRAT-DO)

Abstract

The Direct Observation Instrument (BRAT-DO) is an objective, comprehensive environmental audit instrument developed by Ariane L. Bedimo-Rung, Jeanette Gustat, Barbara J. Tompkins, J. Rice, and J. Thomson in 2006 as a central component of the Bedimo-Rung Assessment Tools. Built to operationalize the built environment determinants of physical activity in municipal park settings, the BRAT-DO systematically measures the physical, social, and policy characteristics of public parks. Comprising 181 distinct observation variables across discrete spatial units termed Target Areas (TAs), the tool evaluates five primary environmental domains: Features, Condition, Access, Esthetics, and Safety. Unlike subjective self-report questionnaires that are vulnerable to recall bias, social desirability, and cognitive filtering, the BRAT-DO provides an empirical, direct visual methodology for quantifying the affordances and barriers within municipal recreation infrastructures. Psychometric evaluations of the BRAT-DO demonstrate moderate to almost perfect inter-rater reliability across its observational dimensions (Cohen’s kappa coefficients ranging from 0.41 to 1.00; percent agreement frequently exceeding 80% to 95%), with highest concordance observed on objective physical feature inventories and lower, yet acceptable, reliability on subjective esthetic and maintenance gradations. Construct and criterion validity have been substantiated through significant correlations with systematic direct observation of physical activity (e.g., using SOPARC protocols), population-level recreation patterns, and municipal geographic information system (GIS) spatial data layers. The instrument functions both as a granular diagnostic tool for urban planners and park administrators and as an empirical research instrument in active living research, environmental psychology, and public health epidemiology.

Keywords

Direct Observation Instrument, BRAT-DO, Bedimo-Rung Assessment Tools, built environment, physical activity, park audit, active living research, environmental assessment, inter-rater reliability, urban public health, recreational infrastructure, environmental psychology.

Authors

The Direct Observation Instrument (BRAT-DO) was conceived, operationalized, and psychometrically validated by an interdisciplinary research team specializing in public health, epidemiology, and environmental design:

• Ariane L. Bedimo-Rung, Ph.D., MPH: Principal investigator; associated with the Department of Public Health Sciences, School of Public Health, Louisiana State University Health Sciences Center, New Orleans, Louisiana, USA. Dr. Bedimo-Rung’s scholarship focuses on environmental and policy determinants of physical activity, chronic disease prevention, and community health assessment methodologies.
• Jeanette Gustat, Ph.D., MPH: Co-investigator; affiliated with the Tulane Prevention Research Center, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, Louisiana, USA. Dr. Gustat is an authority on built environment measurement, community intervention design, and cardiovascular disease epidemiology.
• Barbara J. Tompkins, MPH: Research associate and project manager within the Department of Public Health Sciences, Louisiana State University Health Sciences Center, contributing to field audit protocol standardization and spatial sampling designs.
• J. Rice, Ph.D.: Biostatistician and psychometrician; involved in the statistical modeling, reliability matrix calculation (kappa coefficients and concordance statistics), and psychometric structural validation.
• J. Thomson, Ph.D.: Public health scientist and quantitative analyst specializing in spatial epidemiology, physical activity measurement paradigms, and environmental audit validation.

Purpose

The primary purpose of the Direct Observation Instrument (BRAT-DO) is to provide an empirical, standardized, and reproducible measurement protocol for systematically assessing the multi-dimensional environmental characteristics of municipal parks that influence human physical activity. Public parks represent foundational community infrastructures that support leisure-time physical activity, active transportation, sports participation, and prosocial community engagement. However, traditional surveillance and active living research methodologies have historically suffered from a reliance on self-report questionnaires. Self-report measures are susceptible to significant psychometric limitations, including perceptual distortion, cognitive recall biases, neighborhood attachment biases, and social desirability effects. The BRAT-DO was designed to transcend these subjective constraints by introducing an objective, standardized direct-observation audit protocol conducted by trained raters.

From an applied epidemiological perspective, public health professionals require detailed structural data to ascertain which environmental attributes directly encourage moderate-to-vigorous physical activity (MVPA) versus those that foster sedentary behaviors. The BRAT-DO fulfills this function by categorizing parks into discrete geographic and functional zones, evaluating not merely the nominal presence or absence of facilities, but the qualitative nuances of those facilities—such as physical wear, accessibility, sensory pleasantness, and overt physical safety risks. By measuring park infrastructure at this level of granularity, the BRAT-DO enables investigators to identify specific environmental mediators that explain why two geographically proximate parks may yield radically disparate community utilization rates.

In municipal planning, urban design, and civic governance, the instrument serves as an evidence-based capital investment and diagnostic framework. Municipal parks and recreation departments utilize the BRAT-DO to establish baseline infrastructure inventories, prioritize deferred maintenance budgets, evaluate the impact of park redesign or urban greening initiatives, and audit equity in public resource allocation across socioeconomically diverse urban sectors. By standardizing physical audits, municipal stakeholders can conduct pre- and post-intervention natural experiments to evaluate whether targeted capital investments (e.g., pathway resurfacing, sports lighting installation, or playground renovation) directly result in measurable surges in population-level park usage and community-wide physical activity.

Psychological Construct

The overarching psychological and behavioral construct quantified by the BRAT-DO is the park physical activity affordance environment. Grounded in ecological and environmental psychology, the instrument conceives the physical park environment not as an inert spatial backdrop, but as a dynamic matrix of environmental stimuli, physical affordances, structural constraints, and perceptual cues that either facilitate or extinguish physical activity behavior. The instrument operationalizes this overarching construct into five interdependent, lower-order conceptual domains:

1. Features

The Features domain assesses the nominal and quantitative presence of physical activity affordances—the tangible facilities, recreational structures, and amenities that functionally accommodate physical exertion and active play. Within environmental psychology, features operate as behavior-specific affordances (Gibson, 1979) that determine the behavioral repertoire supported by the setting. In the BRAT-DO, this domain operationalizes active infrastructure (e.g., walking and jogging trails, bike paths, basketball courts, tennis courts, baseball/softball diamonds, multi-use soccer/football fields, and playgrounds) alongside essential support amenities (such as benches, picnic shelters, drinking fountains, and sanitary restrooms). Without basic physical features, the environmental capacity to elicit purposeful physical activity is fundamentally circumscribed.

2. Condition

The Condition domain evaluates the objective physical state, maintenance level, functional integrity, and surface wear of existing features. Psychologically, condition operates as an indicator of environmental order and functional competence. Even when a park possesses diverse features, deteriorated, damaged, or unmaintained amenities act as strong behavioral deterrents. Damaged play equipment, cracked court surfaces, severely rutted or flooded sports fields, eroded running trails, and unsanitary or non-functioning restrooms introduce structural hazards and convey institutional neglect. Condition ratings within the BRAT-DO evaluate the extent of corrosion, wear, structural failure, and cleanliness across each functional target area.

3. Access

The Access domain quantifies the physical, geographical, and institutional permeability of the park environment. This construct encompasses both the external connectivity of the park to surrounding urban fabrics and its internal traversability. External access assesses whether pedestrian sidewalks surrounding the park are continuous and maintained, whether marked crosswalks and traffic calming devices facilitate safe pedestrian arrival, whether public transit stops are adjacent, and whether bicycle racks provide active transport storage. Internal access captures universal design parameters, including Americans with Disabilities Act (ADA) accessible pathways, clear entrance visibility, perimeter fencing, and the presence of physical gates or locks that govern temporal or institutional access to sports fields and recreational zones.

4. Esthetics

The Esthetics domain captures the sensory, visual, and experiential qualities of the park landscape. In psychological theory, esthetic environments activate restorative attentional processes (Kaplan & Kaplan, 1989) and elicit positive affective states that lengthen dwell times and increase visit frequency. The BRAT-DO measures esthetic factors including the extent of tree canopy and shade cover, landscaping maturity and maintenance, the presence of natural elements or scenic water features, visual appeal from within target areas, and the absence of visual disorder such as litter, discarded debris, and unmaintained vegetation.

5. Safety

The Safety domain assesses environmental markers of personal and physical security, incorporating concepts from Crime Prevention Through Environmental Design (CPTED) and the Broken Windows Theory. Safety operationalizes both real and perceived risks that deter park entry and usage, particularly among vulnerable cohorts such as children, women, and older adults. The BRAT-DO records structural indicators of crime and disorder, including graffiti and vandalism, evidence of illicit activity, the adequacy and functioning of pedestrian and sports lighting fixtures, unobstructed sightlines that preserve natural surveillance from adjacent roadways, and the buffering of play areas from high-speed vehicular traffic corridors.

Theoretical Framework

The development of the BRAT-DO is rooted within the multidisciplinary convergence of ecological models of health behavior, environmental psychology, and crime prevention theory. Specifically, the instrument synthesizes three primary theoretical architectures:

1. The Social-Ecological Model of Physical Activity

The BRAT-DO’s primary conceptual architecture is the Social-Ecological Model of health behavior (McLeroy et al., 1988; Sallis et al., 2006; Stokols, 1992). Ecological models posit that health behaviors like physical activity are not driven exclusively by intra-individual psychological factors (such as self-efficacy, behavioral intention, or autonomous motivation), but are fundamentally shaped by dynamic, reciprocal interactions across nested spheres of influence: intrapersonal, interpersonal, institutional, physical environmental, and public policy domains. The BRAT-DO specifically operationalizes the physical built environment level of this model. It posits that environmental settings act as direct upstream determinants that either enable, catalyze, or thwart individual motivation for active living.

2. The Bedimo-Rung Conceptual Model of Park Use and Physical Activity

Directly preceding the empirical formulation of the audit instrument, Bedimo-Rung, Mowen, and Cohen (2005) published an overarching conceptual framework explicating the precise pathways through which park environments drive physical activity and subsequent community health outcomes. This model theorizes that park characteristics—categorized into Features, Condition, Access, Esthetics, and Safety—interact with user characteristics (demographics, psychosocial factors) and park policy/management regimes. These environmental attributes directly govern two sequential behavioral stages: first, park visitation (the decision to enter and linger in the park), and second, physical activity behavior within the park (the duration, frequency, and intensity of movement). Positive environmental attributes across the five domains minimize cognitive and physical friction, eliciting higher levels of physical exertion and broader public health benefits.

3. Environmental Psychology and Defensible Space Theories

The psychological rationale for the Condition, Esthetics, and Safety domains is informed by Attention Restoration Theory (ART) (Kaplan, 1995) and Stress Recovery Theory (Ulrich, 1983), which document how natural, well-maintained green elements induce cognitive restoration, alleviate psychological fatigue, and promote approach-related health behaviors. Concurrently, the audit’s safety and access operationalization draws heavily from Newman’s (1972) Defensible Space Theory, Crime Prevention Through Environmental Design (CPTED), and Wilson and Kelling’s (1982) Broken Windows Theory. These criminological frameworks state that signs of physical disorder (graffiti, broken fixtures, overgrown sightlines, and litter) signal a breakdown in territorial functioning and social control, creating subjective fears of victimization that dramatically reduce outdoor recreational activity.

Validity

The psychometric validity of the BRAT-DO was established through rigorous multi-phase validation protocols involving content, construct, and criterion-related methodologies:

1. Content and Face Validity

Content validity was developed through an extensive review of public health, kinesiology, landscape architecture, and urban planning literature, followed by structured expert panel evaluations. A multidisciplinary panel of epidemiologists, active living researchers, municipal park managers, and environmental design specialists evaluated draft items for clarity, comprehensiveness, and theoretical alignment with physical activity affordances. Items exhibiting semantic ambiguity or low conceptual distinctiveness were iteratively refined, collapsed, or removed. The spatial decomposition strategy—partitioning sprawling parks into discrete, functionally coherent Target Areas (TAs)—was adopted to resolve spatial aggregation bias, ensuring that localized features and micro-environmental hazards were accurately recorded without inflating the entire park’s score.

2. Construct Validity

Construct validity has been supported by demonstrating that BRAT-DO domain scores accurately discriminate between diverse typologies of municipal parks and correspond with objective socioeconomic indicators. Studies assessing municipal park environments across distinct geographic sectors revealed statistically significant disparities: parks located in high-income neighborhoods exhibited systematically higher scores on the Features, Condition, and Esthetics domains, while parks in economically marginalized sectors exhibited elevated physical disorder, poorer court conditions, and compromised safety markers. These findings confirmed the instrument’s capacity to detect built environmental inequities established in environmental justice literature.

3. Criterion and Convergent Validity

Criterion-related validity was substantiated by linking BRAT-DO audit profiles with empirical observations of human physical activity. Investigators deployed the System for Observing Play and Recreation in Communities (SOPARC; McKenzie et al., 2006) concurrently with BRAT-DO environmental audits. Target areas characterized by superior BRAT-DO Feature counts, higher Condition ratings, and elevated Esthetic scores displayed statistically significantly higher volumes of park visitors engaged in moderate-to-vigorous physical activity (MVPA) compared to areas with degraded or missing amenities ($p < .01$). Furthermore, convergent validity was established against independent Geographic Information System (GIS) land-use metrics; direct observation ratings of external park access, transit adjacency, and sidewalk completeness exhibited high positive correlations with municipal spatial network datasets ($r = .72$ to $.86$).

Reliability

The reliability of direct observation environmental audit tools depends primarily on inter-rater agreement and temporal stability. Bedimo-Rung et al. (2006) conducted comprehensive psychometric testing to quantify inter-rater reliability (IRR) across independent trained observers auditing identical park environments:

1. Inter-Rater Reliability (IRR) Metrics

Reliability testing utilized pairs of independently trained observers who audited park target areas simultaneously but without conferring. Inter-rater concordance was evaluated using Cohen’s kappa ($kappa$) for categorical variables, prevalence-adjusted bias-adjusted kappa (PABAK) where low base rates artificially depressed standard kappa values, and raw percent agreement:

• Features Domain: Exhibited the highest psychometric reliability across the instrument. Nominal inventories of sports fields, court types, playground equipment, and basic amenities yielded Cohen’s kappa values ranging from $kappa = 0.81$ to $1.00$ (almost perfect agreement), with percent agreement consistently exceeding $90%$ to $98%$.
• Access Domain: Structural access variables, such as the presence of perimeter fencing, sidewalk continuity, and ADA curb ramps, demonstrated robust concordance ($kappa = 0.72$ to $0.94$; agreement $> 88%$). Items evaluating traffic volume on surrounding streets exhibited moderate-to-substantial reliability ($kappa = 0.58$ to $0.74$).
• Condition Domain: Categorical and ordinal ratings of structural wear, surface rust, and maintenance exhibited substantial agreement ($kappa = 0.61$ to $0.80$; agreement $75%$ to $88%$). Objective surface integrity (e.g., path surfaces, court fractures) demonstrated higher reliability than subjective overall landscaping condition ratings.
• Esthetics Domain: Shade cover estimation, visual viewscape appeal, and landscaping beauty generated moderate reliability coefficients ($kappa = 0.44$ to $0.68$; agreement $70%$ to $84%$), reflecting the cognitive subjectivity inherent in esthetic evaluations.
• Safety Domain: Items tracking overt physical disorder—such as the presence of graffiti, broken bottles, broken light fixtures, and perimeter sightline obstructions—yielded substantial to almost perfect concordance ($kappa = 0.65$ to $0.89$).

2. Intra-Rater and Test-Retest Stability

Test-retest evaluations demonstrated high temporal stability across the built structural dimensions of the instrument (e.g., pathway existence, court surface type, fencing). Dynamic variables—such as transient litter, overflowing trash receptacles, and temporary gate closures—exhibited expected diurnal fluctuations, confirming that repeated administrations over differing seasonal and weekly cycles are necessary to capture transient maintenance variations.

Factor Analysis

Direct observation environmental audit instruments such as the BRAT-DO deviate fundamentally from classical psychological self-report scales in their latent structural properties. Psychometricians distinguish between reflective measurement models (where observed responses are manifestations of an underlying latent psychological trait) and formative (composite) measurement models (where observed indicators collectively define or construct an environmental index). The BRAT-DO operates primarily through formative and hierarchical index structures:

1. Formative vs. Reflective Structural Reality

In standard psychometrics, factor analysis (Exploratory Factor Analysis [EFA] and Confirmatory Factor Analysis [CFA]) assumes that items within a subscale are reflective indicators driven by a single common latent factor, meaning they should exhibit high internal consistency (Cronbach’s alpha). In an environmental audit, however, park features do not necessarily correlate positively with one another; the installation of a soccer field does not cause or necessarily co-occur with the installation of a swimming pool or tennis court. Consequently, calculating internal consistency statistics across all 181 audit variables is conceptually inappropriate.

2. Structural Verification of Domain Dimensions

During the developmental psychometric phases, researchers applied principal component analysis (PCA) and exploratory factor analysis to subsets of evaluative ordinal items (specifically within the Condition, Esthetics, and Safety domains). Factor analyses confirmed distinct latent dimensions corresponding with the theoretical taxonomy:

• Maintenance and Structural Degradation Factor: Captured item loadings exceeding $0.60$ for playground corrosion, court structural damage, path surface rutting, and sports field turf erosion, explaining over $38%$ of the shared variance across physical quality items.
• Physical Incivilities and Social Disorder Factor: Characterized by strong factor loadings ($0.65$ to $0.82$) on litter density, presence of graffiti, uncollected waste, and broken security fixtures.
• Natural and Restorative Esthetics Factor: Loaded heavily with percentage of shade cover, garden upkeep, natural water elements, and interior viewscape ratings.

These empirical factor extractions supported the operational separation of the five core domains, validating the theoretical framework proposed by Bedimo-Rung et al. (2005) while establishing formative composite scoring rules for macro-level park evaluations.

Instrument / Measurement Tool

The Direct Observation Instrument (BRAT-DO) is a specialized environmental audit protocol administered by trained observers who physically traverse a park. It is structured around the systematic decomposition of a park into discrete spatial subunits.

• Instrument Type: Direct Observation Environmental Audit Form / Field Assessment Checklist.
• Spatial Unit of Measurement: The Target Area (TA). Large or heterogeneous parks are mapped and divided into distinct, contiguous, functional zones prior to field observation (e.g., Playground TA, Ballfield TA, Court TA, Natural Green Space TA, Trail/Path TA). An audit form is completed for each individual Target Area, alongside a park-level summary audit.
• Total Number of Variables: 181 individual observational fields and variables spanning the five conceptual domains.
• Observational Domains:
  • Features: Physical infrastructure, sports amenities, play structures, seating, and utilities.
  • Condition: Maintenance, physical wear, surface deterioration, functional usability.
  • Access: Gates, perimeter fencing, locks, ADA ramps, sidewalks, transit stops, bike parking.
  • Esthetics: Natural elements, shade canopy, landscaping appeal, cleanliness, litter burden.
  • Safety: Lighting fixtures, sightlines/natural surveillance, graffiti, broken glass, traffic mitigation.
• Response Formats:
  • Dichotomous (Present / Absent; Yes / No).
  • Counts (continuous integer tallies of specific amenities, e.g., number of basketball hoops, picnic tables, benches).
  • Categorical / Ordinal Condition Ratings (e.g., Poor / Fair / Good / Excellent; Clean / Somewhat clean / Dirty; None / Low / Moderate / High; Unobstructed / Partially obstructed / Blocked).
• Observer Training Protocol: To achieve acceptable inter-rater reliability, field observers undergo a standardized training curriculum lasting approximately 8 to 12 hours. This curriculum entails classroom reviews of operational definitions, photo manual evaluations of ambiguous condition ratings, and field-based concurrent practice audits in benchmarked parks until an inter-rater concordance threshold of $kappa ge 0.75$ or percent agreement $ge 85%$ is attained.
• Scoring and Index Computation:
  • Domain-Specific Scores: Computed for each Target Area by summing affirmative feature presences, condition points (e.g., Poor = 1 to Excellent = 4), and reverse-scored incivilities (e.g., Heavy litter = 0, None = 3).
  • Park-Level Composite Indices: Calculated either by area-weighted averaging across all audited Target Areas within the park boundary or through an unweighted additive aggregation of core infrastructure scores.

Permissions & Fee and Test Year

The Direct Observation Instrument (BRAT-DO) was developed in 2006 under research grants supported by the Active Living Research (ALR) program of the Robert Wood Johnson Foundation (RWJF) and the Centers for Disease Control and Prevention (CDC). The instrument resides in the academic public domain and is accessible free of charge for non-commercial research, academic inquiry, municipal planning, and public health surveillance purposes.

Investigators and organizations seeking to deploy, modify, or electronically digitize the BRAT-DO (such as for mobile geospatial or GIS-enabled audit applications) are not required to pay licensing fees. However, appropriate scholarly attribution must be maintained by citing the primary validation publication: Bedimo-Rung et al. (2006), Journal of Physical Activity and Health. Researchers adapting the tool for specific regional contexts or language translations are encouraged to maintain structural alignment with the original five domains to preserve comparative psychometric validity.

References

• Bedimo-Rung, A. L., Gustat, J., Tompkins, B. J., Rice, J., & Thomson, J. (2006). Development of a direct observation instrument to measure environmental characteristics of parks for physical activity. Journal of Physical Activity and Health, 3(Suppl 1), S176–S189. https://doi.org/10.1123/jpah.3.s1.s176
• Bedimo-Rung, A. L., Mowen, A. J., & Cohen, D. A. (2005). The significance of parks to physical activity and public health: A conceptual model. American Journal of Preventive Medicine, 28(2), 159–168. https://doi.org/10.1016/j.amepre.2004.10.024
• Gibson, J. J. (1979). The ecological approach to visual perception. Houghton Mifflin.
• Kaplan, R., & Kaplan, S. (1989). The experience of nature: A psychological perspective. Cambridge University Press.
• Kaplan, S. (1995). The restorative benefits of nature: Toward an integrative framework. Journal of Environmental Psychology, 15(3), 169–182. https://doi.org/10.1016/0272-4944(95)90001-2
• McKenzie, T. L., Cohen, D. A., Sehgal, A., Williamson, S., & Golinelli, D. (2006). System for Observing Play and Recreation in Communities (SOPARC): Reliability and feasibility measures. Journal of Physical Activity and Health, 3(Suppl 1), S208–S222. https://doi.org/10.1123/jpah.3.s1.s208
• McLeroy, K. R., Bibeau, D., Steckler, A., & Glanz, K. (1988). An ecological perspective on health promotion programs. Health Education Quarterly, 15(4), 351–377. https://doi.org/10.1177/109019818801500401
• Newman, O. (1972). Defensible space: Crime prevention through urban design. Macmillan.
• Sallis, J. F., Cervero, R. B., Ascher, W., Henderson, K. A., Kraft, M. K., & Kerr, J. (2006). An ecological approach to creating active living communities. Annual Review of Public Health, 27(1), 297–322. https://doi.org/10.1146/annurev.publhealth.27.021405.102100
• Stokols, D. (1992). Establishing and maintaining healthy environments: Toward a social ecology of health promotion. American Psychologist, 47(1), 6–22. https://doi.org/10.1037/0003-066X.47.1.6
• Ulrich, R. S. (1983). Aesthetic and affective response to natural environment. In I. Altman & J. F. Wohlwill (Eds.), Behavior and the natural environment (pp. 85–125). Springer. https://doi.org/10.1007/978-1-4613-3539-9_4
• Wilson, J. Q., & Kelling, G. L. (1982). Broken windows: The police and neighborhood safety. The Atlantic Monthly, 249(3), 29–38.

13. Items of the Scale (Questionnaire)