EdTech Context Inventory

Abstract

The EdTech Context Inventory, developed by Emily A. Kohler, Lauren Molloy Elreda, and Kate Tindle in 2023, is a psychometrically validated suite comprising ten distinct measurement instruments engineered to evaluate the organizational, cultural, infrastructural, and human factors that govern educational technology (edtech) adoption in primary and secondary educational ecosystems. Derived from the multi-stakeholder consensus initiative known as the EdTech Genome Project, the inventory quantifies ten foundational contextual dimensions: Vision for Teaching & Learning, Infrastructure & Operations, Selection Processes, Implementation Systems & Processes, Teacher Agency, Strategic Leadership Support, Staff Culture, Teacher Beliefs & Knowledge, Professional Learning, and Competing Priorities. The entire inventory encompasses a total of 188 items distributed across the ten instruments, with individual instrument lengths ranging between 8 and 24 items. Responses are predominantly recorded via 5-point Likert scales capturing either degrees of agreement (1 = Strongly Disagree to 5 = Strongly Agree) or behavioural frequency (1 = Never to 5 = Always). Psychometric evaluation across two independent educator samples demonstrated robust structural validity through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) within a structural equation modeling (SEM) framework, identifying between three and six factors per instrument. Internal consistency reliability is high, with both Cronbach’s alpha (α) and McDonald’s omega (ω) exceeding .80 across all scales. Inter-scale correlations range from r = .32 to r = .82, confirming meaningful divergence among specific operational domains alongside convergent cohesion around broader systemic leadership and cultural constructs. The inventory provides researchers, district administrators, and educational evaluators with an actionable diagnostic architecture to mitigate implementation failure and optimize instructional technology investments.

Keywords

EdTech Context Inventory, educational technology implementation, psychometrics, teacher agency, strategic leadership, implementation science, confirmatory factor analysis, structural equation modeling, school infrastructure, professional learning

Authors

The EdTech Context Inventory was designed, validated, and published by a collaborative research team associated with the University of Virginia School of Education and Human Development and the EdTech Evidence Exchange:

• Emily A. Kohler, Ph.D. — University of Virginia School of Education and Human Development, Charlottesville, Virginia, United States. ORCID: 0000-0003-0470-0891. Email: [email protected] (Corresponding Author).
• Lauren Molloy Elreda, Ph.D. — University of Virginia School of Education and Human Development, Charlottesville, Virginia, United States.
• Kate Tindle, Ed.D. — EdTech Evidence Exchange, Charlottesville, Virginia, United States.

Purpose

The primary purpose of the EdTech Context Inventory is to resolve one of the most persistent and costly dilemmas in modern educational systems: the high rate of educational technology underutilization, abandonment, and implementation failure. Annually, school districts and national educational agencies allocate tens of billions of dollars toward digital learning software, hardware, and digital curricular platforms. However, empirical studies in educational policy consistently indicate that a substantial fraction of acquired software licenses are rarely or never activated, and tools with documented efficacy in randomized controlled trials frequently fail to produce measurable learning gains when deployed in routine classroom environments. Historically, evaluation frameworks focused narrowly on product-level efficacy—asking “Does this software work?”—while neglecting the organizational, psychological, and systemic ecology into which the technology is inserted.

The EdTech Context Inventory fundamentally shifts this paradigm toward implementation science, asking instead: “Under what specific contextual conditions does educational technology succeed or fail?” The instrument suite provides a psychometrically standardized, diagnostic mechanism to profile the multi-layered environment of a school or district prior to, during, or following technology deployment. Rather than evaluating a specific software application or hardware device, the inventory measures the macro- and micro-level contextual features that moderate implementation success across any digital tool.

In applied educational research, the inventory serves as an operationalized toolkit to assess contextual moderators in field trials, quasi-experiments, and large-scale efficacy studies. In administrative and district practice, it equips superintendents, chief technology officers, curriculum directors, and school principals with actionable diagnostic profiles. These profiles uncover institutional bottlenecks—such as technical infrastructure limitations, misalignment between technological capabilities and pedagogical visions, disenfranchised teacher agency, top-down product selection devoid of practitioner input, or overwhelming competing initiatives. By identifying organizational deficits before capital allocation, school systems can target pre-implementation interventions, tailor professional development pathways, and build sustainable organizational capacity.

Psychological Construct

The EdTech Context Inventory conceptualizes educational technology implementation context not as a monolithic environment, but as a dynamic, multi-dimensional socio-technical system. The construct reflects the interplay between individual practitioner cognition, interpersonal staff dynamics, physical/digital systems, and executive leadership behaviors. Across its ten dedicated instruments, the inventory measures ten distinct constructs:

1. Vision for Teaching & Learning

This construct captures the degree to which a school or district maintains an articulated, shared, and actionable philosophy regarding pedagogical practice and student-centered learning goals, and how digital tools are purposefully integrated to advance that philosophy rather than serving as superficial substitutes for analog instruction.

2. Infrastructure & Operations

This construct measures the systemic technical, logistical, and hardware/software foundation supporting digital learning. It encompasses reliable broadband connectivity, adequate device ratios, interoperability, robust data privacy safeguards, technical troubleshooting responsiveness, and predictable device lifecycle management.

3. Selection Processes

This construct examines the formal and informal mechanisms by which educational technology solutions are vetted, evaluated, and procured. It quantifies the transparency of procurement criteria, reliance on empirical evidence of product efficacy, alignment with curricular standards, and the active involvement of classroom educators and support specialists in pilot testing and purchasing decisions.

4. Implementation Systems & Processes

This dimension evaluates the operational infrastructure designed to manage technology rollouts over time. It assesses project management rigor, feedback loops, roll-out timelines, monitoring of utilization fidelity, communication protocols between central administration and school sites, and ongoing logistical support.

5. Teacher Agency

Grounding itself in psychological constructs of autonomy, professional empowerment, and decision latitude, teacher agency assesses the degree of control and discretion educators have over how, when, and why they integrate specific digital tools into their instructional designs, as opposed to feeling coerced by prescriptive top-down mandates.

6. Strategic Leadership Support

This construct reflects instructional and administrative leadership behaviors at both the district and building levels. It measures how effectively administrators allocate resources, model technology integration, cultivate shared ownership, protect instructional time, and actively buffer educators from external implementation pressures.

7. Staff Culture

Rooted in organizational psychology, staff culture captures the social and relational climate of the school workplace. Key facets include mutual trust, psychological safety for pedagogical experimentation, collaborative problem-solving, norms of peer support, and collective efficacy regarding school-wide improvement initiatives.

8. Teacher Beliefs & Knowledge

This internal practitioner construct assesses cognitive dispositions, epistemological beliefs about digital pedagogical integration, and self-efficacy regarding technology use. It incorporates dimensions akin to the Technological Pedagogical Content Knowledge (TPACK) framework, focusing on how confident and prepared educators feel to utilize technology to advance cognitive engagement and differentiate instruction.

9. Professional Learning

This construct measures the quality, accessibility, modality, and pedagogical relevance of ongoing professional development. It distinguishes between one-off, compliance-oriented technical tutorials and high-impact, sustained, collaborative, and job-embedded coaching models that address authentic classroom challenges.

10. Competing Priorities

A pragmatic organizational construct measuring systemic friction, initiative fatigue, cognitive overload, and time poverty. It assesses the extent to which concurrent district initiatives—such as high-stakes standardized testing mandates, curriculum overhauls, compliance burdens, and administrative reporting—crowd out the mental bandwidth and instructional time required to successfully master and implement new technologies.

Theoretical Framework

The EdTech Context Inventory is theoretically situated at the nexus of implementation science, socio-technical systems theory, and psychological theories of human motivation and organizational behavior. Historically, educational measurement tools suffered from fragmented perspectives, isolating psychological traits of teachers (e.g., computer anxiety or personal self-efficacy) from systemic structural barriers (e.g., network bandwidth or administrative mandates).

Consolidated Framework for Implementation Research (CFIR)

The conceptual foundation aligns directly with the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). CFIR posits that implementation outcomes are governed by five interactive domains: intervention characteristics, outer setting, inner setting, individual characteristics, and the implementation process itself. The EdTech Context Inventory translates CFIR constructs into actionable K-12 operational realities:

• Inner Setting: Operationalized via Staff Culture, Infrastructure & Operations, and Vision for Teaching & Learning.
• Characteristics of Individuals: Operationalized through Teacher Beliefs & Knowledge and Teacher Agency.
• Implementation Process: Measured through Selection Processes, Implementation Systems & Processes, and Professional Learning.
• Leadership & Outer Environment Dynamics: Reflected in Strategic Leadership Support and Competing Priorities.

Diffusion of Innovations and Organizational Climate

The inventory draws heavily upon Everett Rogers’ Diffusion of Innovations Theory, which establishes that adoption rates depend critically on social systems, communication channels, perceived relative advantages, and reinvention opportunities (captured here as teacher agency). Concurrently, organizational climate theories (e.g., Schneider, Ehrhart, & Macey) inform the conceptualization of Staff Culture, framing psychological safety and organizational trust as necessary preconditions for practitioner risk-taking when transitioning from traditional to digitally augmented instructional paradigms.

Self-Determination Theory and Social Cognitive Theory

At the individual practitioner level, the inventory integrates Self-Determination Theory (Deci & Ryan), particularly the basic psychological need for autonomy, through its dedicated Teacher Agency instrument. When educators perceive technological mandates as extrinsic controlling mechanisms, resistance and token compliance increase. Conversely, when technological integration supports autonomous motivation and competence (Bandura’s Social Cognitive Theory), integration fidelity increases. The inventory simultaneously measures these individual cognitive-affective dynamics and the structural environments that nurture or suppress them.

Validity

The validation program of the EdTech Context Inventory utilized rigorous psychometric methodologies across multiple independent developmental and validation phases, adhering to the Standards for Educational and Psychological Testing (AERA, APA, & NCME).

Content and Face Validity

The initial generation of contextual features and item pools was conducted under the auspices of the EdTech Genome Project. A national steering committee composed of researchers, district administrators, classroom teachers, and educational policy experts identified the initial ten contextual dimensions. Subsequently, dedicated expert working groups were convened for each specific feature. These groups conducted targeted literature reviews, formulated construct definitions, and drafted initial item pools, ensuring that every item directly represented authentic instructional realities and possessed strong face and content validity.

Construct and Structural Validity

Construct validity was evaluated empirically using cross-sectional data collected from two large, independent samples of practicing educators across the United States. Following initial exploratory factor analyses (EFA) on Sample 1 to refine item assignments and eliminate psychometrically poor or cross-loading items, confirmatory factor analyses (CFA) within a structural equation modeling (SEM) framework were conducted on Sample 2. The CFA models confirmed multi-factorial architectures for each of the ten instruments, with each instrument resolving into between three and six well-defined latent sub-factors demonstrating satisfactory goodness-of-fit indices (e.g., Comparative Fit Index [CFI] ≥ .90 to .95, Root Mean Square Error of Approximation [RMSEA] ≤ .06 to .08, and Standardized Root Mean Square Residual [SRMR] ≤ .08).

Convergent and Discriminant Validity

Construct validity was further substantiated through the examination of bivariate latent and manifest inter-scale correlations across the ten instruments:

• Discriminant Evidence: Meaningful divergence was demonstrated between distinct operational and cognitive dimensions. For instance, the correlation between Selection Processes and Teacher Beliefs & Knowledge was modest at r = .32, confirming that individual teacher pedagogical beliefs are empirically distinct from formal district procurement mechanisms.
• Convergent Evidence: Substantial convergent associations emerged where theoretically predicted. Most notably, a strong correlation of r = .82 was observed between Vision for Teaching & Learning and Strategic Leadership Support, reflecting the theoretical principle that instructional vision is intrinsically tied to executive administrative leadership behaviors and resource direction. Across all ten instruments, correlations demonstrated sufficient divergence to justify their use as separate, standalone diagnostic measures, while exhibiting the shared variance expected within a coherent organizational ecology.

Reliability

The internal consistency reliability of the EdTech Context Inventory was comprehensively evaluated across all ten instruments and their underlying factor dimensions. Because traditional metrics such as Cronbach’s alpha (α) can be biased when the assumption of tau-equivalence is violated, the research team also calculated McDonald’s omega (ω), which provides a more accurate estimate of composite reliability within congeneric factor models.

Empirical analyses confirmed excellent internal consistency across the entire suite:

• All ten primary instruments exhibited Cronbach’s alpha coefficients strictly exceeding α = .80, with many scales surpassing α = .90.
• McDonald’s omega coefficients similarly demonstrated high composite reliability across all scales (ω > .80), indicating minimal measurement error in the composite scores.
• Sub-factors within each instrument demonstrated robust internal consistency, generally ranging between .75 and .92, supporting the reliable profiling of granular sub-constructs (such as technical support responsiveness within Infrastructure, or collective risk-taking within Staff Culture).

These reliability parameters confirm that the inventory instruments possess the measurement precision required not only for aggregate group-level research comparisons, but also for diagnostic institutional profiling at the school and district levels.

Factor Analysis

The factor structure of the EdTech Context Inventory was established using a rigorous two-step structural equation modeling strategy incorporating both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) across two separate educator validation samples.

Exploratory Factor Analysis (Sample 1)

EFAs were conducted independently for each of the ten instrument item pools. Factor extraction was guided by eigenvalues greater than 1.0, scree plot examination, and parallel analysis, utilizing oblique rotations (e.g., Promax or Geomin) to allow for realistic correlations among underlying latent dimensions. The EFAs revealed that items generally loaded cleanly onto their theorized sub-factors, with primary factor pattern loadings predominantly exceeding .50 and minimal cross-loadings (rarely exceeding .25 on secondary factors). The empirical EFA findings prompted minor modifications to original working-group hypotheses, resulting in the trimming of psychometrically redundant items and slight reorganizations of specific factor assignments to optimize construct purity.

Confirmatory Factor Analysis (Sample 2)

The refined factor models were formally evaluated using CFA within a Structural Equation Modeling (SEM) framework on the second validation sample. CFAs confirmed that each of the ten instruments possesses a robust multi-factorial structure consisting of between three and six distinct factors per instrument:

• Item Count per Scale: The final validated instruments contain between 8 and 24 items each, culminating in an aggregate inventory pool of 188 items.
• Model Fit Indices: The multi-factor CFA solutions yielded strong fit to the observed data across instruments. Standardized root mean square residuals (SRMR) consistently met or fell below the .08 threshold, Root Mean Square Error of Approximation (RMSEA) values generally remained below .06 to .08 with tight confidence intervals, and Comparative Fit Indices (CFI) and Tucker-Lewis Indices (TLI) systematically surpassed the acceptable .90 to .95 benchmarks.
• Factor Loadings: Standardized factor loadings in the final CFA models were uniformly high and statistically significant (p < .001), with the vast majority ranging between .60 and .90, confirming that the latent constructs explain substantial variance in the individual observable indicators.

Instrument / Measurement Tool

• Test Type: Standardized Organizational and Psychological Diagnostic Inventory (Suite of 10 standalone survey instruments).
• Construct Measured: Contextual features of educational technology implementation across technical, human, cultural, and administrative domains.
• Target Population: Human; educators, instructional coaches, school principals, technology coordinators, and district administrators in primary and secondary (K-12) education.
• Age Group: Adults (18 years of age and older).
• Administration Format: Electronic questionnaire administration (computer-assisted self-interviewing via secure web survey platforms).
• Modularity: The ten instruments can be deployed collectively as a complete district diagnostic battery (188 items) or utilized completely independently depending on specific research or administrative evaluation priorities.
• Length: 188 items total across the 10 instruments; individual instruments contain between 8 and 24 items.
• Response Formats:
  • For nine of the ten instruments, items are recorded on a 5-point Likert scale. Depending on the instrument construct, response anchors utilize either:
    • Agreement Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree nor Disagree, 4 = Agree, 5 = Strongly Agree.
    • Frequency Scale: 1 = Never, 2 = Rarely / Once in a while, 3 = Sometimes, 4 = Fairly Often, 5 = Always.
• Scope of Assessment: Macro- and micro-environmental context; instruments assess the broader organizational and instructional context in which digital tools are adopted, rather than evaluating user satisfaction with a single software application.
• Scoring: Instrument scores and sub-factor scores are calculated as mean or sum composite indices. Higher scores denote more supportive implementation conditions (or, in the case of Competing Priorities, higher perceived friction/barriers unless reverse scored).

Permissions & Fee and Test Year

• Year of Publication: 2023.
• Copyright & Intellectual Property: Emily A. Kohler, Lauren Molloy Elreda, and Kate Tindle / University of Virginia and the EdTech Evidence Exchange.
• Usage Permissions: The EdTech Context Inventory is available for research and non-commercial educational practice. Interested researchers, district leaders, and evaluators wishing to administer the instruments or obtain the complete item specifications should contact the corresponding author, Dr. Emily A. Kohler, at [email protected].
• Fee: There is no fee required to use the instrument suite for non-commercial research, academic inquiry, or internal school district evaluation.
• Commercial Use: The inventory is not authorized for commercial sale, re-licensing, or integration into proprietary commercial diagnostic software without explicit written permission from the copyright holders.

References

• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman and Company.
• Damschroder, L. J., Aron, D. C., Rosalind, E., Kirsh, S. R., & Bradley, E. H. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(50), 1–15. https://doi.org/10.1186/1748-5908-4-50
• Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/S15327965PLI1104_01
• Kohler, E. A., Elreda, L. M., & Tindle, K. (2023). EdTech context inventory: Factor analyses for ten instruments to measure edtech implementation context features. Computers & Education, 195, 104709. https://doi.org/10.1016/j.compedu.2022.104709
• Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
• Rogers, E. M. (2003). Diffusion of innovations (5th ed.). Free Press.
• Schneider, B., Ehrhart, M. G., & Macey, W. H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388. https://doi.org/10.1146/annurev-psych-113011-143809

Items of the Scale