Topic Emotions Survey

Abstract

The Topic Emotions Survey (TES) is a psychometric self-report instrument engineered to assess discrete, topic-specific affective reactions experienced by learners when engaging with academic subjects, challenging coursework, or controversial conceptual domains. Developed originally within educational psychology and science learning research by Suzanne H. Broughton, Gale M. Sinatra, and E. Michael Nussbaum (2012, 2013), the instrument adapts the foundational tenets of Reinhard Pekrun’s Control-Value Theory of Achievement Emotions to the granular level of specific instructional topics and socioscientific controversies. The TES comprises 16 discrete emotion items evaluating both positive emotions (e.g., Hopeful, Interested, Enjoyment, Confident) and negative emotions (e.g., Angry, Anxious, Confused, Frustrated, Hopeless). Respondents evaluate each emotion using a 5-point Likert-type intensity scale ranging from 1 (Not at all) to 5 (Very strong) relative to a specific instructional referent, such as a contentious scientific subject (e.g., biological evolution, genetically modified organisms, or planetary reclassification) or a high-stakes academic course. Psychometric evaluations across elementary, secondary, and undergraduate samples demonstrate robust structural validity, showing either a clean two-dimensional structure (Positive Topic Emotions vs. Negative Topic Emotions) or nuanced epistemic-achievement emotion subfactors. The scale demonstrates high internal consistency, with Cronbach’s alpha coefficients regularly exceeding .85 for positive valence and .82 for negative valence composites. By bridging the gap between broad trait affect and micro-level cognitive processing, the TES serves as a crucial diagnostic and research instrument for conceptual change, epistemic cognition, and academic motivation.

Keywords

Topic Emotions Survey, academic emotions, control-value theory, conceptual change, science education, epistemic affect, positive emotions, negative emotions, learning psychology, psychometrics

Authors

The conceptualization and initial empirical validation of the Topic Emotions Survey emerged from the collaborative scholarship of educational psychologists specializing in cognitive processing, motivated reasoning, and conceptual change:

• Suzanne H. Broughton, Ph.D. — Department of Teacher Education and Leadership, Utah State University, Logan, UT, USA. Dr. Broughton’s research focuses on epistemic beliefs, conceptual change, and the affective mechanisms governing elementary and adolescent science literacy.
• Gale M. Sinatra, Ph.D. — Stephen H. Crocker Professor of Education and Psychology, Rossier School of Education, University of Southern California, Los Angeles, CA, USA. An internationally recognized leader in cognitive psychology and educational science, Dr. Sinatra investigates the intersection of motivation, epistemic emotions, and public understanding of science.
• E. Michael Nussbaum, Ph.D. — Department of Educational Psychology and Higher Education, University of Nevada, Las Vegas, Las Vegas, NV, USA. Dr. Nussbaum specializes in argumentation, critical thinking, epistemic cognition, and computer-supported collaborative learning.
• Subsequent Contributors & Adaptors: Notable extensions of the instrument to socioscientific contexts (e.g., genetically modified foods) and collegiate achievement contexts have been spearheaded by Benjamin C. Heddy (University of Oklahoma), Robert W. Danielson (Washington State University), Jesse Graham (University of Utah), and Tracy Anne Farkas (University of Oklahoma).

Purpose

The primary purpose of the Topic Emotions Survey is to measure the precise emotional profile elicited when an individual encounters, processes, or reflects upon a specific domain, controversial concept, or challenging academic task. For decades, educational researchers treated academic affect primarily through the broad lens of general test anxiety, general trait neuroticism, or subject-level attitudes (e.g., “attitude toward science”). However, contemporary cognitive-affective theories demonstrate that cognitive restructuring and deep learning are heavily mediated by localized, moment-to-moment affective states tied directly to the subject matter itself.

The TES was originally created to investigate elementary school students’ affective and conceptual responses to the 2006 decision by the International Astronomical Union to reclassify Pluto from a major planet to a dwarf planet (Broughton, Sinatra, & Nussbaum, 2012, 2013). The reclassification generated widespread emotional dissonance, intellectual resistance, and public controversy. Broughton and colleagues recognized that standard achievement emotion inventories—which predominantly focus on classroom settings, test-taking, or general studying habits—were structurally incapable of isolating the specific emotional valence directed toward a scientific concept under revision.

Beyond its initial application to astronomical conceptual change, the TES has been deployed widely across educational and psychological research paradigms:

• Socio-Scientific Controversy Processing: Investigating emotional reactions toward controversial topics such as climate change, vaccination, and genetically modified organisms (GMOs; Heddy et al., 2016).
• Conceptual Change and Knowledge Revision: Evaluating how epistemic friction (e.g., confusion, surprise, frustration) interacts with refutation text interventions to promote or impede schema restructuring.
• Higher Education Course-Level Strain: Measuring affective responses in undergraduate and graduate populations navigating gatekeeper courses, high-difficulty degree requirements, and academic perfectionism (Farkas, 2014).
• Instructional Design Evaluation: Providing educators with formative data on whether curricular materials induce constructive activating emotions (e.g., curiosity, interest, hope) or counterproductive deactivating emotions (e.g., hopelessness, shame, chronic confusion).

Psychological Construct

The psychological construct evaluated by the Topic Emotions Survey is topic-specific emotion—defined as the subjective, multi-component psychological state (comprising experiential, physiological, and cognitive processes) elicited by the mental representation of a distinct subject matter, instructional domain, or academic challenge. Unlike broad mood states that lack a dedicated intentional object, topic emotions are strictly object-directed. They reflect how the semantic and conceptual content of a discipline interacts with an individual’s personal identity, prior knowledge, epistemic values, and self-efficacy beliefs.

The 16 items of the TES operationalize affect across two fundamental axes derived from circumplex models of affect and Pekrun’s taxonomies: emotional valence (positive versus negative) and physiological/cognitive activation (activating versus deactivating), while also incorporating crucial epistemic emotions:

1. Positive Topic Emotions

This dimension encompasses feelings that reflect positive value appraisal, psychological safety, and intrinsic motivation toward the topic:

• Activating Positive Emotions: Excited, Interested, Happy, Proud. These states mobilize attentional resources, promote creative cognitive elaboration, facilitate exploratory questioning, and encourage sustained task engagement. For example, high topic interest drives students to seek supplementary literature and tolerate instructional ambiguity.
• Deactivating / Stabilizing Positive Emotions: Hopeful, Satisfaction, Confident, Enjoyment. These emotions reflect successful cognitive resolution, high outcome expectations, and master-oriented appraisals. Confidence and satisfaction consolidate newly restructured schemas and validate intellectual self-efficacy.

2. Negative Topic Emotions

This dimension captures states of cognitive dissonance, threat appraisal, perceived inadequacy, or epistemic impasse:

• Activating Negative Emotions: Angry, Anxious, Confused, Frustrated. In learning environments, activating negative emotions play a dual role. While anger and frustration may trigger defensive motivated reasoning or rejection of anomalous data, moderate levels of cognitive confusion and anxiety can serve as epistemic catalysts, alerting the learner to internal contradictions and motivating cognitive reappraisal.
• Deactivating Negative Emotions: Hopeless, Shame, Sad, Disappointed. These debilitating states are characterized by low perceived control and negative value. When learners experience hopelessness or shame regarding a difficult course or counter-intuitive concept, cognitive engagement collapses, resulting in behavioral avoidance, academic disengagement, and superficial rote processing.

Theoretical Framework

The conceptual foundation of the Topic Emotions Survey is anchored at the intersection of three major theoretical paradigms within contemporary cognitive and educational psychology:

1. The Control-Value Theory of Achievement Emotions

Articulated by Reinhard Pekrun (2006), the Control-Value Theory posits that academic emotions are direct functions of two proximal cognitive appraisals: control appraisals (perceived subjective control over learning activities and outcomes, closely related to self-efficacy and outcome expectations) and value appraisals (the subjective importance, utility, or intrinsic interest assigned to the task or topic). When applied to specific instructional content, the TES measures the affective consequences of these appraisals. For instance, high subjective value combined with low perceived control generates anxiety and hopelessness; conversely, high perceived control combined with high intrinsic value yields enjoyment, hope, and excitement.

2. The Cognitive-Affective Model of Conceptual Change (CAMCC)

Developed by Michelle Gregoire (2003), the CAMCC illustrates how individuals process message content that contradicts their prior beliefs. When presented with anomalous scientific information, learners engage in rapid, pre-conscious appraisals of personal relevance, threat, and cognitive control. If the new information threatens established personal beliefs or cognitive identity, it triggers negative emotional arousal (e.g., anger, anxiety, confusion). These affective responses determine whether the individual pursues systematic cognitive processing—resulting in genuine conceptual change—or defensive processing characterized by dismissive cognitive heuristics and superficial assimilation.

3. Epistemic Emotions and Epistemic Cognition

Recent advances by Pekrun, Vogl, Muis, and Sinatra emphasize that learning involves emotions whose intentional object is the knowledge-generating qualities of cognitive tasks. Items such as Confused, Interested, and Frustrated represent prototypical epistemic emotions. In conceptual change scenarios, encountering anomalous evidence disrupts schema equilibrium, eliciting cognitive confusion. If learners possess sufficient epistemic self-efficacy, this confusion triggers epistemic curiosity; if control is lacking, confusion degenerates into frustration and cognitive resignation.

Validity

Empirical investigations across diverse educational and developmental cohorts have established substantial psychometric validity for the Topic Emotions Survey.

Construct and Structural Validity

Construct validity has been confirmed through both exploratory and confirmatory factor analytic procedures. Broughton et al. (2012, 2013) examined the scale among elementary students navigating planetary science revisions. Despite the developmental age of the participants, principal components analysis and factor analysis yielded clear separation of positive and negative valence factors, with items loading heavily onto their designated theoretical constructs without significant cross-loadings.

Convergent and Discriminant Validity

The TES demonstrates strong convergent validity with validated instruments evaluating general affect and motivation:

• Positive topic emotion scores correlate positively and significantly with the Positive Affect subscale of the Positive and Negative Affect Schedule (PANAS) (typically r = .55 to .71), mastery achievement goals, and academic self-efficacy scales.
• Negative topic emotion scores correlate moderately to strongly with the PANAS Negative Affect scale (r = .48 to .65) and established subscales of the Achievement Emotions Questionnaire (AEQ), including class-related boredom and anxiety.
• Discriminant validity is supported by low correlations with unrelated cognitive constructs (e.g., working memory capacity, baseline spatial ability) and distinct divergence from general trait affect, confirming that the TES isolates domain-specific and topic-specific appraisals rather than chronic emotional predispositions.

Predictive and Criterion-Related Validity

The predictive validity of the TES is well-documented in experimental and quasi-experimental conceptual change research:

• Heddy, Danielson, Sinatra, and Graham (2016) demonstrated that topic emotions measured prior to and during refutation text interventions regarding genetically modified organisms significantly predicted post-intervention conceptual knowledge change and attitude adjustment. Negative activating emotions (e.g., frustration, confusion) accounted for significant incremental variance in conceptual restructuring beyond prior knowledge and cognitive ability.
• In higher education research, Farkas (2014) established that collegiate negative topic emotions directed toward degree-challenging coursework predicted lower GPA, maladaptive perfectionistic avoidance, and heightened psychological burnout.

Reliability

The internal consistency and temporal stability of the Topic Emotions Survey have been rigorously documented across multiple independent empirical samples:

Internal Consistency

Across validation studies, the composite subscales demonstrate exceptional internal consistency reliability, typically well above the psychometric threshold of .80:

• Positive Topic Emotions Composite (8 items: Hopeful, Happy, Interested, Enjoyment, Satisfaction, Proud, Excited, Confident): In the initial Broughton et al. (2013) study, Cronbach’s alpha was documented at α = .88. Heddy et al. (2016) reported α = .91 in an experimental science intervention, while Farkas (2014) obtained α = .89 in a collegiate achievement sample.
• Negative Topic Emotions Composite (8 items: Angry, Anxious, Confused, Frustrated, Hopeless, Shame, Sad, Disappointed): Broughton et al. (2013) reported Cronbach’s α = .84. Heddy et al. (2016) observed α = .86, and Farkas (2014) reported α = .87 for undergraduate students reflecting on their most challenging academic course.

Test-Retest Reliability

Because topic emotions are designed to capture dynamic psychological states sensitive to instruction and knowledge revision, test-retest correlations fluctuate intentionally across learning interventions. However, short-term stability analyses (over 48- to 72-hour intervals without intervening instruction) show strong temporal consistency (r ranging from .76 to .83), indicating that the scale reliably captures coherent affective orientations while remaining sensitive to genuine conceptual change.

Factor Analysis

Extensive psychometric investigations employing both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) substantiate the theoretical architecture of the Topic Emotions Survey.

Exploratory Factor Structure

Initial factor analytic investigations conducted by Broughton, Sinatra, and Nussbaum (2012, 2013) using principal axis factoring with Promax (oblique) rotation cleanly extracted a primary two-factor solution explaining over 58% of the total variance:

• Factor 1: Positive Valence / Epistemic Approach (eigenvalues typically > 4.5), with primary item loadings ranging from .62 to .86 for Enjoyment, Satisfaction, Interested, Happy, Excited, Proud, and Confident.
• Factor 2: Negative Valence / Epistemic Dissonance (eigenvalues typically > 2.8), with primary item loadings ranging from .54 to .83 for Frustrated, Angry, Anxious, Hopeless, Sad, Disappointed, and Shame.
• Confused occasionally demonstrated minor cross-loading across studies, reflecting its unique status as a neutral-to-negative epistemic transition emotion that frequently accompanies active cognitive restructuring.

Confirmatory Factor Analysis (CFA) and Model Fit

Structural evaluations using maximum likelihood estimation (Heddy et al., 2016; Farkas, 2014) compared single-factor, orthogonal two-factor, correlated two-factor, and four-factor (valence × activation) models. The correlated two-factor model demonstrated superior fit across diverse educational samples, satisfying conventional psychometric criteria:

• Comparative Fit Index (CFI): .932 to .958 (exceeding the ≥ .90 threshold)
• Tucker-Lewis Index (TLI): .921 to .947
• Root Mean Square Error of Approximation (RMSEA): .051 to .064 (with 90% confidence intervals within acceptable ranges)
• Standardized Root Mean Square Residual (SRMR): .042 to .058

These findings substantiate that treating the TES as two overarching dimensions—Positive Topic Emotions and Negative Topic Emotions—is statistically robust, while permitting researchers to isolate specific discrete emotions (e.g., confusion, interest) for micro-level cognitive analyses.

Instrument / Measurement Tool

• Test Type: Psychometric self-report rating inventory measuring discrete, topic-level affective states.
• Format: Paper-and-pencil or computer-administered questionnaire. Can be administered as a pre-test, post-test, or repeated-measures state evaluation.
• Number of Items: 16 discrete emotion items (adjectives or short nouns).
• Target Population: Elementary school students (grades 4–6 with reading assistance if required), secondary school students, undergraduate learners, and general adult populations.
• Administration Time: Approximately 3 to 5 minutes.
• Response Scale: 5-point Likert-type intensity scale:
  • 1 = Not at all
  • 2 = Very little
  • 3 = Moderate
  • 4 = Strong
  • 5 = Very strong
• Scoring System:
  • Positive Topic Emotions Subscale: Mean score of items 2, 4, 6, 9, 10, 13, 14, 16 (Hopeful, Happy, Interested, Enjoyment, Satisfaction, Proud, Excited, Confident). Scores range from 1.0 to 5.0.
  • Negative Topic Emotions Subscale: Mean score of items 1, 3, 5, 7, 8, 11, 12, 15 (Angry, Anxious, Confused, Frustrated, Hopeless, Shame, Sad, Disappointed). Scores range from 1.0 to 5.0.
  • Discrete Emotion Profiling: Researchers analyzing specific cognitive phenomena frequently isolate individual item scores (e.g., Confused as an index of epistemic dissonance; Interested as an index of epistemic curiosity).
  • Adaptability of Prompt: The frame of reference is customizable to the research context. Standard prompts include: “When I think about [target topic, e.g., Pluto’s reclassification / Genetically Modified Organisms / Climate Change], I feel:” or “When I think about the most difficult class in my degree program, I feel:”

Permissions & Fee and Test Year

The Topic Emotions Survey was introduced in peer-reviewed scientific literature between 2012 and 2013 through the published work of Suzanne H. Broughton, Gale M. Sinatra, and E. Michael Nussbaum. As an assessment developed through publicly funded university academic research, the TES is an open-access, fee-free psychometric instrument. Researchers, educational psychologists, and classroom instructors are granted permission to adapt and utilize the scale for non-commercial educational and empirical research endeavors, provided appropriate academic citation and attribution are given to the original developers. Commercial applications, test banking, or inclusion in proprietary software packages require formal written permission from the copyright holders and publishing bodies.

References

• Broughton, S. H., Sinatra, G. M., & Nussbaum, E. M. (2012). “Pluto has been a planet my whole life!” Emotions, attitudes, and conceptual change in elementary students learning about Pluto’s reclassification. Teacher Education and Leadership Faculty Publications, Paper 288. https://digitalcommons.usu.edu/teal_facpub/288
• Broughton, S. H., Sinatra, G. M., & Nussbaum, E. M. (2013). “Pluto has been a planet my whole life!” Emotions, attitudes, and conceptual change in elementary students learning about Pluto’s reclassification. Research in Science Education, 43(2), 529–550. https://doi.org/10.1007/s11165-011-9274-x
• Farkas, T. A. (2014). Perfectionism: The relationship to well-being and achievement motivation in college learners (Doctoral dissertation, University of Oklahoma). ShareOK Repository. https://shareok.org/bitstream/handle/11244/14056/2014_Farkas_Tracy_Dissertation.docx?sequence=1
• Gregoire, M. (2003). Is it a challenge or a threat? A dual-process model of teachers’ cognition and appraisals during conceptual change. Educational Psychology Review, 15(2), 147–179. https://doi.org/10.1023/A:1023477111286
• Heddy, B. C., Danielson, R. W., Sinatra, G. M., & Graham, J. (2016). Modifying knowledge, emotions, and attitudes regarding genetically modified foods. The Journal of Experimental Education, 85(3), 513–533. https://doi.org/10.1080/00220973.2016.1260523
• Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9
• Sinatra, G. M., Broughton, S. H., & Lombardi, D. (2014). Emotions in science education. In R. Pekrun & L. M. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 415–436). New York, NY: Routledge.

13. Items of the Scale (Questionnaire)