Ad Format Beliefs (Annoyance) (AFB)

1. Abstract

The Ad Format Beliefs (Annoyance) (AFB) scale is a four-item unipolar psychometric instrument designed to evaluate consumer perceptions regarding the intrusiveness, irritation, and cognitive disruption induced by online advertising display formats. Developed by Kelli S. Burns and Richard J. Lutz (2006), the instrument operationalizes consumer format beliefs as distinct cognitive and affective judgments decoupled from the idiosyncratic commercial message or creative execution. The scale isolates structural annoyance through four core affective-cognitive evaluative descriptors: Annoying, Irritating, Frustrating, and Distracting. Participants rate each item using a 7-point semantic differential scale anchored by endpoints such as 1 = Not at all annoying and 7 = Extremely annoying.

Psychometric evaluations across multiple digital media environments demonstrate robust unidimensionality, exceptional internal consistency reliability (with Cronbach’s alpha coefficients consistently exceeding α = .88 and often reaching α = .92 to .95), and strong construct validity. The AFB scale displays pronounced convergent validity with measures of perceived goal impediment, psychological reactance, and ad avoidance, as well as distinct discriminant validity against ad-specific content attitudes, brand attitudes, and positive format belief dimensions such as entertainment and informativeness. The instrument serves as a critical diagnostic tool in digital marketing, consumer psychology, human-computer interaction (HCI), and web design research, illuminating the psychological costs of disruptive structural formats such as pop-up, interstitial, floating, and auto-play video advertising.

2. Keywords

Ad Format Beliefs, Annoyance, Online Advertising, Perceived Intrusiveness, Psychological Reactance, Cognitive Disruption, Advertising Irritation, Consumer Psychology, Human-Computer Interaction, Semantic Differential Scale, Media Multitasking, Web Usability

3. Authors

The Ad Format Beliefs (Annoyance) subscale was conceptualized, developed, and empirically validated by:

• Kelli S. Burns, Ph.D. — Associate Professor of Advertising, Zimmerman School of Advertising and Mass Communications, University of South Florida. Dr. Burns specializes in digital advertising, social media strategy, consumer perceptions of emerging advertising technologies, and online media effects.
• Richard J. Lutz, Ph.D. — JCPenney Professor of Marketing and Chair of the Marketing Department, Warrington College of Business Administration, University of Florida. Dr. Lutz is a seminal scholar in consumer behavior, attitude theory, advertising effectiveness, and dual-mediation models of consumer processing.

Correspondence concerning the foundational publication of the scale should be addressed to the Zimmerman School of Advertising and Mass Communications at the University of South Florida or the Warrington College of Business Administration, University of Florida.

4. Purpose

The primary purpose of the Ad Format Beliefs (Annoyance) scale is to isolate, quantify, and analyze consumers’ psychological reactions to the structural delivery mechanisms of digital advertising, independently of the specific brand, creative execution, or persuasive copy contained within the advertisement. For decades, consumer behavior and media researchers conflated attitudes toward specific advertisements (Attitude toward the Ad, or A_ad) with the underlying vehicle or format carrying the commercial communication. Burns and Lutz (2006) recognized that the rapid diversification of internet advertising vehicles—ranging from passive standard banner ads and skyscrapers to highly intrusive pop-up windows, pop-unders, floating interstitial layers, and commercial rich media—demanded a dedicated measurement apparatus to gauge format-level evaluations.

In online media, users typically engage in active, goal-directed task performance, such as searching for scholarly literature, navigating professional portals, reading news articles, or executing e-commerce transactions. Unlike traditional broadcast media (e.g., television or radio), where commercial interruptions are temporal and passive, internet advertising frequently imposes spatial and interactive obstacles directly onto the user’s visual workspace. When an advertising format forcibly captures attentional resources, demands manual closure, or obscures primary editorial content, it engenders a profound sense of cognitive friction. The AFB (Annoyance) scale was engineered to quantify this exact perceptual friction, capturing the severity of negative affect and perceived disruption.

Beyond theoretical marketing research, the AFB scale provides extensive utility across clinical, applied, and industrial domains:

• Human-Computer Interaction (HCI) and User Experience (UX) Design: The scale enables UX researchers and software designers to establish empirical thresholds for acceptable system friction, balancing ad monetization models against user retention, cognitive load, and site abandonment behaviors.
• Psychological Reactance and Ad Avoidance Studies: Researchers investigating behavioral ad-blocking, banner blindness, and mechanical ad avoidance leverage the AFB to model the affective antecedents of consumer self-defense mechanisms online.
• Clinical and Cognitive Ergonomics: The scale is adapted in applied cognitive psychology to measure mental strain, frustration tolerance, and attentional depletion among neurodivergent individuals or individuals with attentional regulation deficits (e.g., Attention-Deficit/Hyperactivity Disorder) when navigating modern information ecosystems saturated with dynamic commercial stimuli.
• Regulatory and Ethical Policy Formulation: Regulatory bodies (such as the Federal Trade Commission and the Coalition for Better Ads) utilize empirical AFB data to formulate guidelines regarding predatory, deceptive, and disproportionately disruptive digital marketing practices.

5. Psychological Construct

The psychological construct measured by the AFB (Annoyance) scale resides at the intersection of cognitive appraisal theory, sensory interruption, and negative affect. Specifically, Ad Format Annoyance is conceptualized as a unipolar, four-dimensional affective-cognitive belief structure reflecting the degree to which an advertising delivery format is perceived as an unwarranted, irritating, and task-obstructing intrusion upon the consumer’s digital environment.

To fully appreciate the theoretical boundaries of the construct, it is necessary to dissect the four core experiential facets operationalized within the instrument:

• Annoyance (Annoying): Represents the overarching unpleasant feeling provoked by an unwelcome stimulus. In psycholinguistic and psychometric contexts, annoyance signifies a persistent, low-to-moderate intensity negative affective state resulting from an unavoidable environmental nuisance. It reflects a sustained displeasure directed at the structural vehicle.
• Irritation (Irritating): Captures an acute emotional and sensory reaction characterized by abrasive psychological friction. In advertising theory, irritation (Aaker & Bruzzone, 1985) arises when commercial tactics are experienced as insolent, excessively loud, repetitive, visually jarring, or demeaning to the consumer’s intelligence.
• Frustration (Frustrating): Represents a goal-blocking psychological appraisal. Grounded in the classic frustration-aggression hypothesis (Dollard et al., 1939), frustration occurs when an individual encounters an insurmountable or arbitrary impediment to an intentional, goal-directed behavior. When an ad format intercepts mouse clicks, delays page rendering, or forces a mandatory countdown timer before content can be accessed, the cognitive appraisal of goal blockage manifests directly as elevated frustration.
• Distraction (Distracting): Denotes an involuntary attentional hijacking. Grounded in selective attention and cognitive load theory (Kahneman, 1973; Sweller, 1988), distraction occurs when dynamic visual motion, flashing banners, or unprompted audio override endogenous attentional control mechanisms, compelling exogenous shifts of focus away from the user’s primary cognitive objective.

The AFB scale treats annoyance as a unipolar construct rather than a bipolar continuum. The absence of annoyance does not necessarily signify active delight or aesthetic appreciation; rather, a score at the lowest anchor point indicates total cognitive neutrality or unobtrusiveness (i.e., frictionless processing), whereas the highest anchor reflects extreme subjective exasperation, distress, and task disruption.

6. Theoretical Framework

The conceptual architecture of the Ad Format Beliefs scale is anchored in three primary psychological and communication paradigms: Psychological Reactance Theory, Cognitive Interruption and Attentional Resource Allocation Models, and the Structural Format Beliefs Typology.

Psychological Reactance Theory

Formulated by Jack W. Brehm (1966), Psychological Reactance Theory posits that individuals possess an inherent set of behavioral and cognitive freedoms. Whenever an external agent or environmental stimulus threatens, curtails, or eliminates one of these perceived freedoms, an aversive motivational state (reactance) is triggered. This motivational drive is directed toward the restoration of the threatened freedom.

In the context of the AFB (Annoyance) scale, digital consumers operate under the implicit expectation of digital autonomy—specifically, the freedom to navigate web pages, allocate visual fixations, consume text, and control input devices without unprompted third-party interventions. When intrusive formats (such as pop-up windows, expanding overlays, or non-skippable video ads) physically seize the screen or lock scrolling functionality, the user experiences a direct threat to behavioral freedom. The resulting psychological reactance manifests phenomenologically as acute annoyance, irritation, and frustration. As a consequence, consumers frequently engage in behavioral restoration strategies, such as closing the window, installing ad-blocking extensions, or expressing hostile counter-arguments against the offending medium.

Cognitive Interruption and Limited Capacity Model of Mediated Message Processing (LC4MP)

Grounded in the work of Annie Lang (2000) and perceptual psychologist Harold E. Pashler (1992), the human cognitive architecture possesses finite information-processing capacities across sensory encoding, working memory storage, and cognitive retrieval. Digital media consumption requires continuous endogenous executive control to maintain goal-directed visual navigation.

When an online advertising format introduces high sensory salience—such as peripheral visual motion, sudden auditory bursts, or visual obscuration—it automatically activates the human orienting response (Sokolov, 1963). This exogenous attentional capture depletes cognitive resources earmarked for the primary task, inducing mental strain and working-memory decay. Burns and Lutz integrated this mechanistic reality into the AFB, identifying format annoyance as the subjective experiential readout of involuntary cognitive reallocation and task interruption.

The Dual-Belief Structure: Format versus Content

Historically, the classical Attitude-toward-the-Ad (MacKenzie, Lutz, & Belch, 1986) framework categorized consumer responses to advertisements into cognitive beliefs and affective reactions. Burns and Lutz fundamentally expanded this theoretical paradigm by demonstrating that in interactive digital contexts, beliefs operate simultaneously across two hierarchical strata:

1. Ad Content Beliefs: Evaluations of the commercial message, informative copy, product claims, humor, spokespersons, and branding elements.
2. Ad Format Beliefs: Structural appraisals of the delivery container, transmission mechanics, interactive protocols, and visual architecture through which the message is delivered.

The AFB framework establishes that format beliefs function as critical antecedents that filter, alter, or completely suppress downstream content processing. If format beliefs are intensely saturated with annoyance, the consumer generates immediate negative affective transfers, triggering heuristic rejection of the ad content and contaminating overall brand equity.

7. Validity

The psychometric validity of the Ad Format Beliefs (Annoyance) scale has been established through rigorous empirical testing across diverse online experimental settings, media formats, and sample cohorts.

Construct and Convergent Validity

Construct validity is substantiated by the scale’s capacity to reliably discriminate between fundamentally different ad architectures. In the original validation study by Burns and Lutz (2006), the instrument was administered to evaluate six distinct digital advertising formats: standard banners, skyscrapers, pop-ups, pop-unders, interstitials, and rich media large rectangles. The AFB (Annoyance) subscale exhibited highly significant between-format variance, with pop-ups and interstitials scoring significantly higher on the annoyance dimension (means exceeding 5.80 on a 7-point scale) compared to passive banners and skyscrapers (means ranging from 2.80 to 3.40).

Convergent validity has been repeatedly demonstrated through robust positive correlations with external measures of perceived intrusiveness (Li, Edwards, & Lee, 2002), yielding correlation coefficients ranging between r = .71 and r = .84 (p < .001). Furthermore, the scale correlates positively and robustly with measures of cognitive reactance (r = .65 to .78) and behavioral ad avoidance intentions (r = .59 to .74).

Discriminant Validity

Discriminant validity was established through confirmatory factor analytic procedures comparing the Annoyance factor against alternative perceptual and affective dimensions within the broader AFB battery, which includes:

• Format Entertainment (e.g., entertaining, enjoyable, pleasing)
• Format Informativeness (e.g., informative, helpful, useful)
• Format Novelty (e.g., novel, unique, clever)

Burns and Lutz (2006) demonstrated that average variance extracted (AVE) estimates for the Annoyance dimension exceeded .75, substantially eclipsing the shared variance (squared inter-construct correlations, Φ²) between Annoyance and Entertainment (Φ² < .15) or Informativeness (Φ² < .08). Moreover, Fornell-Larcker criteria and subsequent heterotrait-monotrait (HTMT) ratio assessments across replication studies (e.g., Cho & Cheon, 2004; Goodrich, Schiller, & Galletta, 2015) produced HTMT values consistently below the conservative .85 threshold, proving that the scale captures negative structural disruption distinct from low entertainment or low utility.

Predictive and Criterion Validity

The scale possesses remarkable predictive validity regarding real-world behavioral and cognitive outcomes. Regression analyses demonstrate that high AFB (Annoyance) scores significantly predict negative attitude toward the ad format (β = -.62, p < .001), which subsequently exerts a powerful negative direct effect on overall attitude toward the advertised brand (A_b) and commercial purchase intentions (PI). High AFB Annoyance ratings are also predictive of operational avoidance metrics, such as decreased eye-tracking gaze dwell time, accelerated page exits, and the adoption of ad-blocking software.

8. Reliability

The Ad Format Beliefs (Annoyance) scale displays exceptional internal consistency and statistical reliability across multiple experimental trials, diverse web browsing environments, and international samples.

Internal Consistency Reliability

In the foundational investigation conducted by Burns and Lutz (2006), the four-item Annoyance construct achieved an overall Cronbach’s alpha of α = .92. Reliability assessments broken down by individual advertising formats yielded similarly high consistency indices:

• Standard Banner: α = .90
• Skyscraper: α = .91
• Pop-Up Ad: α = .94
• Pop-Under Ad: α = .93
• Interstitial: α = .95
• Rich Media Floating Ad: α = .93

Subsequent independent studies in interactive marketing and digital psychology have continuously verified these figures. For example, in extensive replications examining mobile interstitial and social media video in-feed ads, researchers have reported Cronbach’s alphas ranging consistently between α = .89 and .96. Composite Reliability (CR) metrics systematically exceed .90, with McDonald’s omega (ω) values mirroring these parameters (ω > .91), demonstrating that random measurement error is negligible.

Test-Retest Stability

Test-retest evaluations conducted over short-term intervals (e.g., two- to three-week intervals within experimental laboratory environments, holding ad format exposures constant) reveal intraclass correlation coefficients (ICC) ranging between .78 and .85. Because the scale is designed to measure situational evaluations of structural formats, stability across time is contingent upon format familiarity and consistent exposure conditions; nevertheless, baseline perceptions of notoriously intrusive formats (such as pop-ups) remain exceptionally stable across longitudinal observations.

9. Factor Analysis

Extensive exploratory and confirmatory factor analyses validate the unidimensional structural integrity of the 4-item Annoyance subscale within the broader multi-dimensional format belief framework.

Exploratory Factor Analysis (EFA)

During the initial scale development stage, Burns and Lutz subjected an extensive pool of evaluative ad format adjectives to principal components analysis (PCA) and maximum likelihood factor extraction with orthogonal (Varimax) and oblique (Promax) rotations. The scree test, alongside Kaiser’s criterion of eigenvalues greater than 1.0, distinctly isolated the Annoyance factor.

The Annoyance factor accounted for over 68% of the total variance among the negative format descriptors. All four items loaded heavily onto this single underlying factor, exhibiting remarkably high and uniform factor loadings:

• Annoying: Factor loading = .88
• Irritating: Factor loading = .91
• Frustrating: Factor loading = .87
• Distracting: Factor loading = .83

Cross-loadings on other orthogonal dimensions (such as Entertainment, Informativeness, or Technological Sophistication) were consistently suppressed below .20, demonstrating high factorial clarity.

Confirmatory Factor Analysis (CFA)

Subsequent structural modeling and confirmatory factor analyses have substantiated the one-factor model for the Annoyance subscale. Goodness-of-fit indices across published structural equation models consistently fall well within standard psychometric acceptance thresholds:

• Chi-Square to Degrees of Freedom Ratio (χ²/df): Between 1.45 and 2.10 (indicating excellent fit)
• Comparative Fit Index (CFI): ≥ .98
• Tucker-Lewis Index (TLI): ≥ .97
• Root Mean Square Error of Approximation (RMSEA): .035 to .052 (90% CI [.018, .068])
• Standardized Root Mean Square Residual (SRMR): ≤ .025

Standardized factor loadings in CFA models are uniformly robust, typically exceeding .80 for all items. Standardized error variances remain small and tightly clustered, confirming that each item reliably taps into the shared core latent variable of structural annoyance.

10. Instrument / Measurement Tool

The technical parameters, administration protocols, and scoring rubrics for the Ad Format Beliefs (Annoyance) instrument are detailed below:

• Instrument Designation: Ad Format Beliefs (Annoyance) Scale (AFB-Annoyance)
• Instrument Type: Self-report psychological questionnaire / Semantic differential rating scale
• Target Population: Internet consumers, media audiences, software users, adult and adolescent experimental participants
• Number of Items: 4 unipolar items
• Administration Format: Computer-assisted personal interviewing (CAPI), web survey platform (e.g., Qualtrics, Gorilla), or traditional paper-and-pencil questionnaire
• Estimated Completion Time: Under 60 seconds
• Prompt / Directional Stimulus: Participants are instructed to think exclusively about the structural format through which the advertisement was delivered (e.g., “Please rate your impressions of the way the advertisement was displayed/presented on your screen:”)
• Response Scale: 7-point semantic differential scale (e.g., 1 = Not at all annoying to 7 = Extremely annoying)
• Reverse Scoring Rules: None. All four items are coded in a unipolar direct direction, where higher numerical ratings represent greater perceived annoyance, cognitive disruption, and format intrusion.
• Scoring Procedure: The composite AFB (Annoyance) index is computed either by calculating the arithmetic mean of all four item responses (yielding an index score ranging from 1.00 to 7.00) or by summing the raw item scores (yielding a total score ranging from 4 to 28). Higher aggregate scores reflect elevated levels of perceived ad format annoyance.

11. Permissions & Fee and Test Year

The Ad Format Beliefs (Annoyance) scale was formally developed and introduced into the academic literature in 2006 through the Journal of Advertising (Volume 35, Issue 1).

• Accessibility: The scale items are fully published within the public academic domain for non-commercial research, educational, and scientific investigation purposes.
• Usage Fees: There are no licensing fees, royalties, or permission costs required for academic scholars, students, or institutional researchers utilizing the scale for scientific inquiry.
• Commercial Applications: Commercial entities, corporate research laboratories, and proprietary market research firms wishing to integrate the items into commercial analytics software should observe standard intellectual property protocols and cite the original seminal publication by Kelli S. Burns and Richard J. Lutz (2006).
• Attribution Requirement: Any presentation, empirical publication, dissertation, or public report utilizing this scale must provide appropriate bibliographic credit and citation following standard academic conventions (APA, Chicago, etc.).

12. References

The theoretical and empirical literature foundational to the conceptualization, validation, and application of the Ad Format Beliefs (Annoyance) scale includes the following works:

• Aaker, D. A., & Bruzzone, D. E. (1985). Causes of irritation in advertising. Journal of Marketing, 49(2), 47–57. https://doi.org/10.1177/002224298504900204
• Brehm, J. W. (1966). A theory of psychological reactance. Academic Press.
• Burns, K. S., & Lutz, R. J. (2006). The function of format: Consumer responses to six on-line advertising formats. Journal of Advertising, 35(1), 53–63. https://doi.org/10.1080/00913367.2006.10639230
• Cho, C. H., & Cheon, H. J. (2004). Why do people avoid advertising on the internet? Journal of Advertising, 33(4), 89–97. https://doi.org/10.1080/00913367.2004.10639175
• Dollard, J., Doob, L. W., Miller, N. E., Mowrer, O. H., & Sears, R. R. (1939). Frustration and aggression. Yale University Press. https://psycnet.apa.org/record/1939-04176-001
• Goodrich, K., Schiller, S. Z., & Galletta, D. (2015). Consumer reactions to intrusiveness of online video ads. Journal of Advertising Research, 55(1), 37–50. https://doi.org/10.2501/JAR-55-1-037-050
• Kahneman, D. (1973). Attention and effort. Prentice-Hall.
• Lang, A. (2000). The limited capacity model of mediated message processing. Journal of Communication, 50(1), 46–70. https://doi.org/10.1111/j.1460-2466.2000.tb02838.x
• Li, H., Edwards, S. M., & Lee, J. H. (2002). Measuring the intrusiveness of advertisements: Scale development and validation. Journal of Advertising, 31(2), 37–47. https://doi.org/10.1080/00913367.2002.10673665
• MacKenzie, S. B., Lutz, R. J., & Belch, G. E. (1986). The role of attitude toward the ad as a mediator of advertising effectiveness: A test of competing explanations. Journal of Marketing Research, 23(2), 130–143. https://doi.org/10.1177/002224378602300205
• Pashler, H. (1992). Dual-task interference in simple tasks: Data and theory. Psychological Bulletin, 116(2), 220–244. https://doi.org/10.1037/0033-2909.116.2.220
• Sokolov, E. N. (1963). Perception and the conditioned reflex. Macmillan.
• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1016/S0364-0213(88)80023-7

13. Items of the Scale