Communication Training Impact Questionnaire

Abstract

The Communication Training Impact Questionnaire (CTIQ) is a psychometrically validated self-report instrument developed by Sue DeWine (1987) designed to evaluate the organizational transfer and behavioral manifestation of communication skills training within workplace contexts. Anchored theoretically in Donald Kirkpatrick’s four-level model of training evaluation—specifically operationalizing Level 3: Behavioral Application and On-the-Job Transfer—the CTIQ systematically assesses whether pedagogical gains acquired during professional development translate into sustained occupational competencies. Comprising 20 items scored along an authentic 5-point Likert-type response scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree), the instrument captures multidimensional workplace dynamics, including perceived instructional utility, environmental constraints, supervisory reinforcement, and post-training behavioral utilization. Psychometric evaluation across diverse occupational cohorts (total N = 267), including corporate middle managers, medical student physicians, financial professionals, and collegiate leadership cohorts, demonstrated robust internal consistency, yielding an overall Cronbach’s alpha coefficient of .88 (with subgroup reliabilities spanning .87 to .93). Exploratory factor analysis employing oblique rotation substantiated a distinct two-factor solution: Relationship of Training to Job and Skilled Performance. The CTIQ addresses a critical gap in industrial-organizational psychology and corporate communication by moving beyond mere affective participant satisfaction (Kirkpatrick Level 1) to quantitatively capture post-training behavioral outcomes and systemic workplace barriers to skill execution. This article presents an exhaustive academic review of the CTIQ’s theoretical architecture, structural validity, reliability parameters, factor structure, scoring mechanics, and applied utility across organizational research and human resource development.

Keywords

Communication Training Impact Questionnaire, CTIQ, Sue DeWine, Kirkpatrick Training Evaluation Model, Transfer of Training, Organizational Communication Competency, Behavioral Assessment, Industrial-Organizational Psychology, Workplace Learning, Psychometrics.

Authors

The Communication Training Impact Questionnaire was conceptualized, operationalized, and empirically validated by Sue DeWine, Ph.D.

• Author: Dr. Sue DeWine
• Institutional Affiliation at Validation: School of Interpersonal Communication, College of Communication, Ohio University, Athens, Ohio, United States.
• Academic Career & Leadership: Dr. DeWine has served as an eminent scholar in organizational communication, former President of Hanover College, and an active consultant in human resource development and corporate communication audits.
• Historical Collaborative Context: The instrument emerged alongside parallel investigations into communication assessment methodologies conducted with colleagues such as Judy C. Pearson and built upon empirical frameworks pioneered in occupational training by J. B. Cornwall and Donald Kirkpatrick.

Purpose

The overarching purpose of the Communication Training Impact Questionnaire (CTIQ) is to systematically quantify, diagnose, and interpret the occupational transfer of interpersonal, leadership, and operational communication competencies acquired during structured organizational training interventions. For decades, corporate human resource development programs were heavily critiqued for relying nearly exclusively on post-course “smile sheets”—cursory subjective reaction surveys administered immediately upon workshop completion that evaluate instructor charisma, room comfort, or general participant enjoyment (Kirkpatrick’s Level 1: Reaction). Such metrics consistently fail to demonstrate whether instructional interventions produce genuine behavioral changes, enhanced performance, or quantifiable organizational outcomes.

The CTIQ was engineered to resolve this diagnostic void by providing a rigorous empirical mechanism aligned specifically with Kirkpatrick’s Level 3: Behavior. Administered temporally distant from initial training—conventionally two to four weeks post-intervention—the CTIQ captures the critical period during which trained employees attempt to implement newly acquired communication strategies within dynamic, often resistant, organizational environments. The instrument addresses both research and applied corporate needs by answering fundamental evaluative questions:

• Are trainees actively and routinely enacting the targeted communication behaviors during their day-to-day job workflows?
• Do environmental workplace impediments—such as contrary managerial directives, task incongruence, or systemic resource deficits—suppress the execution of validly taught competencies?
• Did the training intervention produce cognitive self-efficacy that allows independent execution, or does the employee encounter skill deficits that necessitate remedial, on-the-job re-learning?
• Does the behavioral acquisition of communication skills positively modulate secondary psychological outcomes, such as employee role comfort, organizational commitment, and promotional aspirations?

In research contexts, the CTIQ serves as a standardized dependent or mediating measure in organizational communication scholarship, evaluating pedagogical interventions, instructional design variations, and transfer-of-training theoretical models. In applied organizational settings, the instrument provides corporate trainers, industrial-organizational psychologists, and organizational development specialists with actionable diagnostic data. By delineating instructional failures (e.g., confusing instructional delivery or inadequate reference documentation) from structural organizational barriers (e.g., supervisory resistance or job role mismatch), the CTIQ empowers institutions to refine training program designs and remediate environmental transfer impediments.

Psychological Construct

The construct operationalized by the CTIQ is Organizational Communication Training Transfer and Behavioral Competency. Within psychometrics and human resource development, training transfer is conceptualized not as a monolithic, binary outcome (transferred versus non-transferred), but as a complex, multifaceted socio-cognitive process. This process encompasses cognitive retention, behavioral generalization across novel contexts, maintenance over time, and the negotiation of interpersonal and institutional affordances or obstacles. The CTIQ evaluates this construct through distinct, interrelated behavioral and contextual dimensions:

1. Behavioral Enactment and Skill Utilization

This dimension assesses the frequency, fluency, and automaticity with which the employee deploys the target communication behavior in real-time occupational situations. Effective transfer requires that behaviors move beyond cognitive awareness into regular, routinized workplace practice (e.g., Item 1: “I use this skill regularly on the job”). It incorporates self-efficacy and behavioral autonomy, capturing whether the individual can execute the interpersonal skill independently without requiring constant supplementary practice or external hand-holding (e.g., Item 2: “After this training program I would perform this skill without practicing”; Item 9: “I get help to perform the skill because I didn’t learn it in the training program”).

2. Systemic Workplace and Environmental Constraints

A central psychometric innovation of the CTIQ is its recognition that a failure to demonstrate trained behavior is frequently an artifact of ecological workplace constraints rather than instructional inadequacy. This dimension reflects transfer climate variables:

• Supervisory Alignment and Social Support: Direct supervisory resistance or contradictory instruction frequently extinguishes learned behaviors (Item 6: “I perform the skill differently on the job because my supervisor told me to do it differently”).
• Workplace Environmental Incompatibility: Heavy task pacing, rigid protocols, or incompatible workplace physical and social arrangements can prevent direct execution (Item 4: “I perform the skill differently on the job because work conditions don’t permit me to perform it the way I learned in the training program”).
• Role Relevance and Job Match: An absolute barrier to transfer occurs when training content is detached from the employee’s actual occupational duties (Item 8: “The skill isn’t part of my job”; Item 12: “I don’t perform the skill because I was assigned a different job”).

3. Instructional Efficacy and Pedagogical Ecology

This dimension isolates the perceived quality, clarity, and fidelity of the training curriculum itself. It examines whether the communication pedagogy provided sufficient cognitive structuring, psychological safety, and supportive materials to facilitate transfer, or whether design flaws impeded learning (e.g., Item 13: “I learned to perform the task well in the training program because the program was effective”; Item 15: “I had trouble learning the skill because the training program was confusing”; Item 16: “The skill would have been easier to learn with more reference materials”). Furthermore, it gauges pedagogical necessity versus alternative self-directed learning modalities (Item 14: “The skill could be learned from a manual or an instruction sheet as easily as in a training program”).

4. Affective-Motivational Outcomes and Organizational Engagement

Beyond isolated motor or verbal skill enactment, effective communication training influences psychological alignment with the broader enterprise. This dimension measures psychological comfort in job execution, enhanced affective commitment toward the employer, motivation toward lifelong professional development, and perceived career mobility resulting from competency development (e.g., Item 17: “Because of learning this skill I feel more comfortable about doing my job”; Item 18: “Because of attending this training program, I feel better about the company”; Item 20: “I think my participation in this training program will help me to advance in the company”).

Theoretical Framework

The Communication Training Impact Questionnaire is theoretically grounded in the convergence of three foundational paradigms within applied psychology: Kirkpatrick’s Hierarchical Evaluation Model, Social Cognitive Theory, and Workplace Transfer Climate Theory.

Kirkpatrick’s Four-Level Model of Training Evaluation

Formulated by Donald Kirkpatrick in the late 1950s and extensively codified in subsequent decades (Kirkpatrick, 1967, 1979), this framework posits that comprehensive educational evaluation must traverse four progressive strata:

1. Level 1: Reaction — Trainee satisfaction, subjective impressions, and perceived immediate utility.
2. Level 2: Learning — Measurable acquisition of knowledge, facts, techniques, or cognitive concepts before leaving the classroom.
3. Level 3: Behavior — The observable transformation, transfer, and application of newly acquired skills within the trainee’s authentic operational job environment.
4. Level 4: Results — Macro-level organizational outcomes, including cost savings, revenue generation, error rate reductions, and enhanced organizational productivity.

DeWine (1987) explicitly positioned the CTIQ as a targeted operationalization of Level 3. Historically, organizational evaluators conflated Level 1 reactions with Level 3 behavior, erroneously presuming that enthusiastic workshop reception directly predicted on-the-job execution. DeWine integrated insights from Cornwall (1980) on measuring “back-on-the-job” performance, designing the CTIQ to bridge the psychometric gap between pedagogical completion and enduring behavioral transformation.

Bandura’s Social Cognitive Theory and Self-Efficacy

Albert Bandura’s Social Cognitive Theory underscores that behavioral execution is governed by reciprocal determinism—the dynamic interplay between personal cognitive factors, environmental influences, and behavioral patterns. A central mechanism is self-efficacy: an individual’s subjective confidence in their capacity to successfully execute the behavior required to produce desired outcomes. In the CTIQ, items probing the perceived difficulty of the skill (e.g., Item 10), the necessity of external assistance (Item 9), and behavioral confidence without extensive practice (Item 2) directly reflect the trainee’s post-training communication self-efficacy.

Baldwin and Ford’s Transfer of Training Model

The structural underpinnings of the CTIQ heavily parallel the systemic model formalized by Baldwin and Ford (1988), which delineates training inputs (training design, trainee characteristics, work environment), training outputs (learning and retention), and conditions of transfer (generalization and maintenance). The CTIQ explicitly measures these interacting components, acknowledging that interpersonal communication competencies are particularly vulnerable to environmental suppression if supervisory feedback, peer norms, and institutional reward structures contradict the newly learned behavioral repertoires.

Validity

The construct, content, and predictive validity of the CTIQ were established during its initial psychometric construction and subsequent field administrations across varied organizational environments.

Content and Construct Validity

Content validation originated through an extensive synthesis of organizational communication literature and established workplace performance evaluation taxonomies, most notably the behavioral frameworks recommended by Cornwall (1980). An initial pool of 31 items was authored to represent five theoretically derived operational categories: direct skill utilization, workplace impediment factors, instructional design clarity, managerial alignment, and organizational affective outcomes. Psychometric culling via item-total correlations, inter-item covariance evaluations, and exploratory factor analytic procedures refined the instrument down to 20 robust, unambiguous items that maximized variance extraction while eliminating redundant or low-loading items.

Factorial and Structural Validity

Factorial validity was established through exploratory factor analysis utilizing an oblique rotation and Cattell’s scree test criteria. The analysis confirmed a robust two-factor latent structure (detailed in Section 9), accounting for substantial variance in training impact scores. The empirical emergence of distinct factors delineating Relationship of Training to Job from Skilled Performance substantiated the construct validity of separating environmental/instructional relevance from individual behavioral mastery.

Predictive and Criterion-Related Validity

DeWine (1987) presented empirical evidence of predictive validity by examining post-training performance trajectories. In validation protocols, CTIQ scores collected at two to four weeks post-intervention significantly predicted external indices of employee job performance and communication effectiveness. Furthermore, criterion-related validity was corroborated through triangulated ratings using the 13-item Supervisor’s Training Impact Questionnaire. Statistically significant positive correlations were established between subordinate CTIQ self-reports and independent supervisory assessments of employee behavioral improvement, mitigating common-method variance concerns and confirming that self-reported transfer mirrored observable workplace behavior.

Convergent and Discriminant Validity

Convergent validity was evidenced by significant positive associations with general communication competency measures and standardized organizational climate surveys. Discriminant validity was supported by the instrument’s capacity to distinguish between high-transfer organizational environments (characterized by supportive leadership and direct task alignment) and low-transfer settings (characterized by supervisory conflict or post-training reassignment), demonstrating that the CTIQ does not merely capture undifferentiated worker happiness or social desirability.

Reliability

The Communication Training Impact Questionnaire demonstrates robust psychometric reliability across heterogeneous organizational samples. Internal consistency was extensively evaluated by DeWine (1987) using Cronbach’s alpha coefficients across multiple professional strata:

The stability of Cronbach’s alpha across disparate occupational profiles—from healthcare practitioners to corporate banking personnel—demonstrates the structural resilience of the CTIQ items against occupational role bias. Temporal stability (test-retest reliability) was reinforced by the standardized administration window (two to four weeks post-training), which captures stable behavioral integration rather than fleeting post-instructional affective states. Standardized item-total correlations across the items predominantly exceed the recommended .40 threshold, confirming that all 20 items contribute cohesively to the overarching construct of training impact.

Factor Analysis

The dimensional architecture of the 20-item CTIQ was resolved through exploratory factor analysis (EFA). The initial 31 items, derived from theoretical literature and Cornwall’s (1980) frameworks, underwent psychometric reduction. Using principal factor extraction followed by an oblique rotation (allowing latent factors to correlate, consistent with psychological and behavioral theory), DeWine inspected eigenvalues alongside Cattell’s scree test.

Two-Factor Latent Structure

The scree test clearly delineated a two-factor model, leading to the retention of 20 items exhibiting strong primary factor loadings (typically ≥ .40) and minimal cross-loadings. The two identified factors were:

• Factor 1: Relationship of Training to Job
  This dimension reflects the macro-alignment between the training intervention, the organizational system, and the functional demands of the trainee’s specific occupational role. It captures environmental compatibility, job match, perceived instructional quality relative to job needs, and broader organizational outcomes (such as morale, promotional prospects, and job comfort).
• Factor 2: Skilled Performance
  This dimension captures individual-level behavioral execution, fluency, self-efficacy, and operational barriers encountered during actual skill deployment. It reflects whether the employee can independently enact the communication competency in daily workflows or whether performance is hindered by perceived difficulty, memory decay, supervisory interference, or instruction-reality divergence.

The factor correlation between the two oblique factors affirmed that while individual behavioral performance and systemic job-training alignment represent distinct empirical dimensions, they operate synergistically in driving overall training transfer. Structural equation modeling and confirmatory factor analyses in subsequent training evaluation literature have supported the bifactorial representation of post-training transfer outcomes, validating the initial factor structure established by DeWine.

Instrument / Measurement Tool

• Instrument Name: Communication Training Impact Questionnaire (CTIQ)
• Alternative Title: DeWine Communication Training Impact Scale
• Author: Sue DeWine, Ph.D. (1987)
• Construct Assessed: Post-training communication skill transfer, workplace behavioral application, environmental impediments, and instructional efficacy (Kirkpatrick Level 3)
• Format: Self-administered paper-and-pencil or digital psychological survey
• Item Count: 20 items
• Administration Window: Ideally 2 to 4 weeks following training completion
• Target Population: Working professionals, managers, healthcare workers, administrative personnel, and trainees across diverse organizational settings
• Response Scale (5-Point Likert-Type Format):
  • 1 = Strongly Disagree
  • 2 = Disagree
  • 3 = Sometimes Agree and Sometimes Disagree
  • 4 = Agree
  • 5 = Strongly Agree
• Scoring Mechanics & Directionality:
  • Direct Scoring Items (Higher score indicates greater transfer/impact): Items 1, 2, 13, 17, 18, 19, 20.
  • Reverse-Scored Items: Items 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, and 16.
  • Reverse Transformation Formula: $\text{Reversed Score} = 6 – \text{Original Score}$ (i.e., $1 to 5$, $2 to 4$, $3 to 3$, $4 to 2$, $5 to 1$).
  • Total Score Range: 20 to 100 points, calculated by summing all 20 items after reverse-scoring. Higher composite scores represent superior behavioral transfer, supportive transfer climates, and higher perceived training impact.
• Complementary Parallel Instrument: Supervisor’s Training Impact Questionnaire (a 13-item observational instrument completed by direct supervisors to validate subordinate transfer).

Permissions & Fee and Test Year

The Communication Training Impact Questionnaire was published in 1987 in the peer-reviewed journal Journal of Applied Communication Research (Vol. 15, Issues 1–2, pp. 113–127). The instrument was developed under academic research auspices and has been placed in the public domain for non-commercial educational, scholarly, and organizational research purposes, provided appropriate academic citation and attribution are given to Dr. Sue DeWine and the original publication.

Organizations and commercial consultants wishing to incorporate the instrument into proprietary, fee-for-service enterprise training software platforms or commercial evaluation systems should consult standard fair-use copyright guidelines or seek formal permissions via the original publisher (Taylor & Francis / Eastern Communication Association) or the author.

References

• Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105. https://doi.org/10.1111/j.1744-6570.1988.tb00632.x
• Cornwall, J. B. (1980). Measuring back-on-the-job performance. Training, 17(3), 289–292.
• DeWine, S. (1987). Evaluation of organizational communication competency: The development of the Communication Training Impact Questionnaire. Journal of Applied Communication Research, 15(1–2), 113–127. https://doi.org/10.1080/00909888709360346
• DeWine, S., & Pearson, J. C. (1985). The most frequently used self-report instrument in communications. Paper presented at the Annual Meeting of the International Communication Association, Honolulu, HI.
• Kirkpatrick, D. L. (1967). Evaluation of training. In R. L. Craig & L. R. Bittel (Eds.), Training and Development Handbook (pp. 87–112). McGraw-Hill.
• Kirkpatrick, D. L. (1979). Techniques for evaluating training programs. Training and Development Journal, 33(6), 78–92.
• Ledebur, G. W. (1982). Effects of interpersonal skill training upon the quality of parent-teacher conferences (Doctoral dissertation, Lehigh University). ProQuest Dissertations Publishing.

13. Items of the Scale (Questionnaire)