Additional Module Computer-Aided Teaching – Münster Questionnaire for Evaluation (MFE-ZcL)

Abstract

The Additional Module Computer-Aided Teaching – Münster Questionnaire for Evaluation (German: Münsteraner Fragebogen zur Evaluation – Zusatzmodul computergestützte Lehre; abbreviated as MFE-ZcL) is a specialized psychometric instrument designed to evaluate the instructional quality, pedagogical pacing, hands-on practice opportunities, and skill acquisition in computer-assisted university courses and academic seminars. Developed at the Department of Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster) by Dr. Meinald Thielsch and colleagues, the MFE-ZcL constitutes an elective supplementary module embedded within the broader Münster Questionnaire for Evaluation (MFE) ecosystem. While traditional Student Evaluations of Teaching (SET) focus broadly on lecture delivery, general instructor rapport, or overarching seminar discussion, modern higher education increasingly relies on specialized digital and software-intensive curricula—such as computational data analysis, statistical modeling (e.g., R, SPSS, Python), digital experimental programming, and computer-aided qualitative inquiry—that require targeted evaluation metrics.

The MFE-ZcL comprises six standardized items: one supplementary diagnostic item assessing software instruction pacing across a 5-point bipolar categorical scale, and five core psychometric items measuring instructional clarity, pre-existing digital knowledge compatibility, autonomous computer practice opportunities, self-efficacy in independent application, and computer competence enhancement. The core five items utilize a 7-point Likert-type response format ranging from 1 ("stimme gar nicht zu" / "strongly disagree") to 7 ("stimme vollkommen zu" / "strongly agree"), accompanied by an explicit "nicht sinnvoll beantwortbar" ("cannot be meaningfully answered") option to prevent forced non-substantive responses. Psychometric investigations demonstrated strong internal consistency (Cronbach's alpha = .84), verified unidimensionality via exploratory factor analysis accounting for 52.7% of total variance (eigenvalue = 2.64; factor loadings ranging from .57 to .80), and confirmed robust convergent validity with core teaching evaluation dimensions such as instructor didactics (r = .79), instructional materials (r = .74), and perceived learning gain (r = .81), alongside an expected negative correlation with cognitive overload (r = -.57). The instrument demonstrates high discriminant sensitivity across distinct university courses (η² = .52), establishing its utility for academic quality management and didactic improvement.

Keywords

Student Evaluations of Teaching (SET), computer-assisted instruction, higher education evaluation, pedagogical didactics, computer self-efficacy, Münster Questionnaire for Evaluation, MFE-ZcL, digital competence, educational measurement, psychometrics, course quality assurance, instructional clarity

Authors

The Münster Questionnaire for Evaluation system and its specialized supplementary modules were conceptualized, developed, and validated by researchers at the Department of Psychology, University of Münster (Westfälische Wilhelms-Universität Münster), Germany:

• Dr. Meinald Thielsch, Dipl.-Psych. — Psychological Institute, Department of Psychology, Westfälische Wilhelms-Universität Münster, Fliednerstraße 21, 48149 Münster, Germany. Email: [email protected]. Website: www.uni-muenster.de/PsyEval.
• Ina Stegemöller, B.Sc. — Institute of Psychology, Westfälische Wilhelms-Universität Münster, Fliednerstraße 21, 48149 Münster, Germany. Website: www.uni-muenster.de/PsyEval.
• Additional Key Collaborators & System Architects: Dr. Corinna Moeck, Frank Grabbe, Dr. Christine Weltzin, and Tim Haaser, who contributed extensively to the structural conceptualization, online infrastructure (PsyEval), and iterative psychometric refinements of the Münster evaluation inventory.

Purpose

The primary purpose of the Additional Module Computer-Aided Teaching (MFE-ZcL) is to provide higher education institutions, faculty members, academic departments, and educational researchers with a brief, highly reliable, and empirically validated instrument to assess the specific quality dimensions of computer-aided and software-driven coursework. In contemporary university curricula—particularly within empirical social sciences, psychology, data science, engineering, and the natural sciences—teaching formats have fundamentally diverged from purely ex-cathedra lectures or traditional reading seminars. Students are frequently required to interact directly with complex software environments, statistical programming languages, laboratory instrumentation interfaces, and simulated data environments. Conventional Student Evaluations of Teaching (SET) instruments routinely fail to capture these distinct didactic realities, relying on broad items regarding general speaker audibility, literature relevance, or lecture room acoustics.

The MFE-ZcL addresses critical educational and administrative needs:

• Targeted Didactic Feedback for Faculty: Instructors who lead computer-lab sessions, statistical workshops, or technical computational seminars receive granular diagnostic information regarding whether their step-by-step screen demonstrations are comprehensible, whether their instructional pacing is appropriate (neither sluggish nor overwhelming), and whether sufficient laboratory time is allocated for students to practice independently.
• Mitigating Survey Fatigue through Modular Architecture: Standard institutional evaluation batteries often suffer from excessive length, provoking survey fatigue, non-response bias, and superficial response sets. By deploying a modular framework where the baseline questionnaire (MFE-Sr for seminars or MFE-Vr for lectures) is kept concise, instructors can intentionally append the MFE-ZcL only when computer-assisted tools represent a substantial component of their pedagogical design. Students are therefore spared from answering irrelevant items, maximizing response quality and institutional compliance.
• Academic Quality Assurance and Curricular Optimization: In accordance with higher education quality management frameworks (e.g., Rindermann, 1996), systematic course evaluations serve formative and summative functions. The MFE-ZcL provides institutional administrators and curriculum committees with quantifiable metrics to identify where laboratory hardware, software licenses, instructional pacing, or student prerequisite preparation require structural interventions or faculty development workshops.
• Empirical Research in Educational Psychology and Digital Learning: The scale enables educational researchers to examine the interplay between instructional clarity during technical software execution, student cognitive load, digital self-efficacy development, and objective academic achievement.

Psychological Construct

The MFE-ZcL assesses student perceptions of the instructional effectiveness, self-efficacy enhancement, and technical delivery of computer-assisted teaching within higher education courses. Rather than operationalizing computer-aided teaching as a passive technological infrastructure (e.g., whether computers functioned mechanically), the construct is anchored in educational psychology as a triadic interaction between the instructor's scaffolding behavior, the student's baseline competence and experiential practice, and the resultant transfer self-efficacy.

The instrument captures five substantive psychological and pedagogical facets across its unidimensional core scale (Items 2 through 6), complemented by an independent bipolar didactic pacing parameter (Item 1):

• Prerequisite Knowledge Alignment (Cognitive Entry Characteristics): Captured by Item 2 ("Meine Computer(vor-)kenntnisse waren ausreichend für dieses Seminar" / "My prior computer knowledge was sufficient for this seminar"). Rooted in cognitive educational theories, learning new technological skills is strongly contingent upon prior domain knowledge. When pre-existing knowledge is inadequate, students experience acute cognitive overload; when instruction underestimates prior knowledge, boredom ensues. This facet measures the perceived alignment between the course's technological entry threshold and the learner's existing competence.
• Instructional Decomposition and Step-by-Step Clarity (Pedagogical Scaffolding): Measured by Item 3 ("Der/Die Lehrende erläuterte die einzelnen Arbeitsschritte am Computer verständlich" / "The instructor explained the individual computer working steps comprehensibly"). In digital learning environments, task performance requires decomposing complex graphical user interface (GUI) or syntax-driven workflows into discrete procedural chunks. This facet taps into the instructor's ability to make procedural sequences transparent, cognitively digestible, and accessible to novices.
• Opportunity for Deliberate, Autonomous Practice (Active Learning): Tapped by Item 4 ("Der/Die Lehrende gab ausreichend Gelegenheiten, die Arbeitsschritte am Computer eigenständig zu üben" / "The instructor provided sufficient opportunities to practice the computer working steps independently"). Passive demonstration without active execution fails to solidify procedural memory. This facet evaluates the allocation of instructional time dedicated to autonomous, experiential problem-solving where students actively manipulate the digital environment.
• Transfer Self-Efficacy and Independent Mastery: Operationalized by Item 5 ("Ich fühle mich in der Lage, das im Seminar Gelernte auch eigenständig am Computer umzusetzen" / "I feel capable of implementing what was learned in the seminar independently on the computer"). Anchored directly in Albert Bandura's construct of perceived self-efficacy, this dimension measures the student's subjective confidence in their capability to execute learned computational workflows autonomously in novel contexts (e.g., during thesis research or external data projects) without immediate instructor scaffolding.
• Perceived Competence Gain (Digital Skill Development): Assessed by Item 6 ("Ich habe meine Kompetenzen im Umgang mit Computern verbessert" / "I have improved my computer competence"). This facet represents the summative self-perceived advancement in general digital literacy, software navigation, and technological problem-solving directly attributable to the course.
• Instructional Pace Adaptation (Diagnostic Process Parameter): Measured supplementarily via Item 1 ("Das Tempo der einzelnen Arbeitsschritte am Computer war?" / "The pace of the individual working steps on the computer was?"). Pacing is a critical non-linear didactic parameter. Both excessively rapid procedural demonstrations and overly lethargic progression derail the cognitive equilibrium necessary for software mastery. Because this item functions along an optimal-point continuum ("far too slow" to "far too fast" with "appropriate" at center), it operates as an independent formative diagnostic rather than a linear component of the core quality scale.

Theoretical Framework

The theoretical architecture of the MFE-ZcL is grounded in the convergence of three foundational paradigms within educational psychology, instructional design, and psychometrics:

1. Quality Assurance and Multi-Perspective Educational Evaluation

The broader Münster Questionnaire for Evaluation framework was built upon the empirical quality models synthesized by Heiner Rindermann (1996) and Herbert W. Marsh (1984). Rindermann emphasized that university teaching quality cannot be captured through a single, undifferentiated global rating. Effective teaching comprises distinct didactic dimensions—clarity of explanation, instructional pace, structuredness, active student involvement, and learning climate—that exert distinct causal influences on student outcomes. Marsh's pioneering work on the Students' Evaluations of Educational Quality (SEEQ) established that teaching quality is multidimensional and that evaluations are valid when specific educational components are isolated and evaluated individually. The MFE modular framework reflects this paradigm: rather than using a sprawling, rigid survey, modularization allows targeted evaluation of specific didactic practices, such as computer-assisted laboratory execution (Grabbe, 2003; Moeck & Thielsch, 2004).

2. Cognitive Load Theory (CLT) and Procedural Skill Acquisition

The specific items of the MFE-ZcL correspond directly to the tenets of Cognitive Load Theory formulated by John Sweller and colleagues. Learning to manipulate complex digital software imposes heavy intrinsic and extraneous cognitive loads upon the learner's working memory. Extraneous cognitive load is generated by poor instructional design—such as rapid, unclear explanations of mouse trajectories, terminal commands, or nested software menus. When an instructor systematically explains each procedural step comprehensibly (Item 3) and adjusts the temporal pacing (Item 1), extraneous load is reduced, freeing working memory capacity for germane cognitive load—the cognitive processing devoted to schema acquisition and procedural mastery. Furthermore, cognitive skill acquisition theories (e.g., John R. Anderson's ACT-R model) posit that proceduralization requires a transition from declarative knowledge (observing the instructor) to procedural compilation through hands-on practice (Item 4).

3. Bandura's Social Cognitive Theory and Self-Efficacy

Albert Bandura's (1997) construct of self-efficacy is integral to the conceptualization of the MFE-ZcL. Computer self-efficacy—defined as an individual's belief in their capability to utilize a computer system to accomplish designated tasks—is not merely an affective byproduct; it is a primary driver of sustained technology adoption and academic persistence. According to Bandura, self-efficacy develops through four primary sources: mastery experiences (hands-on execution, Item 4), vicarious experiences (clear modeled demonstrations by the teacher, Item 3), verbal persuasion/guidance, and physiological/affective states (lowered anxiety due to matched prerequisite skills, Item 2). The terminal items of the MFE-ZcL specifically evaluate whether the educational environment successfully translated instructional scaffolding into autonomous transfer capability (Item 5) and broader digital competence (Item 6).

Validity

The validation of the MFE-ZcL was conducted within the psychological evaluation program at the University of Münster, adhering to established psychometric standards for construct, convergent, discriminant, and criterion-related validity.

Convergent Validity

As noted by Marsh (1984) and Rindermann (1996), validating instructional evaluation instruments presents methodological complexities because instructional success is influenced by student interest, prerequisite ability, and pedagogical structure. To establish construct validity, the authors evaluated convergent correlations between the MFE-ZcL scale score and established dimensions from the validated Münster Seminar Evaluation Baseline Module (MFE-Sr):

• Instructor and Didactics (Dozent & Didaktik): r = .79. This strong positive correlation confirms that structured computer guidance is perceived as an integral component of overall pedagogical competence.
• Instructional Materials (Materialien): r = .74. Demonstrates that effective computer-assisted instruction is closely coupled with high-quality digital hand-outs, exercise scripts, and annotated syntax files.
• Student Learning Community (Teilnehmer): r = .69. Indicates that structured computer exercises foster collaborative engagement, peer consultation, and productive classroom interaction.
• Self-Assessed Learning Gain (Subjektiver Lernerfolg): r = .81. The very high correlation with subjective learning gain provides powerful evidence that students perceive mastery of computer tools as directly synonymous with substantial academic gain.
• Overall Course Evaluation (Gesamtbeurteilung): r = .76. Confirms that the execution of computer-aided instruction significantly determines the global perceived quality of technical seminars.
• Willingness to Recommend (Weiterempfehlung): r = .47. Reflects a moderate-to-high positive relationship with behavioral endorsement intentions.

Divergent and Discriminant Validity

Divergent validity was corroborated through the relationship with the MFE-Sr subscale measuring Cognitive Overload / Excessive Demands (Überforderung). The MFE-ZcL demonstrated a strong negative correlation of r = -.57 with student-reported cognitive overload. This finding empirically substantiates the theoretical model: high-quality computer-assisted teaching provides structured scaffolding that challenges learners productively without triggering cognitive overload or technological frustration.

Furthermore, to examine whether the instrument exhibits discriminative validity—the empirical ability to reliably differentiate between distinct courses of varying instructional quality—an analysis of variance (ANOVA) was conducted with evaluated course sections as the independent variable and the MFE-ZcL scale mean as the dependent variable. The analysis revealed highly significant, substantial differences across evaluated university courses (F = 12.75, df = 6, p < .01), yielding a very large effect size of η² = .52 (partial eta-squared). This confirms that 52% of the variance in MFE-ZcL scores was attributable to actual course-level differences in instructional quality rather than undifferentiated halo effects or homogenous student rating biases.

Reliability

The internal consistency reliability of the Additional Module Computer-Aided Teaching (MFE-ZcL) was evaluated in an empirical validation study encompassing course evaluations from the Winter Semester 2010/2011 through the Winter Semester 2011/2012 at the University of Münster (N = 77 completed evaluations; 79.2% female, 20.8% male; mean age = 22.4 years, SD = 3.7; 96.1% undergraduate psychology majors).

Internal Consistency (Cronbach's Alpha)

For the five-item core unidimensional scale (Items 2 through 6), the overall internal consistency was determined to be:

• Cronbach's Alpha (α): .84

A reliability coefficient of .84 for a brief five-item scale demonstrates high internal consistency, comfortably exceeding the standard benchmark of .70 recommended for exploratory research and the .80 threshold required for institutional diagnostic decision-making (Nunnally & Bernstein, 1994). This level of precision confirms that the items reflect a coherent, mutually reinforcing latent construct, rendering the instrument robust for aggregated course-level and instructor-level benchmarking.

Item Discrimination and Reliability If Item Deleted

Corrected item-total correlations (item discrimination, r_it) and alpha-if-item-deleted statistics confirm that all five items contribute harmoniously to the scale's reliability:

• Item 2 (Prior Knowledge Sufficiency): Item-total correlation r_it = .62; Alpha if item deleted = .82
• Item 3 (Pacing and Clarity of Steps): Item-total correlation r_it = .71; Alpha if item deleted = .80
• Item 4 (Opportunities for Autonomous Practice): Item-total correlation r_it = .71; Alpha if item deleted = .79
• Item 5 (Autonomous Transfer Capability): Item-total correlation r_it = .70; Alpha if item deleted = .80
• Item 6 (Computer Competence Improvement): Item-total correlation r_it = .51; Alpha if item deleted = .85

No item omission leads to a meaningful increase in alpha beyond the baseline .84 (omitting Item 6 yields a marginal change to .85, but its substantive content regarding competence acquisition is essential for face and construct validity). Item discrimination values are high, with four of the five items exhibiting r_it values between .62 and .71, indicating excellent discriminatory power.

Factor Analysis

During the structural revision of the Münster evaluation modules in the summer semester of 2010, the computer-aided teaching module underwent comprehensive psychometric refinement. Initial iterations exhibited factorial instability due to complex secondary cross-loadings on two items. Following targeted item reformulation (specifically revising Items 1 and 2), an empirical factor analysis was executed on the refined instrument.

Sampling Adequacy and Factorability

The statistical suitability of the correlation matrix for factor analysis was evaluated using the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. The overall Measure of Sampling Adequacy (MSA) was .80, indicating meritorious suitability for latent variable modeling.

Exploratory Factor Analysis (EFA) Results

An exploratory principal axis factor analysis (PAF) with oblique (Promax) rotation was conducted on the five substantive items (Items 2–6). Both the Kaiser-Guttman criterion (eigenvalues > 1.0) and visual inspection of the scree plot confirmed a definitive unidimensional 1-factor structure:

• Factor 1 Eigenvalue: 2.64
• Variance Explained: The single factor accounted for 52.7% of the total variance.
• Factor Loadings: All five items loaded substantially and cleanly onto the general factor, with standardized factor loadings (FL) ranging between .57 and .80:

Item	Mean (M)	Std. Deviation (SD)	Discrimination (rit)	Factor Loading (FL)	Alpha If Deleted
Item 2: Prior computer knowledge was sufficient	4.74	1.83	.62	.68	.82
Item 3: Instructor explained steps comprehensibly	5.28	1.73	.71	.80	.80
Item 4: Sufficient opportunities to practice independently	5.25	1.81	.71	.80	.79
Item 5: Able to implement learned skills independently	4.61	1.86	.70	.77	.80
Item 6: Improved competence in handling computers	5.20	1.52	.51	.57	.85

The unifactorial solution confirms that items tapping instructional delivery (Items 3 and 4) and items measuring individual competence/transfer (Items 2, 5, and 6) load onto a single unified dimension of Computer-Aided Instructional Quality in academic seminar contexts.

Instrument / Measurement Tool

The Münster Questionnaire for Evaluation – Additional Module Computer-Aided Teaching (MFE-ZcL) is structured as follows:

• Instrument Type: Standardized student self-report rating scale / course evaluation inventory module.
• Application Mode: Administered primarily via secure web-based evaluation platforms (such as the Münster PHP/MySQL PsyEval architecture) at the conclusion of an academic semester. It can also be administered via paper-and-pencil questionnaires.
• Administrative Structure: The module does not present independent administrative instructions; it is appended seamlessly to the core Münster Seminar Evaluation Module (MFE-Sr). General demographic data and technical instructions are provided upon initial portal login.
• Total Item Count: 6 items total (comprising 1 supplementary pacing item and 5 core psychometric items).
• Response Formats:
  • Item 1 (Diagnostic Pacing): 5-point bipolar categorical scale with anchored options: "viel zu langsam" (far too slow), "zu langsam" (too slow), "angemessen" (appropriate), "zu schnell" (too fast), and "viel zu schnell" (far too fast).
  • Items 2–6 (Core Quality Scale): 7-point Likert-type agreement scale ranging from 1 = "stimme gar nicht zu" (strongly disagree) to 7 = "stimme vollkommen zu" (strongly agree), with a middle neutral point at 4 = "neutral". An explicit non-substantive option, "nicht sinnvoll beantwortbar" (cannot be meaningfully answered / not applicable), is provided to prevent artificial distortion.
• Scoring and Index Computation:
  • Given the demonstrated unidimensionality of Items 2 through 6, scoring is conducted by computing the unweighted arithmetic mean across the five items (or summing raw values, ranging from 5 to 35).
  • Responses marked as "nicht sinnvoll beantwortbar" are treated as missing values (pairwise deletion) and excluded from the composite mean computation.
  • Item 1 is analyzed independently as a formative frequency distribution to detect directional pacing deviations (e.g., percentage of students reporting excessive speed versus excessive slowness).
• Descriptive Scale Norms: In the validation cohort (N = 77), the scale distribution parameters for the composite core scale were: Median = 4.98, Mean = 5.20, Standard Deviation = 1.40, Skewness = -0.40 (SE = .27), Kurtosis = -1.05 (SE = .54).

Permissions & Fee and Test Year

The Additional Module Computer-Aided Teaching (MFE-ZcL) was developed and iteratively revised between 2003 and 2010 at the University of Münster. The definitive psychometrically revised edition documented here was established and validated between 2010 and 2012.

• Availability and Fees: The MFE-ZcL is available free of charge for non-commercial academic research and university teaching evaluation purposes. The instrument was developed under institutional funding at the Westfälische Wilhelms-Universität Münster to enhance academic quality assurance.
• Permissions and Usage Rights: Academic institutions, faculty members, and researchers may adopt the scale items for course evaluations, educational assessments, and empirical studies. Commercial deployment, inclusion in fee-based commercial software suites, or unauthorized redistribution requires explicit formal authorization from the primary authors.
• Contact and Inquiries: Inquiries regarding licensing, integration into external institutional learning management systems (LMS), or access to the comprehensive Münster Questionnaire documentation can be directed to Dr. Meinald Thielsch at [email protected] or via the institutional portal at www.uni-muenster.de/PsyEval.

References

• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
• Bechler, C., & Thielsch, M. T. (2012). Online-Evaluation im Bachelorstudium: Belastung der Studierenden und Qualität der Rückmeldungen. Forschungsbericht des Instituts für Psychologie, Westfälische Wilhelms-Universität Münster.
• Göritz, A. S., Soucek, R., & Bacher, J. (2005). Determinanten des Lernerfolgs in webbasierten Lehrveranstaltungen: Zur Rolle von Motivation und Usability. Zeitschrift für Medienpsychologie, 17(4), 138–148. https://doi.org/10.1026/1617-6383.17.4.138
• Grabbe, F. (2003). Konstruktion und Erprobung eines Fragebogens zur Evaluation von Seminaren am Institut für Psychologie der Universität Münster (Unveröffentlichte Diplomarbeit). Westfälische Wilhelms-Universität Münster.
• Haaser, T., Thielsch, M. T., & Moeck, C. (2007). PsyEval: Konzeption und Umsetzung einer webbasierten Plattform zur universitären Lehrevaluation. In M. T. Thielsch (Hrsg.), Dokumentation zur Lehrevaluation am Fachbereich Psychologie und Sportwissenschaft (S. 15–28). Universität Münster.
• Marsh, H. W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76(5), 707–754. https://doi.org/10.1037/0022-0663.76.5.707
• Moeck, C., & Thielsch, M. T. (2004). Die Zusatzmodule des Münsteraner Fragebogens zur Evaluation (MFE): Konstruktion und erste Bewährung. Forschungsbericht des Psychologischen Instituts I, Westfälische Wilhelms-Universität Münster.
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
• Rindermann, H. (1996). Untersuchungen zur Validität von Studentischen Lehrevaluationen. Empirische Pädagogik, Landau.
• Schmidt, B., & Loßnitzer, T. (2010). Lehrevaluation an deutschen Hochschulen: Bestandsaufnahme und methodische Standards. Verlag für Hochschuldidaktik.
• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
• Thielsch, M. T., & Weltzin, C. (2012). Online-Lehrevaluation: Methodische Herausforderungen und empirische Befunde zu Rücklaufquoten und Datenqualität. Zeitschrift für Hochschulentwicklung, 7(3), 112–126. https://doi.org/10.3217/zfhe-7-03/12

Items of the Scale