Additional Module Moderation – Münster Questionnaire for Evaluation (MFE-ZMo)

Abstract

The Additional Module Moderation – Münster Questionnaire for Evaluation (German: Zusatzmodul Moderation – Münsteraner Fragebogen zur Evaluation, abbreviated as MFE-ZMo) is an economically constructed, psychometrically validated evaluation instrument designed to assess the quality of instructor-led moderation, group facilitation, and communicative orchestration in higher education learning environments. Developed at the Department of Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster) by Meinald T. Thielsch and colleagues, the module operates as an optional, specialized extension within the modular architecture of the Münster Questionnaire for Evaluation (Münsteraner Fragebogen zur Evaluation, MFE). While standardized course evaluation instruments often employ broad, monolithic survey batteries that burden student respondents and fail to capture specific pedagogical methods, the MFE system utilizes core modules for lectures (MFE-Vr) and seminars (MFE-Sr) augmented by targeted supplementary modules selectable by instructors based on their specific course design.

The MFE-ZMo specifically measures students’ perceptions of instructional facilitation, including structural clarity, conversational pacing, pedagogical engagement, conversational climate, and the facilitation of student participation. Administered primarily through a tailored, web-based survey management architecture, the instrument features a 7-point Likert-type response scale ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), complemented by an explicit opt-out category (“nicht sinnvoll beantwortbar” / not reasonably answerable) to mitigate forced-choice measurement artifact. Empirical investigations demonstrate excellent psychometric properties, characterized by high internal consistency (Cronbach’s alpha between .86 and .88) and robust unidimensionality confirmed through exploratory factor analysis and scree test criteria. The scale exhibits pronounced convergent validity with primary dimensions of instructional quality—most notably instructor didactics and self-reported learning gains—while maintaining discriminant distinctiveness against irrelevant artifacts and high sensitivity in differentiating between course offerings. Consequently, the MFE-ZMo serves as a dependable diagnostic instrument for institutional quality management, pedagogical personnel development, and empirical research on higher education classroom dynamics.

Keywords

course evaluation, student evaluation of teaching, MFE-ZMo, moderation quality, higher education pedagogy, teaching evaluation, classroom facilitation, psychometrics, academic quality assurance, Münster Questionnaire for Evaluation

Authors

The Münster Questionnaire for Evaluation (MFE) system and its supplementary modules, including the MFE-ZMo, were conceptualized, standardized, and psychometrically validated by researchers and psychometricians affiliated with the Department of Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster), Germany:

• Dr. Dipl.-Psych. Meinald T. Thielsch — Senior Researcher and Academic Director, Institut für Psychologie, Westfälische Wilhelms-Universität Münster, Fliednerstraße 21, 48149 Münster, Germany. Primary contact email: [email protected]. Dr. Thielsch specializes in psychological assessment, human-computer interaction, web-based diagnostics, and institutional evaluation methodologies.
• B.Sc. Ina Stegemöller — Research Associate and Psychometrician, Institut für Psychologie, Westfälische Wilhelms-Universität Münster, Fliednerstraße 21, 48149 Münster, Germany. Co-investigator in the programmatic evaluation revision, item calibration, and empirical validation studies of the institutional PsyEval platform.
• Collaborating Contributors and Historical Developers: Initial conceptual blueprints and departmental survey frameworks were established through early foundational works on university teaching quality by Grabbe (2003), followed by structural software implementations and iterative modular scale developments by Moeck and Thielsch (2004), Haaser, Thielsch, and Moeck (2007), and Bechler and Thielsch (2012).

Purpose

The primary purpose of the Additional Module Moderation (MFE-ZMo) is to deliver a targeted, methodologically sound, and economically viable diagnostic measurement of instructor moderation competence within academic courses. Modern tertiary education frequently moves away from traditional, teacher-centered didactic monologues toward participatory, student-centered learning arrangements such as interactive seminars, problem-based colloquia, thematic discussions, and collaborative workshops. In these student-centered settings, the instructor’s role shifts fundamentally from a primary transmitter of propositional knowledge to a communicative facilitator, moderator, and intellectual coach. Despite this shift, conventional Student Evaluations of Teaching (SET) frequently rely on generic classroom metrics that evaluate instructional lecture delivery or general syllabus organization, entirely overlooking the pedagogical nuances of discourse leadership.

The MFE-ZMo addresses this diagnostic gap by capturing the specific instructional behaviors that define effective academic moderation. These behaviors include providing structural clarity to discussions, sustaining an open and psychologically safe conversational climate, managing conversational pacing, encouraging hesitant students to actively contribute, and synthesizing complex student viewpoints. The module was explicitly created to satisfy dual institutional demands: high diagnostic specificity and psychometric brevity. Because university course evaluations typically coincide with the close of the academic semester—a period during which students face acute time constraints, cognitive fatigue, and impending examination deadlines—lengthy evaluation batteries often induce survey fatigue, satisficing behaviors, elevated non-response rates, and uncalibrated answers. By condensing moderation diagnostics into a concise battery, the MFE-ZMo maximizes test economy while minimizing respondent burden.

In applied higher education governance, the MFE-ZMo fulfills several interrelated diagnostic and administrative functions:

• Formative Instructional Feedback: Providing instructors with nuanced, behaviorally anchored feedback regarding their discursive leadership. Instructors receive precise diagnostic indicators highlighting whether their moderation provides sufficient conversational structure, cultivates engagement, and allows appropriate time for collective deliberation.
• Targeted Pedagogical Development: Serving as an objective baseline for university didactic centers and academic development units. Diagnostic data from the MFE-ZMo guide the design of bespoke pedagogical coaching, rhetoric workshops, and group facilitation seminars for academic teaching staff.
• Summative Quality Assurance and Accreditation: Functioning as an empirical component within departmental quality management cycles. It offers faculty deans and evaluation committees reliable metrics capable of discriminating instructional effectiveness across heterogeneous course structures without penalizing interactive pedagogical methods.
• Higher Education Research: Supplying researchers with a standardized, unidimensional measurement instrument to investigate the empirical interrelations between discussion facilitation, active student engagement, cognitive load, and subjective academic learning outcomes.

Psychological Construct

The central psychological construct measured by the MFE-ZMo is Instructional Moderation Quality in higher education settings. Rooted in applied educational psychology, small-group communication theory, and social constructivist pedagogical models, moderation quality represents a multidimensional yet structurally coherent pedagogical competence. In an academic discourse context, high-quality moderation is defined as the instructor’s dynamic orchestration of communicative interactions to foster collective intellectual inquiry, critical debate, collaborative knowledge construction, and autonomous student reasoning.

Rather than reflecting a rigid, top-down behavioral script, instructional moderation represents an adaptive communicative skill set composed of several key facets:

• Discursive Structuring (Structural Clarity): The extent to which the facilitator establishes, communicates, and maintains a coherent conceptual framework for the ongoing discussion. Moderation requires organizing complex academic deliberations, highlighting thematic transitions, redirecting tangential discourse back to core curricular objectives, and preventing disorganized conversational fragmentation. A well-structured moderation provides students with external cognitive scaffolding, reducing extraneous cognitive load and helping them contextualize their contributions within broader theoretical domains.
• Motivational Engagement and Facilitation: The instructor’s ability to activate students’ intrinsic motivation and promote broad participation. Effective facilitators foster a climate that encourages both vocal and reticent students to contribute arguments, raise critical queries, and engage with conflicting perspectives. This dimension aligns with self-determination theory, supporting student autonomy and communicative competence.
• Socio-Emotional Climate and Psychological Safety: The deliberate cultivation of a collegial, respectful, and constructive conversational atmosphere. Academic dialogue requires interpersonal vulnerability, as students must expose tentative hypotheses, intellectual uncertainties, or unorthodox interpretations to peer review. High-quality moderation fosters psychological safety, ensuring that contributions are received constructively and that errors are treated as valuable learning opportunities rather than targets for derision.
• Temporal and Deliberative Flexibility: The facilitator’s capacity to balance syllabus constraints against emergent, pedagogically rich student dialogues. Rigid time management can stifle spontaneous intellectual discovery, whereas a complete lack of temporal boundaries risks derailment. Competent moderation provides adequate conversational room and deliberative pause for students to formulate thoughts, process challenging ideas, and engage in constructive debate.
• Instructor Presence and Passion: The non-verbal and verbal communicative engagement demonstrated by the instructor. Perceived commitment, intellectual enthusiasm, communicative attentiveness, and active listening validate student contributions and signal that the facilitator is genuinely invested in the shared learning process.

Within the psychometric architecture of the MFE-ZMo, these communicative behaviors function as mutually reinforcing indicators of a singular, overarching construct: perceived competence in instructional discourse leadership.

Theoretical Framework

The MFE-ZMo is grounded in educational and psychological frameworks that model classroom learning as an active, socially mediated process. It draws primarily upon three theoretical traditions: Social Constructivist Learning Theory, Cognitive Load Theory, and the Community of Inquiry (CoI) Framework.

Social Constructivism and Dialogic Pedagogy

Informed by the foundational work of Lev Vygotsky (1978), social constructivism posits that higher cognitive functions originate through interpersonal, semiotic interactions before becoming internalized as individual thought processes. In higher education, knowledge is not merely passively acquired from an authoritative source; it is constructed through collaborative discourse, peer negotiation, and intellectual argumentation. Within this framework, classroom discussion serves as an intellectual arena where cognitive models are challenged, refined, and synthesized. However, unmoderated student discussions often collapse into unstructured exchanges or dominant speaker monopolies. The MFE-ZMo is theoretically anchored in the premise that the instructor serves as a mediator within the student’s Zone of Proximal Development (ZPD), providing scaffolding that enables deeper cognitive engagement than students could achieve independently.

Cognitive Load Theory and Instructional Scaffolding

From the perspective of Cognitive Load Theory (Sweller, 2010), participatory discussions impose substantial cognitive demands. Students must simultaneously process complex academic content (intrinsic load), decipher ambiguous social dynamics, and navigate disorganized discourse (extraneous load). When moderation lacks structure or conversational goals are poorly defined, extraneous cognitive load increases, limiting the working memory capacity available for meaningful schema construction (germane processing). The structural clarity captured by the MFE-ZMo reflects instructional scaffolding that reduces extraneous communicative ambiguity, clarifying the conceptual trajectory and helping students direct their cognitive resources toward complex intellectual synthesis.

The Community of Inquiry (CoI) Model

The MFE-ZMo strongly aligns with the Community of Inquiry framework developed by Garrison, Anderson, and Archer (2000), which identifies three core presences necessary for effective higher education learning: Cognitive Presence, Social Presence, and Teaching Presence. The MFE-ZMo operationalizes key dimensions of Teaching Presence, specifically “Facilitating Discourse” and “Direct Instruction”:

• Teaching Presence: In the CoI model, facilitating discourse requires identifying areas of agreement and disagreement, seeking consensus, encouraging participation, and maintaining intellectual focus. MFE-ZMo items reflect this operationalization by assessing structural guidance, engagement, and temporal management.
• Social Presence: Learning requires a supportive social climate where participants feel safe expressing their ideas. The MFE-ZMo measures this by evaluating whether the instructor establishes a comfortable, productive working environment that encourages active student involvement.

Validity

The validity of the MFE-ZMo has been examined through empirical studies conducted across multiple academic semesters at the University of Münster, focusing on construct, convergent, discriminant, and criterion-related validity evidence in naturalistic higher education environments.

Construct and Factorial Validity

Construct validity was initially confirmed using comprehensive item and factor analyses across diverse course formats (seminars and lectures). Kaiser-Meyer-Olkin (KMO) measures of sampling adequacy reached .90, verifying that the correlation matrices were well-suited for factor extraction. Principal axis factoring and exploratory factor analyses consistently demonstrated that the scale’s items load onto a single dominant latent factor that accounts for a substantial proportion of shared variance, supporting the construct validity of a unified moderation dimension.

Convergent Validity

To demonstrate convergent validity without relying on a monolithic global score of teaching quality, researchers examined bivariate correlations between the MFE-ZMo and established subscales from the standardized Münster core modules (MFE-Sr for seminars and MFE-Vr for lectures):

• Instructor and Didactics (Dozent & Didaktik): Substantial positive correlations emerged between the moderation score and general instructional didactics across both seminars ($r = .78$) and lectures ($r = .77$). These strong coefficients demonstrate that students view moderation competence as a core component of overall instructional quality.
• Subjective Learning Gains (Lernerfolg): Student ratings of moderation correlated strongly with self-reported academic learning progress ($r = .62$ in seminars; $r = .63$ in lectures). This confirms that high-quality discourse facilitation is closely linked to perceived intellectual growth.
• Course Recommendation (Weiterempfehlung): Moderation scores correlated positively with students’ willingness to recommend the course to peers ($r = .37$ in seminars; $r = .29$ in lectures).
• Overall Course Evaluation (Gesamtbeurteilung): Global ratings of overall course excellence demonstrated strong positive associations with the moderation module ($r = .60$ in seminars; $r = .74$ in lectures).
• Instructional Materials (Materialien): Moderation showed moderate to strong positive associations with the perceived quality of course literature and pedagogical materials ($r = .75$ in seminars; $r = .61$ in lectures).

Discriminant and Divergent Validity

Discriminant validity was established through predictable negative correlations with instructional deficit indicators. Specifically, the MFE-ZMo correlated negatively with the validated “Excessive Demands / Cognitive Overload” subscale (Überforderung) across both seminars ($r = -.34$) and lectures ($r = -.23$). These inverse associations confirm that well-moderated courses help alleviate cognitive overload rather than exacerbating student strain.

Discriminative Criterion Validity (Course Differentiation)

In course evaluation methodology, a key criterion for instrument validity is discriminative validity: the capacity of an aggregated student metric to reliably detect and differentiate performance variations between distinct courses. To assess this, a one-way analysis of variance (ANOVA) was conducted with evaluated courses functioning as the independent factor ($df = 22$) and the aggregated MFE-ZMo moderation score as the dependent variable. The omnibus test demonstrated significant between-course variance ($F = 4.59, p < .01$) with a medium-to-large effect size ($eta^2 = .20$). This confirms that the instrument effectively captures meaningful pedagogical differences across classrooms rather than reflecting uniform ceiling effects or undifferentiated response biases.

Reliability

The reliability of the MFE-ZMo has been confirmed through classical test theory metrics, focusing on internal consistency, item-total correlations, and measurement stability across diverse student cohorts.

Internal Consistency

Across validation cohorts ($N = 438$ to $483$ students enrolled in undergraduate and postgraduate degree programs), the MFE-ZMo demonstrates high internal consistency. The standardized Cronbach’s alpha coefficient for the moderation battery reaches $lpha = .86$ to $.88$. This level of internal consistency exceeds standard psychometric benchmarks for short-form diagnostic scales ($lpha ge .70$ or $.80$), confirming strong measurement precision while avoiding redundant item formulations.

Item Discrimination and Homogeneity

Corrected item-total correlations (part-whole corrected discriminatory indices, $r_{it}$) demonstrate high scale homogeneity. Across all evaluated items, corrected item-total correlations range consistently between $.62$ and $.75$:

• Item 1 (Structural clarity of moderation): $r_{it} = .75$
• Item 2 (Instructor engagement and commitment): $r_{it} = .67$
• Item 3 (Motivation to actively participate): $r_{it} = .62$
• Item 4 (Linguistic clarity and working atmosphere): $r_{it} = .70$
• Item 5 (Discursive pacing and discussion time): $r_{it} = .66$

Furthermore, stepwise deletion analyses indicate that removing any single item yields alpha coefficients between $.85$ and $.88$, demonstrating that each item contributes meaningfully to the common measurement model without destabilizing overall scale reliability.

Scale Descriptive Characteristics

Psychometric evaluations from institutional datasets demonstrate typical distribution parameters for higher education evaluations. Across evaluated cohorts ($N = 438$), the overall moderation scale mean reached $M = 6.14$ ($SD = 0.91$) with a scale median of $Me = 5.94$. As is common in voluntary university course evaluations, the raw scores exhibit moderate negative skewness (Skewness $= -1.70, SE = 0.12$) and positive kurtosis (Kurtosis $= 4.49, SE = 0.23$). These distribution characteristics reflect generally positive student evaluations, yet the retained variance remains sufficient to preserve high statistical sensitivity for institutional benchmarking.

Factor Analysis

The dimensional structure of the MFE-ZMo was evaluated using both exploratory factor analysis (EFA) and structural verification procedures to ensure that its items measure a single, coherent psychological construct.

Exploratory Factor Structure

Prior to factor extraction, the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy was calculated. The resulting value of $.90$ falls into the “marvelous” range according to Kaiser’s classification, confirming that the inter-item correlation matrix is well-suited for factor decomposition. Bartlett’s Test of Sphericity was statistically significant ($p < .001$), confirming meaningful inter-item covariance.

A Principal Axis Factoring (PAF) analysis was conducted, accompanied by oblique Promax rotation to accommodate potential sub-facet interdependencies. Both the Kaiser-Guttman eigenvalue criterion (eigenvalue $> 1.0$) and Cattell’s scree plot inspection unequivocally supported a unidimensional, single-factor solution:

• Eigenvalue Extraction: The primary latent factor yielded an initial eigenvalue of $3.80$, accounting for $54.2%$ of the total variance among the items. All subsequent extracted factors produced eigenvalues well below $1.00$, confirming the absence of secondary dimensions.
• Factor Loadings: Standardized primary factor loadings across the items were consistently high, ranging from $lambda = .65$ to $lambda = .82$ ($ ext{Mean } lambda pprox .74$).

Item-Level Parameter Summary

These robust factor loadings and homogeneous psychometric parameters demonstrate that the MFE-ZMo behaves as an empirical unidimensional composite, justifying the calculation of a single overall mean score.

Instrument / Measurement Tool

The MFE-ZMo is structured as a standardized, brief self-report evaluation scale designed for digital or paper-based academic administration. Its formal parameters are structured as follows:

• Instrument Type: Standardized student evaluation of teaching (SET) supplementary module; rating battery.
• Application Format: Primarily administered online via automated course evaluation platforms (e.g., PsyEval PHP/MySQL platforms, EvaSys, or institutional learning management systems); equally adaptable for paper-and-pencil surveying.
• Target Population: Undergraduate, graduate, and continuing education students enrolled in tertiary education courses featuring interactive or facilitated discourse elements.
• Item Count: 5 standardized items assessing instructor moderation performance.
• Response Scale: 7-point Likert-type response scale with an additional opt-out category:
  • 1 = “stimme gar nicht zu” (strongly disagree)
  • 2 = “stimme nicht zu” (disagree)
  • 3 = “stimme eher nicht zu” (somewhat disagree)
  • 4 = “neutral” (neutral)
  • 5 = “stimme eher zu” (somewhat agree)
  • 6 = “stimme zu” (agree)
  • 7 = “stimme vollkommen zu” (strongly agree)
  • Additional option: “nicht sinnvoll beantwortbar” (not reasonably answerable / opt-out)
• Scoring and Aggregation Rules: Because the module is empirically unidimensional, an overall moderation quality score is computed by calculating the arithmetic mean across all completed items (excluding opt-out responses):
  $$\text{MFE-ZMo Score} = \frac{1}{k} \sum_{i=1}^{k} X_i$$
  where $X_i$ represents the student’s rating on item $i$, and $k$ represents the number of validly answered items ($k le 5$). Scale scores range from $1.00$ to $7.00$, with higher values indicating greater perceived moderation competence. Aggregate course-level ratings are generated by averaging individual student scores across the classroom cohort.
• Administration Time: High economy; completion requires approximately 60 to 90 seconds.

Permissions & Fee and Test Year

• Year of Development: The initial modular architecture of the Münster Questionnaire for Evaluation (MFE) was established in 2003 (Moeck & Thielsch, 2004). The standardized revised iteration of the MFE-ZMo was calibrated and published in 2010/2011 following comprehensive item re-analyses.
• Copyright and Governance: Academic copyright remains with the primary authors (Dr. Meinald T. Thielsch and colleagues) and the Department of Psychology, Westfälische Wilhelms-Universität Münster.
• Licensing and Availability: The MFE-ZMo is an open-access scientific diagnostic instrument. It is made freely available without financial licensing fees for non-commercial academic research, institutional quality assurance, and pedagogical evaluation in higher education settings.
• Usage Conditions: Academic users are requested to maintain proper scientific citation of the source instrument and notify the primary contact author ([email protected]) when deploying the module within large-scale institutional evaluation initiatives.

References

• Bechler, H., & Thielsch, M. T. (2012). Evaluation im Zeitalter von Bologna: Einflüsse des Erhebungszeitpunkts auf die Ergebnisse universitärer Lehrevaluation. Zeitschrift für Hochschulentwicklung, 7(3), 11–24. https://doi.org/10.3217/zfhe-7-03/02
• Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6
• Göritz, A. S., Soucek, R., & Bacher, J. (2005). Online-Lehrevaluation: Methodische Aspekte und empirische Befunde. Das Hochschulwesen, 53(5), 188–193.
• Grabbe, Y. (2003). Entwicklung und Evaluation eines Fragebogens zur Erfassung der Lehrqualität an Hochschulen [Unpublished Diploma thesis]. Department of Psychology, Westfälische Wilhelms-Universität Münster.
• Haaser, P., Thielsch, M. T., & Moeck, A. (2007). PsyEval: Ein internetgestütztes System zur universitären Lehrevaluation. In M. T. Thielsch & C. Austerschulte (Eds.), Pioniere der Psychologie in Münster (pp. 95–108). Münster: Monsenstein und Vannerdat.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential baises, and utility. Journal of Educational Psychology, 76(5), 707–754. https://doi.org/10.1037/0022-0663.76.5.707
• Moeck, A., & Thielsch, M. T. (2004). Entwicklung und Erprobung von Zusatzmodulen zur Lehrevaluation. Psychologisches Institut I, Westfälische Wilhelms-Universität Münster.
• Rindermann, H. (1996). Unterrichtsevaluation an Hochschulen: Ansätze zur Qualitätssicherung und Qualitätsverbesserung. Landau: Verlag Empirische Pädagogik.
• Schmidt, B., & Loßnitzer, T. (2010). Lehrevaluation an deutschen Universitäten: Ein Überblick über Erhebungsverfahren und methodische Standards. Heidelberg: Universitätsverlag Winter.
• Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5
• Thielsch, M. T., & Weltzin, S. (2012). Online-Lehrevaluation: Praktische Erfahrungen und empirische Befunde eines integrierten Qualitätsmanagementsystems. Qualität in der Wissenschaft, 6(1), 18–25.
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Items of the Scale