Additional Module Discussion – Münster Questionnaire for Evaluation (MFE-ZDi)

1. Abstract

The Münster Questionnaire for Evaluation – Discussion Module (German: Münsteraner Fragebogen zur Evaluation – Zusatzmodul Diskussion, abbreviated as MFE-ZDi) is a specialized, psychometrically validated evaluation instrument developed at the University of Münster. It is designed to capture university students’ subjective perceptions and pedagogical appraisals of in-class academic discussions within higher education course environments, such as seminars, colloquia, and interactive lectures. Originating from an extensive quality assurance initiative launched in the early 2000s, the MFE-ZDi operates as a modular extension that instructors can append to core evaluation instruments (such as the Münster Questionnaire for Evaluating Seminars, MFE-Sr, or the lecture module, MFE-Vr), tailored specifically to course designs where discursive communication and collaborative dialogue constitute primary didactic methods.

The scale consists of 8 items scored on an authentic 7-point Likert-type agreement scale ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), complemented by an explicit non-substantive option (“nicht sinnvoll beantwortbar” / not applicable). The MFE-ZDi measures a unidimensional construct reflecting the perceived pedagogical quality, participatory safety, cognitive productivity, and communicative efficacy of academic classroom discussions. Psychometric investigations based on empirical field evaluations (N = 172) indicate satisfactory internal consistency, with an overall Cronbach’s alpha of .79. Exploratory factor analyses verify a stable single-factor structure accounting for 50.5% of total variance, with an initial eigenvalue of 4.03 and factor loadings spanning .47 to .90 across items.

Validity evaluations substantiate convergent validity through substantial positive correlations with established higher education quality dimensions, such as instructor pedagogical competency (r = .65), student learning gain (r = .58), participant engagement (r = .60), and overall course satisfaction (r = .65). Divergent validity is demonstrated via an expected negative correlation with perceived academic cognitive overload (r = -.32). Furthermore, analysis of variance (ANOVA) reveals strong discriminative validity (F = 2.08, p < .01, η² = .20), confirming the module’s sensitivity in differentiating didactic quality across distinct courses.

2. Keywords

Münster Questionnaire for Evaluation, MFE-ZDi, higher education evaluation, course evaluation, discussion quality, academic discourse, didactic communication, student evaluation of educational quality, SEEQ, psychometrics, scale validation, university teaching quality, interactive learning, instructional dialogue

3. Authors

The Münster Questionnaire for Evaluation and its modular evaluation system were conceptualized, standardized, and validated by researchers within the Department of Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster), Germany:

• Dr. Dipl.-Psych. Meinald T. Thielsch
  Affiliation: Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstr. 21, 48149 Münster, Germany.
  Email: [email protected]
  Website: www.uni-muenster.de/PsyEval
• B. Sc. Ina Stegemöller
  Affiliation: Westfälische Wilhelms-Universität Münster, Institut für Psychologie, Fliednerstr. 21, 48149 Münster, Germany.
  Website: www.uni-muenster.de/PsyEval
• Contributors & Historical System Architects: Dipl.-Psych. Katrin Grabbe, Dipl.-Psych. Christian Moeck, and Dipl.-Psych. Ilka Haaser (Psychological Department, University of Münster).

4. Purpose

The primary purpose of the Münster Questionnaire for Evaluation – Discussion Module (MFE-ZDi) is to provide an empirically grounded, psychometrically sound, and economically parsimonious assessment tool for capturing student evaluations of didactic classroom discourse in higher education. Course evaluations represent a foundational cornerstone of systemic quality assurance and continuous instructional development in contemporary tertiary education (Student Evaluations of Teaching, SET). As articulated by foundational evaluation theorists such as Rindermann (1996), systematic course appraisal serves multiple critical functions: it fosters the self-reflective pedagogical competence of instructors, pinpoints distinct didactic strengths and vulnerabilities at course, departmental, and institutional levels, enhances transparent pedagogical dialogue between faculty and students, informs faculty development workshops, and supports fair, performance-informed resource allocations.

Despite the proliferation of extensive generic course evaluation instruments across German-speaking universities, traditional batteries have frequently suffered from excessive survey length and a monolithic structural design. Comprehensive instruments often contain dozens of items intended to assess a broad range of didactic modalities simultaneously. Consequently, students in courses that rely predominantly on seminar-style discussions are frequently forced to navigate irrelevant items (such as assessments of laboratory safety, slide design, or formal lecture delivery), inducing survey fatigue, cognitive burden, non-response bias, and careless responding. This challenge is acutely amplified toward the end of an academic semester, when evaluation schedules collide directly with students’ examination preparation periods.

The MFE-ZDi resolves this methodological challenge through a customizable, modular framework. Instructors whose course concepts feature student dialogue, philosophical inquiry, case-study deliberations, or structured debates can selectively append this 8-item module to their core evaluation. In doing so, the instrument achieves high survey economy, substantially minimizing participant time investment while delivering highly targeted diagnostic feedback regarding the communicative and discursive parameters of the seminar.

In applied higher education administration, the MFE-ZDi serves formative and summative evaluation functions. Formatively, it provides instructors with granular data indicating whether discussions were experienced as academically productive, cognitively deep, well-moderated, or conversely, tangential and dominated by a vocal minority. Summatively, it supplies academic departments with standardized, course-sensitive metrics capable of tracking instructional quality across longitudinal cohorts and curricular reforms.

5. Psychological Construct

The MFE-ZDi operationalizes the construct of Perceived Discussion Quality in Higher Education (Lehrevaluation – Qualität von Lehrdiskussionen). In university pedagogy, academic discussion is not merely a passive social exchange; it is a complex, interactive sociocognitive process whereby knowledge is co-constructed, challenged, refined, and consolidated through dialogic engagement. The construct captured by the MFE-ZDi is conceptualized as a unidimensional yet multi-faceted pedagogical phenomenon encompassing four core functional dimensions:

• Discursive Quantity and Structural Sufficiency: Refers to whether classroom dialogues occurred with adequate frequency and appropriate time allocations relative to the curriculum (e.g., “Es fanden ausreichend Diskussionen statt”). This dimension assesses whether the didactic framework offered sufficient space for spontaneous and structured dialectical exchange, preventing unilateral teacher-centered monologue.
• Psychological Safety and Active Participation: Evaluates students’ internal sense of self-efficacy and subjective comfort regarding their individual contributions (e.g., “Ich konnte mich in die Diskussionen sinnvoll einbringen”, “Es herrschte eine offene Diskussionsatmosphäre”, “Ich habe mich gern an den Diskussionen beteiligt”). Within social-constructivist theory, effective learning requires an open, non-judgmental communicative climate where learners feel empowered to articulate provisional ideas, defend arguments, and engage in cognitive risk-taking without fear of humiliation.
• Cognitive Elaboration and Conceptual Gain: Captures the degree to which discourse facilitates deeper comprehension of theoretical content, critical thinking, and cognitive restructuring (e.g., “Die Diskussionen trugen zum besseren Verständnis der Inhalte bei”). Rather than functioning merely as social filler, high-quality academic discussions prompt students to engage in higher-order cognitive processing, test hypotheses against peer feedback, and translate abstract academic theory into grounded mental models.
• Discursive Moderation, Structure, and Goal Orientation: Represents the instructor’s and peer group’s ability to maintain communicative discipline, thematic focus, and productive resolution (e.g., “Die Diskussionen uferten häufig aus” [reversed], “Bei den Diskussionen wurde auf die Relevanz für das Thema geachtet”, “Die Diskussionen wurden sachlich geführt”). Productive academic discussions require delicate navigational balance: dialogues must allow exploratory divergence while actively avoiding chaotic, unfocused thematic derailment.

6. Theoretical Framework

The theoretical foundations of the MFE-ZDi integrate principles from Social Constructivism, Dialogic Pedagogy, and modern Cognitive Psychology, framed within established models of higher education quality assurance.

Social Constructivist and Vygotskian Perspectives

At the center of social constructivist educational theory (Lev Vygotsky, 1978) is the proposition that higher cognitive functions originate as interpersonal interactions before being internalized as intra-individual cognitive structures. Through the “Zone of Proximal Development” (ZPD), learners achieve deeper conceptual insights when scaffolding is provided by knowledgeable instructors or collaborative peers. In an academic seminar, verbal discourse serves as the primary medium for this scaffolding. When students articulate arguments, debate theoretical interpretations, and resolve sociocognitive conflict, they engage in active cognitive schema transformation that cannot be replicated via passive lecture reception.

Cognitive Elaboration and Critical Discourse

According to cognitive elaboration models (Wittrock, 1989; Chi, 2009), learning is profoundly enhanced when students actively manipulate information by generating explanations, integrating novel facts with prior knowledge, and synthesizing divergent perspectives. The MFE-ZDi explicitly operationalizes this theoretical mechanism: items measure whether discussions deepen subject comprehension and stimulate critical appraisal. When discourse is conducted constructively, it compels participants to clarify ambiguities, identify logical fallacies, and develop robust defenses of their academic claims.

Communicative Competence and Didactic Moderation

In accordance with Jürgen Habermas’s theory of communicative action, productive academic discourse requires an “ideal speech situation” characterized by rationality, factual argumentation, mutual respect, and freedom from communicative coercion. In university classrooms, establishing this discursive environment depends heavily upon instructional leadership. Instructors must skillfully balance intellectual openness with thematic discipline, ensuring that discussions do not deteriorate into aimless chatter or aggressive confrontation. The MFE-ZDi captures these moderation dynamics, evaluating whether the conversational atmosphere was open, focused, objective, and intellectually safe.

7. Validity

Validating student evaluation of teaching (SET) instruments presents unique methodological complexities due to the multitude of confounding factors that influence instructional outcomes, including student prior knowledge, intrinsic course interest, class size, and academic discipline (Marsh, 1984; Rindermann, 1996). The validation of the MFE-ZDi was conducted by examining its convergent, divergent, and criterion-related discriminative relationships within the institutional evaluation framework at the University of Münster (N = 172 students across multiple psychology and educational science courses).

Convergent Validity

To demonstrate convergent validity, scale composite scores from the MFE-ZDi were correlated with validated benchmark scales and global evaluation items from the core seminar evaluation module (Münsteraner Fragebogen zur Evaluation von Seminaren, MFE-Sr):

• Instructor Didactic Competence (Dozent & Didaktik): Correlated strongly with discussion quality, r = .65 (p < .001). This confirms that students intimately associate effective pedagogical guidance with high-quality discussion moderation and clarification of ambiguities.
• Participant Engagement and Climate (Teilnehmer): Correlated robustly, r = .60 (p < .001), indicating that collaborative peer commitment and constructive discussion dynamics covary systematically.
• Instructional Materials (Materialien): Exhibited a strong positive relationship, r = .58 (p < .001), reflecting the didactic integration between assigned readings and subsequent in-class debates.
• Subjective Learning Gain (Lernerfolg): Correlated positively at r = .58 (p < .001), demonstrating that students who experience deep, well-moderated discussions report substantially greater self-perceived cognitive advancement.
• Overall Course Evaluation (Gesamtbeurteilung): Demonstrated a large positive correlation of r = .65 (p < .001), while course recommendation willingness (Weiterempfehlung) yielded r = .43 (p < .001).

Divergent Validity

Divergent validity was confirmed through empirical examination of the relationship between the MFE-ZDi and perceived student cognitive overload (Überforderung) measured via the MFE-Sr. As theoretically expected, discussion quality exhibited a statistically significant moderate negative correlation with excessive academic strain (r = -.32, p < .01). When course discussions are well-structured, clarifying, and pedagogically sound, students experience significantly reduced feelings of being academically overwhelmed. Crucially, the moderate magnitude of these correlations demonstrates that while discussion quality relates meaningfully to overall course impressions, it retains substantial unique variance and does not merely duplicate generic course satisfaction.

Discriminative (Known-Groups) Validity

A crucial quality criterion for any course evaluation tool is its capacity to reliably differentiate between distinct instructional courses. To evaluate discriminative validity, a one-way analysis of variance (ANOVA) was conducted across the evaluated courses with course identity as the independent factor and the MFE-ZDi composite score as the dependent variable. The results demonstrated statistically significant differences between individual seminars with a substantial effect size (F[14, 157] = 2.08, p < .01, η² = .20). This confirms that the module is highly sensitive to real variations in instructional delivery and conversational culture across distinct academic classrooms.

8. Reliability

The internal consistency of the MFE-ZDi has been thoroughly evaluated within university evaluation cohorts. For the standardized 8-item module, the overall Cronbach’s alpha is α = .79 (with item-level analysis subsets across iterative cohorts reaching up to α = .87–.89 when evaluating specific factor modifications). In educational measurement contexts where evaluation scores are aggregated at the course level to inform institutional feedback, a reliability coefficient approaching .80 is considered robust, psychometrically sound, and well-suited for diagnostic and quality assurance purposes.

Item Psychometrics and Selectivity

Psychometric analyses of the constituent items revealed strong corrected item-total correlations (discrimination indices, r_it) across the module, indicating that each item contributes consistently to the latent construct:

• Item discrimination indices ranged from r_it = .45 to .84, with central items assessing productive discourse (Item 7) exhibiting exceptional discriminatory power (.84).
• Standard deviations across individual items were comfortably broad, ranging from 1.35 to 1.70 on the 7-point response format, confirming that the scale is free from severe floor or ceiling compression and successfully captures meaningful variance in student sentiment.
• Stepwise alpha-if-item-deleted analyses verified that the retention of the core discursive items maintains structural stability without excessive redundancy.

9. Factor Analysis

Prior to the establishment of the standardized 8-item structure, the preliminary 5-item version utilized in early evaluation cycles demonstrated structural ambiguity during the Summer Semester 2010 review. The five original items split across three under-determined factors, none of which contained more than two items with salient loadings. To resolve this structural limitation, researchers thoroughly revised the item pool, eliminating fragmented components and adding targeted items to establish a stable, theoretically cohesive measurement dimension.

Exploratory Factor Analysis (EFA)

The revised instrument was subjected to formal psychometric testing using a sample of N = 172 university students. Sampling adequacy was verified using the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy, which yielded an outstanding value of MSA = .84, confirming the dataset’s suitability for factor analytic decomposition. Bartlett’s Test of Sphericity was highly significant (p < .001).

An exploratory principal axis factoring (PAF) with oblique (Promax) rotation was conducted. Inspection of the scree plot, accompanied by the Kaiser-Guttman criterion (eigenvalues > 1.0), unequivocally pointed to a single dominant latent factor:

• Eigenvalue: The primary general factor exhibited an initial eigenvalue of 4.03, whereas subsequent factors dropped sharply below the 1.0 threshold.
• Variance Explained: This single overarching factor accounted for 50.5% of the total item variance.
• Factor Loadings: All eight items loaded substantially and cleanly onto the general factor, with standardized factor loadings spanning from .47 to .90. Specifically, items evaluating overall discussion productivity (.90), didactic guidance (.77), and comprehension enhancement (.77) served as primary anchors of the construct.

10. Instrument / Measurement Tool

The operational specifications of the Münster Questionnaire for Evaluation – Discussion Module (MFE-ZDi) are structured as follows:

• Tool Name: Münster Questionnaire for Evaluation – Discussion Module (MFE-ZDi).
• Construct Assessed: Perceived quality, pedagogical efficacy, climate, and moderation of academic discussions in university instruction.
• Administration Format: Standardized self-report questionnaire administered via modern web-based course evaluation portals (e.g., PHP/MySQL online architectures) or paper-and-pencil surveying.
• Positioning within Battery: Presented seamlessly as an elective modular extension directly following core seminar (MFE-Sr) or lecture (MFE-Vr) survey blocks. It operates without redundant secondary instructions, utilizing universal institutional briefing text provided at initial login.
• Item Count: 8 standardized items.
• Response Format: Authentic 7-point Likert-type agreement scale featuring the following verbal anchor options:
  • 1 = “stimme gar nicht zu” (strongly disagree)
  • 2 = “stimme nicht zu” (disagree)
  • 3 = “stimme eher nicht zu” (somewhat disagree)
  • 4 = “neutral” (neutral)
  • 5 = “stimme eher zu” (somewhat agree)
  • 6 = “stimme zu” (agree)
  • 7 = “stimme vollkommen zu” (strongly agree)
  • Special Option: “nicht sinnvoll beantwortbar” (not meaningful / not applicable; treated as system missing).
• Scoring and Computational Rules:
  • Inversion: Negatively keyed items (specifically Item 4: “Die Diskussionen uferten häufig aus”) must be reverse-coded prior to composite score calculation (i.e., recoded as: 1→7, 2→6, 3→5, 4→4, 5→3, 6→2, 7→1).
  • Scale Score Calculation: Given the confirmed unidimensionality of the scale, an overall module score is calculated as the arithmetic mean (or alternatively, the sum) of all validly completed items per participant. Non-applicable responses (“nicht sinnvoll beantwortbar”) are excluded from mean computation.
  • Course Aggregation: For institutional quality reporting, individual student scores are aggregated to compute course-level means (M) and standard deviations (SD), typically compared against departmental benchmarks.

11. Permissions & Fee and Test Year

The conceptual framework of the Münster evaluation system was initiated in 2000/2001, expanded to a modular web architecture in 2003/2004, and underwent systematic psychometric re-standardization and revision in 2010–2012, when the finalized 8-item MFE-ZDi was empirically validated. The instrument is documented in academic repositories including the Leibniz Institute for Psychology (ZPID) Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS).

The scale is made available free of charge for academic, research, and non-commercial institutional quality assurance purposes. Higher education institutions and independent researchers may administer the questionnaire within their internal evaluation architectures provided that appropriate citation and authorship attribution are maintained. For commercial software integrations, institutional consulting deployments, or extended adaptations, interested parties should contact the primary author, Dr. Meinald T. Thielsch, at the Institute of Psychology, University of Münster (www.uni-muenster.de/PsyEval).

12. References

• Chi, M. T. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105. https://doi.org/10.1111/j.1756-8765.2008.01005.x
• Göritz, A. S., Soucek, R., & Bacher, J. (2005). Online-Panels in der Forschung: Stand und Perspektiven. Zeitschrift für Medienpsychologie, 17(2), 65–74. https://doi.org/10.1026/1617-6383.17.2.65
• Grabbe, K. (2003). Entwicklung eines Fragebogens zur Evaluation von Seminaren [Unpublished diploma thesis]. Department of Psychology, University of Münster.
• Haaser, I., Thielsch, M. T., & Moeck, C. (2007). PsyEval: Ein internetgestütztes System zur Lehrevaluation am Psychologischen Institut der WWU Münster. In M. Krämer, S. Preiser, & K. Brusdeylins (Eds.), Psychologiedidaktik und Evaluation VI (pp. 305–314). Shaker Verlag.
• Habermas, J. (1984). The Theory of Communicative Action, Volume 1: Reason and the Rationalization of Society. Beacon Press.
• Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential baises, and utility. Journal of Educational Psychology, 76(5), 707–754. https://doi.org/10.1037/0022-0663.76.5.707
• Moeck, C., & Thielsch, M. T. (2004). Lehrevaluation an der WWU Münster: Zusatzmodule des MFE. Psychologisches Institut 1, Westfälische Wilhelms-Universität Münster.
• Rindermann, H. (1996). Untersuchungen zur Validität von Studentischen Veranstaltungsevaluationen. Empirische Pädagogik.
• Schmidt, B., & Loßnitzer, T. (2010). Lehrevaluation an deutschen Hochschulen: Bestandsaufnahme und methodische Standards. Waxmann Verlag.
• Thielsch, M. T., & Stegemöller, I. (2013). Zusatzmodul Diskussion – Münsteraner Fragebogen zur Evaluation (MFE-ZDi). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis59
• Thielsch, M. T., & Weltzin, S. (2012). Online-Lehrevaluation: Methodische Herausforderungen und empirische Befunde. Das Hochschulwesen, 60(3), 88–95.
• Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press.
• Wittrock, M. C. (1989). Generative processes of comprehension. Educational Psychologist, 24(4), 345–376. https://doi.org/10.1207/s15326985ep2404_2

13. Items of the Scale