Academic Setting Evaluation Questionnaire

1. Abstract

The Academic Setting Evaluation Questionnaire (ASEQ) is a multidimensional psychometric instrument developed to evaluate faculty members’ perceptions of and satisfaction with their higher education organizational environment. Designed by psychometricians and educational researchers J. Fernandez and M. A. Mateo at the Complutense University of Madrid, the ASEQ systematically appraises the multi-tiered working conditions, organizational climate, and instructional dynamics experienced by university professors. Originally operationalized across a broad 69-item pool yielding six empirical dimensions with a theta reliability coefficient of 0.98, the instrument was systematically refined through iterative factor analyses into a psychometrically parsimonious 33-item inventory that consolidates three robust, core latent factors: Satisfaction with Working Conditions (19 items), Social Climate (5 items), and Relationship with Students (7 items), alongside two experimental structural items assessing institutional contracting and research evaluation systems. Administered using a 7-point Likert scale ranging from 1 (the lowest level of agreement) to 7 (the highest level of agreement), the instrument has demonstrated exceptional internal consistency, including an overall Cronbach’s alpha of 0.90, dimension-specific reliabilities between 0.87 and 0.89, and an explained variance exceeding 78% in cross-validation cohorts of university faculty. The ASEQ addresses an essential gap in institutional assessment, industrial and organizational psychology, and higher education policy by linking structural resources, collegial support, and didactic engagement into a coherent diagnostic framework.

2. Keywords

Academic Setting Evaluation Questionnaire, ASEQ, faculty satisfaction, higher education assessment, organizational climate, academic working conditions, student-faculty relationships, psychometrics, university professors, job satisfaction

3. Authors

The Academic Setting Evaluation Questionnaire was developed and validated by:

• J. Fernandez: Department of Differential Psychology and Work Psychology, Faculty of Psychology, Complutense University of Madrid (Universidad Complutense de Madrid), Spain.
• M. A. Mateo: Department of Methodology of Behavioral Sciences, Faculty of Psychology, Complutense University of Madrid, Spain.

The developmental and empirical validation trajectory of the scale is documented in prominent educational and psychometric journals, notably Educational and Psychological Measurement (1993, 1995) and the European Journal of Psychological Assessment (1995). Direct correspondence regarding institutional adaptations has historically been directed through the psychometric and methodology laboratories at the Complutense University of Madrid.

4. Purpose

The foundational purpose of the Academic Setting Evaluation Questionnaire is to provide an empirically grounded, psychometrically validated tool for diagnosing the multidimensional work environment of academic staff within higher education institutions. Historically, institutional evaluations in university systems disproportionately focused on unidimensional metrics, such as bibliometric research productivity, citation indices, or student evaluations of educational quality. While these external outputs gauge academic productivity and consumer satisfaction, they systematically ignore the underlying ecological, psychosocial, and structural environments that facilitate or hinder faculty well-being, scholarly productivity, and instructional excellence.

From an applied perspective, the ASEQ serves several critical functions across administrative, clinical-occupational, and research domains:

• Institutional Diagnostics and Organizational Auditing: University governing boards, deans, and department chairs utilize the ASEQ to identify operational deficiencies within administrative infrastructures, teacher promotion pathways, and material research support. By pinpointing structural bottlenecks, academic institutions can allocate capital, technical assistance, and administrative resources where they are most urgently needed.
• Occupational Health and Faculty Well-Being: In occupational health psychology, chronic organizational friction—characterized by ambiguous evaluation criteria, poor material support, administrative neglect, and strained interpersonal dynamics—represents a prime precursor to faculty burnout, professional alienation, and attrition. The ASEQ acts as a preventative climate survey, pinpointing occupational stressors before they manifest in psychological distress or decreased retention.
• Comparative and Longitudinal Research: Educational researchers employ the scale to track changes in academic climate across longitudinal institutional reforms (such as the integration of standardized European Higher Education Area frameworks or modern tenure track overhauls). It allows cross-departmental, inter-faculty, and cross-national benchmarking of academic working environments.
• Pedagogical and Didactic Quality Enhancement: By explicitly capturing the relational interface between professors and their students, the ASEQ moves beyond administrative satisfaction to examine pedagogical responsiveness, office hour consultation culture, and student engagement from the instructor’s vantage point.

5. Psychological Construct

The ASEQ evaluates the psychological construct of Perceived Academic Organizational Climate and Work Environment. Rather than treating academic work as an isolated intellectual pursuit, the construct conceptualizes higher education faculty as embedded within an intricate socio-technical system. This system involves material resource availability, bureaucratic evaluation structures, collegial camaraderie, and didactic interactions. In its refined 33-item format, the construct is operationalized across three interrelated but distinct primary dimensions, supplemented by institutional evaluation indicators:

1. Satisfaction with Working Conditions

Comprising 19 items (Items 1–4, 6, 9–20, 22, and 23), this dimension reflects faculty appraisals of both the tangible and systemic resources provided by the institution to fulfill teaching and scholarly mandates. It encompasses:

• Material and Financial Infrastructure: Physical work conditions, laboratory equipment, publication subsidies, and research funding (e.g., Item 1: “The material conditions in which I carry out my work are satisfactory”; Item 2: “Economically it is made possible for me to carry out my research”).
• Compensation and Career Prospects: Perceived fairness of remuneration, career trajectories, and institutional stability (e.g., Item 4: “I consider my university teaching activities to be fairly paid”; Item 17: “The prospects for my work as a faculty member are favorable”).
• Institutional Governance and Procedural Fairness: The transparency and appropriateness of faculty selection, tenure promotion, administrative problem-solving, and research assessment criteria (e.g., Item 9: “Teacher selection systems are suitable”; Item 11: “There are clear criteria for evaluating research activities”; Item 15: “Teacher promotion systems are appropriate”).
• Temporal and Motivational Support: Institutional encouragement, time allocation for scholarship, and Societal appreciation (e.g., Item 13: “Society appreciates the work done by university faculty”; Item 20: “I have sufficient time to carry out my research duties”).

2. Social Climate

Comprising 5 items (Items 5, 7, 8, 26, and 33), this dimension measures the interpersonal, emotional, and collaborative fabric of the academic department. Academic work can often become isolating; this subscale operationalizes the degree of mutual support, effective intellectual communication, and collective research effort among peers (e.g., Item 5: “I feel supported by my colleagues in the activities I carry out as a faculty member”; Item 7: “There is satisfactory academic communication among the members of my department”; Item 26: “The faculty of the department cooperate in the preparation/execution of the department’s research programs”). It taps into the psychological sense of community and horizontal collegiality essential for shared academic governance.

3. Relationship with Students

Comprising 7 items (Items 21, 24, 25, 27–29, and 32), this dimension captures the didactic and instructional climate from the instructor’s perspective. It examines student engagement, instructional receptivity, and the reciprocal pedagogical process (e.g., Item 21: “Students show interest in the subject that I teach”; Item 24: “Students ask about their doubts in the time set aside to receive them”). Furthermore, it measures the professor’s own pedagogical flexibility and willingness to adapt instructional methodologies based on student characteristics and feedback (e.g., Item 27: “Each year I take students’ opinions into account when working out my teaching methodology”; Item 28: “I adapt my teaching to the particular characteristics/demands of each group of students”).

Supplementary Institutional Indicators

The scale retains two independent items (Item 30: “Labor contracts would enable faculty duties to be carried out better”; Item 31: “Some ‘objective’ system of evaluation of each faculty member’s research work is necessary”) that evaluate overarching legislative and contractual perspectives within academic employment. Although these items did not load cleanly onto the primary tripartite factorial framework during empirical extraction, they provide diagnostic value for institutional policy analysis.

6. Theoretical Framework

The Academic Setting Evaluation Questionnaire is anchored in the convergence of several major theories from industrial, organizational, and educational psychology:

1. Herzberg’s Motivator-Hygiene Theory

The dual-factor theory formulated by Frederick Herzberg distinguishes between hygiene factors (extrinsic elements such as salary, institutional policies, working conditions, and job security) and motivator factors (intrinsic elements such as recognition, the work itself, responsibility, and professional advancement). The ASEQ directly operationalizes this distinction within an academic context. The Satisfaction with Working Conditions subscale incorporates fundamental hygiene factors (adequate physical space, administrative support, civil service stability) without which professors experience deep dissatisfaction. Simultaneously, it measures intrinsic motivators, including intellectual autonomy, opportunities for professional advancement, and societal recognition of scientific contributions.

2. The Job Demands-Resources (JD-R) Model

Although formalized broadly in organizational literature by Demerouti, Bakker, and colleagues, the foundational tenets of the Job Demands-Resources model underlie the structural logic of the ASEQ. Academic roles involve high cognitive, pedagogical, and administrative demands. The ASEQ measures the availability of structural and social resources—such as institutional publication funding, departmental collaboration, clear evaluation guidelines, and responsive students—which mitigate the stress associated with relentless academic demands (e.g., time limitations on research, complex teaching duties).

3. Moos’ Social Ecological Model of Work Environments

Rudolf Moos’ conceptualization of human environments identifies three primary domains common to all social settings: (a) Relationship dimensions (the extent to which individuals support and help one another), (b) Personal growth or goal orientation dimensions (the areas in which personal development and self-actualization occur), and (c) System maintenance and system change dimensions (the extent to which the environment is orderly, clear in expectations, and responsive to change). The ASEQ’s three core factors closely mirror Moos’ typology: Social Climate aligns with the Relationship dimension; Relationship with Students addresses the pedagogical Personal Growth and Goal Orientation dimension; and Working Conditions reflects System Maintenance and Procedural Clarity.

7. Validity

The psychometric validity of the ASEQ has been established through extensive empirical research across large academic samples in higher education institutions, particularly through the validation studies conducted by Fernandez and Mateo (1993, 1995):

Construct and Structural Validity

Construct validity was initially assessed on a development cohort of 800 university professors using an exhaustive 69-item version. Exploratory structural analyses indicated six latent components accounting for high degrees of variance, supported by a theta reliability coefficient of 0.98. Subsequent purification reduced the scale to 59 items, and ultimately to the definitive 33-item instrument validated on a randomized sample of 748 university professors from the Complutense University of Madrid. Using a split-sample cross-validation design, the 748 participants were divided into two equal groups of 374 professors. The structural topology extracted from the first calibration sample was replicated in the second validation sample, demonstrating robust cross-sample stability. The extracted three-factor model accounted for approximately 78% of the total scale variance, an exceptionally high proportion in social science measurement.

Convergent and Discriminant Validity

Convergent validity has been corroborated by correlating ASEQ subscale scores with validated measures of classroom psychosocial environment, departmental climate inventories, and objective indices of scholarly productivity. Studies examining convergent patterns (e.g., Fraser & Treagust, 1986; Kremer, 1990; Fernandez et al., 1995) indicate that faculty reporting high scores on Satisfaction with Working Conditions and Social Climate exhibit significantly higher rates of institutional commitment, increased willingness to participate in peer review evaluations, and higher frequency of scientific publications. Discriminant validity is evidenced by the distinct factor loadings of the items; items measuring departmental interpersonal rapport load independently from those measuring administrative selection procedures and student didactic interactions, confirming that faculty distinctly separate interpersonal departmental support from institutional bureaucratic efficiency.

8. Reliability

The ASEQ demonstrates excellent reliability across varied psychometric indicators:

• Internal Consistency: In the landmark cross-validation study by Fernandez and Mateo (1993) comprising 748 university professors, the overall scale demonstrated a Cronbach’s alpha coefficient of 0.90. Subscale-level reliability analyses revealed high internal consistency across all dimensions: the combined working conditions and social climate domains yielded an alpha of 0.89, while the relationship with students subscale yielded an alpha of 0.87.
• Item Homogeneity Indices: Corrected item-total correlations across the retained items consistently exceed 0.40, indicating high item discrimination and construct homogeneity. Items cleanly differentiate between faculty experiencing high versus low institutional satisfaction.
• Cross-Validation Consistency: Reliability coefficients remained stable across both independent halves of the 748-participant cross-validation sample, demonstrating that the instrument’s high internal consistency is not an artifact of sample-specific capitalization on chance.

9. Factor Analysis

The factor structure of the ASEQ was derived and verified through systematic analytical procedures:

Extraction and Rotation Procedures

The primary psychometric investigations utilized Principal Component Analysis (PCA) followed by both orthogonal (Varimax) and oblique rotations to inspect the latent dimensionality. Given that organizational dimensions in academic settings naturally share common variance (e.g., material resources correlate moderately with collegial morale), oblique rotations were evaluated to ensure that structural dimensions were not artificially constrained to orthogonality.

Factor Solution and Variance

A clear three-factor solution met the Kaiser-Guttman criterion (eigenvalues > 1.0) and was further corroborated by Cattell’s scree test inspection. Crucially, this three-factor structure accounted for approximately 78% of the total variance. The distribution of items across the empirical dimensions is structured as follows:

• Factor 1: Satisfaction with Working Conditions (19 items): Items 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, and 23. Factor loadings for these items ranged predominantly from 0.48 to 0.81, capturing institutional resources, compensation, career outlook, and evaluation frameworks.
• Factor 2: Social Climate (5 items): Items 5, 7, 8, 26, and 33. Factor loadings ranged from 0.54 to 0.84, representing peer support, communication, and departmental research collaboration.
• Factor 3: Relationship with Students (7 items): Items 21, 24, 25, 27, 28, 29, and 32. Factor loadings ranged from 0.45 to 0.79, reflecting student interest, consultation habits, and teaching methodology adaptation.
• Unassigned / Experimental Items: Two items—Item 30 (“Labor contracts would enable faculty duties to be carried out better”) and Item 31 (“Some ‘objective’ system of evaluation of each faculty member’s research work is necessary”)—demonstrated complex or sub-threshold loadings (< 0.35) and failed to load uniquely onto any of the three primary factors. They are retained in the questionnaire for qualitative and administrative assessment but are traditionally excluded from primary subscale composite scoring.

10. Instrument / Measurement Tool

• Instrument Name: Academic Setting Evaluation Questionnaire (ASEQ)
• Original Acronym: ASEQ (also referenced in Spanish validation literature as Cuestionario de Evaluación del Ámbito Académico)
• Authors: J. Fernandez and M. A. Mateo (1993, 1995)
• Assessment Type: Self-report institutional evaluation survey
• Target Population: University professors, academic researchers, and higher education teaching personnel
• Item Count: 33 items total (31 items forming three primary subscales, plus 2 structural policy items)
• Response Scale: 7-point Likert agreement scale ranging from:
• 1 = Lowest level of agreement
• 2 = Very low level of agreement
• 3 = Low level of agreement
• 4 = Moderate level of agreement
• 5 = High level of agreement
• 6 = Very high level of agreement
• 7 = Highest level of agreement
• Subscale Breakdown:
  • Satisfaction with Working Conditions: 19 items (Items 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23)
  • Social Climate: 5 items (Items 5, 7, 8, 26, 33)
  • Relationship with Students: 7 items (Items 21, 24, 25, 27, 28, 29, 32)
  • Independent Diagnostic Items: 2 items (Items 30, 31)
• Scoring Procedures: Items are directly scored from 1 to 7 according to the respondent’s indicated level of agreement. Subscale scores are obtained by calculating either the sum or the arithmetic mean of the items comprising each subscale. Higher scores indicate greater satisfaction with academic working conditions, a more positive collegial climate, and more favorable student-faculty pedagogical interactions.
• Administration Time: Approximately 10 to 15 minutes.

11. Permissions & Fee and Test Year

The Academic Setting Evaluation Questionnaire was originally published in 1993 by J. Fernandez and M. A. Mateo in the peer-reviewed journal Educational and Psychological Measurement (Sage Publications). As an academic measurement tool developed for scientific and institutional advancement, the ASEQ is accessible for non-commercial academic research, institutional evaluation, and educational inquiry provided appropriate scholarly citation is given to the original authors and the source publications. Institutional bodies or commercial entities seeking large-scale deployment or integration into commercial human resource auditing platforms should consult the original authors or copyright clearance entities associated with the journal publishers.

12. References

Fernandez, J., & Mateo, M. A. (1993). The development and factorial validation of the Academic Setting Evaluation Questionnaire. Educational and Psychological Measurement, 53(2), 425–435. https://doi.org/10.1177/0013164493053002012

Fernandez, J., Mateo, M. A., & Muñiz, J. (1995). Evaluation of the academic setting in Spain. European Journal of Psychological Assessment, 11(2), 133–137. https://doi.org/10.1027/1015-5759.11.2.133

Fraser, B. J., & Treagust, D. F. (1986). Validity and use of an instrument for assessing classroom psychosocial environment in higher education. Higher Education, 15(1–2), 37–57. https://doi.org/10.1007/BF00138091

Kremer, J. F. (1990). Construct validity of multiple measures in teaching, research, and service and reliability of peer ratings. Journal of Educational Psychology, 82(2), 213–218. https://doi.org/10.1037/0022-0663.82.2.213

Mateo, M. A., & Fernandez, J. (1995). Evaluation of the setting in which university faculty carry out their teaching and research functions: The ASEQ. Educational and Psychological Measurement, 55(2), 329–334. https://doi.org/10.1177/0013164495055002018

13. Items of the Scale