Got a comment?

Send your ideas to strategicplan@ iastate.edu.

Report Brief

Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics

PURPOSE

This report recommends a set of strategies to evaluate undergraduate teaching and learning in science, technology, engineering, and mathematics (STEM). It is based on a study conducted by a National Research Council (NRC) committee charged with synthesizing relevant research in pedagogy and practice as a basis for developing resources to help postsecondary STEM faculty and administrators evaluate and reward effective teaching. The committee's principal goal was to determine whether fair and objective methods exist for the evaluation of teaching and learning, and if so, how such methods could be used as a basis for the professional advancement of faculty. The committee found that many such methods exist, and that their utility deserves wider appreciation and application in the evaluation of both individuals and departments.

KEY POINTS FROM THE REPORT

• The committee found that summative evaluations of teaching, such as those used in some faculty promotion and tenure decisions, often do not rely on evidence of student learning, and this relationship needs to be strengthened and formalized.
• The committee also found that formative evaluations (e.g., ongoing informal feedback from students and colleagues) can serve several important educational goals:
  1. Coupling candid teaching evaluation with opportunities for ongoing professional development
  2. Supporting faculty who wish to explore scholarship of teaching and learning.
  3. Applying such formative evaluation techniques to departmental programs, not only to individual faculty.
• Four fundamental premises guided the committee's deliberations:
  1. Effective postsecondary teaching in science, mathematics, and technology should be available to all students, regardless of their major.
  2. The design of curricula and the evaluation of teaching and learning should be collective responsibilities of faculty in individual departments or, where appropriate, through interdepartmental arrangements.
  3. Scholarly activities that focus on improving teaching and learning should be recognized as bona fide endeavors that are equivalent to other scholarly pursuits. Scholarship devoted to improving teaching effectiveness and learning should be accorded the same administrative and collegial support that is available for efforts to improve other research and service endeavors.
  4. Faculty who are expected to work with undergraduates should be given support and mentoring in teaching throughout their careers; hiring practices should provide a first opportunity to signal institutions' teaching values and expectations of faculty.
• The committee found that most faculty who teach undergraduates in the STEM disciplines have received little formal training in teaching techniques, in assessing student learning, or in evaluating teaching effectiveness. Formal programs aimed at improving teaching are still rare.
• A firm commitment to open intradepartmental communication about teaching effectiveness is therefore critical to any convincing evaluation of teaching based on these premises.
• The research literature suggests that some combination of the following kinds of formative and summative evidence about student learning can be helpful in evaluating and improving a faculty member's teaching:
  • Departmental and other colleagues can provide informed input about teaching effectiveness through direct observation, analysis of course content materials, or information about the instructor's effectiveness in service and interdisciplinary courses.
  • Undergraduates and graduate teaching assistants could offer useful information based on their experiences in the instructor's courses and laboratories, the instructor's supervision of research, and the quality of academic advising.
  • The faculty member being evaluated could provide self-assessment of his or her teaching strengths and areas for improvement; this assessment could be compared with the other independent evidence.
  • The instructor's willingness to seek external support to improve teaching and learning also is evidence of her or his commitment to effective undergraduate teaching.
• Effective evaluation also emerges from a combination of sources of evidence.
  • Current students, those who had taken a course in previous years, and graduating seniors and alumni could provide evidence about the instructor's role in their learning.
  • Graduate teaching assistants could discuss the instructor's approaches to teaching, levels of interactions with students, and the mentoring that they receive in improving their own teaching skills.
  • Departmental and other faculty colleagues, both from within and outside the institution, could evaluate the currency of the materials the instructor presents and his or her level of participation and leadership in improving undergraduate education.
  • The faculty member being evaluated can provide critical information about his or her teaching challenges and successes through self-reflection and other evidence of effective teaching, student learning, and professional growth.
  • Institutional data and records offer insights about changes in enrollments in a faculty member's courses over time, the percentage of students who drop the instructor's courses, and the number of students who go on to take additional courses in the discipline and related subject areas.
  • Each of these criteria is subject to multiple interpretations and should be reviewed with care.
• A central idea behind formative evaluation of teaching and learning is a two-way feedback system known as "outcomes assessment." Faculty need to set clear goals for their students and ascertain whether students are meeting these goals throughout the course.

Overview of Research on Effective Assessment of Student Learning

• Although assessments used in various contexts and for differing purposes often look quite different, they share common principles. Assessment is always a process of reasoning from evidence. Moreover, assessment is imprecise to some degree. Assessment results are only estimates of what a person knows and can do. It is essential to recognize that one type of assessment is not appropriate for measuring learning in all students. Multiple measures provide a more robust picture of what an individual has learned.
• Educational assessment does not exist in isolation. It must be aligned with curriculum and instruction if it is to support learning.
• The design of high-quality assessments is a complex process that involves numerous iterative and interdependent components. Decisions made at a later stage of the design process can affect those occurring at an earlier stage. Thus, as faculty develop assessments of student learning, they must often revisits their choices of questions and approaches and refine their designs.
• For assessment to be effective, students must understand and share the goals for learning that are assessed.

Characterizing and Mobilizing Effective Undergraduate Teaching

• Zimpher (1998) predicted:
  1. Teaching will be more public than it ever has before.
  2. The nature and quality of assessment will change.
  3. Evaluation and documentation of teaching will change.
  4. Teaching will become technologically enabled.
  5. Content transmission will not be the focus of teaching.
  6. Curriculum and program design will be inseparable from teaching and learning.
  7. Diversity will be seen as asset-based. Higher education will realize that all benefit when different perspectives and cultures are included.
  8. Different pedagogies that students have experienced prior to college will change their expectations about good teaching.
  9. Higher education facilities will have to look different. Rooms will have to be flexible to accommodate the new pedagogies and they will have to be technologically sophisticated.
  10. A new scholarship of teaching will occur.
• Characteristics of Effective Teaching:
  1. Knowledge of subject matter.
  2. Skill, experience, and creativity with a range of appropriate pedagogies and technologies.
  3. Understanding of and skill in using appropriate assessment practices.
  4. Professional interactions with students within and beyond the classroom.
  5. Involvement with and contributions to one's profession in enhancing teaching and learning.
• Nine Principles of Good Practice for Assessing Student Learning (Astin, 1996)
  1. The assessment of student learning begins with educational values.
  2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
  3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
  4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
  5. Assessment works best when it is ongoing not episodic.
  6. Assessment fosters wider improvement when representatives from across the educational community are involved.
  7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
  8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
  9. Through assessment, educators meet responsibilities to students and to the public.

RECOMMENDATION

Overall Recommendations

1. Teaching effectiveness should be judged by the quality and extent of student learning.
2. Scholarly activities that focus on improving teaching and learning should be recognized and regarded as a bona fide scholarly endeavor and accorded the types of institutional supports aimed at improving scholarship generally.
3. Valid summative assessments of teaching should not only rely on student evaluations, but should include peer reviews and teaching portfolios used for promotion, tenure, and post-tenure review.
4. Individual faculty - beginners as well as more experienced teachers - and their departments should be rewarded for consistent improvement of learning by both major and nonmajor students. All teacher related activities - such as grading, reporting of grades, curriculum development, training of teaching assistants, and related committee work - should be included in evaluation systems adopted for faculty rewards.
5. Faculty should accept the obligation to improve their teaching skills as part of their personal commitment to professional excellence.

Recommendations for Presidents, Overseeing Boards, and Academic Officers

1. Quality teaching and effective learning should be highly ranked institutional priorities.
2. Campus-wide or disciplinary-focused centers for teaching and learning should be tasked with providing faculty with opportunities for ongoing professional development that include understanding how people learn, how to improve current instruction through student feedback, and how educational research can be translated into improved teaching practice.
3. At least one senior university-level administrator should be assigned responsibility for encouraging departmental faculty to adopt effective means (as proven by research) to improve instruction.
4. Faculty who have excelled in teaching should be publicly recognized and rewarded.
5. Faculty should be encouraged to develop curricula that transcend disciplinary boundaries, through a combination of incentives (including funding), expectations of accountability, and development of standards for disciplinary and interdisciplinary teaching.
6. Willingness to emphasize student learning and to make allocations of departmental resources in support of teaching should be an essential requirement in appointing deans, department chairs, and similar administrative positions.
7. Graduate school deans should require that departments employ graduate students in fulfilling their teaching mission should show evidence that their faculties are effectively mentoring graduate teaching assistants and advising them about their duties to undergraduate students.

Recommendations for Deans, Department Chairs, and Peer Evaluators

1. Departments should periodically review a departmental mission statement that includes appropriate emphasis on teaching and student learning. These reviews should address not only the major curriculum, but also service offerings - such as courses designed for nonmajors and prospective teachers.
2. Individual faculty members should be expected to contribute to a balanced program of undergraduate teaching. Participation of established faculty in lower-division, introductory, and general education courses should be encouraged.
3. Departments should contribute to campus-wide awareness of the premium placed on improved teaching.
4. In addition to numerical data from end-of-course student evaluations and on participation in specific courses, effective peer reviews of teaching should provide a subjective assessment of a faculty member's commitment to quality teaching. Generally this should include evaluation of a faculty member's knowledge and enthusiasm for the subject matter; familiarity with a range of appropriate pedagogical methods; skills in using appropriate tests and laboratory experiences; quality of advising and other professional interactions with students within and beyond the classroom; and active scholarly commitment to enhancing top-quality teaching and learning.
5. Department heads, in submitting personnel recommendations, should provide separate ratings of teaching, research, and service, each with supporting evidence, as key components of their overall rating and recommendation.
6. Normal departmental professional development activity should include informing faculty about research findings that can improve student learning.
7. As appropriate for achieving departmental goals, departments should provide funds to faculty to enhance teaching skills and knowledge and encourage them to undertake or rely upon educational research that links teaching strategies causally to student learning.
8. Departments should recognize that in the course of their careers, some faculty may shift the balance of their departmental obligations to place a greater emphasis on instruction or educational leadership. These shifts should be supported, consistent with a departmental mission, so long as active engagement with innovative teaching is being addressed.

Recommendations for Granting and Accrediting Agencies, Research Sponsors, and Professional Societies

1. Funding agencies should support programs to enable an integrated network of national and campus-based centers for teaching and learning. An important goal of such a network is to conduct and disseminate research on approaches that enhance teaching and learning in STEM.
2. Funding agencies and research sponsors should undertake a self-examination by convening expert panels to examine whether agency policies might inadvertently compromise a faculty member's commitment to quality undergraduate teaching.
3. Accreditation agencies and boards should revise policies to emphasize quality undergraduate learning as a primary criterion for program accreditation.
4. Professional societies should offer opportunities to discuss undergraduate education issues during annual and regional meetings.
5. Professional societies should encourage publication of peer-reviewed articles in their general or specialized journals on evolving educational issues in STEM.

Reference

Fox, M.A., & Hackerman, N. (Eds.). (2003). Evaluating and improving undergraduate teaching in science, technology, engineering, and mathematics. National Research Council of the Academies, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press.

Submitted by R.M. Johnson, May 2004. This is a report summary and excerpts are quoted directly from the text.