Completer effectiveness
(R4.1) Data must address: (a) completer impact in contributing to P-12 student-learning growth AND (b) completer effectiveness in applying professional knowledge, skills, and dispositions.
Measure One (Initial)
In 2011, the Virginia Department of Education released requirements for teacher evaluation. To meet these requirements, evaluations of educators would have to include measures of student academic progress. The Commonwealth developed seven standards for evaluating teaching: professional knowledge, instructional planning, instructional delivery, assessment for/of learning, learning environment, professionalism, and student academic progress. The Commonwealth conducted an alignment analysis of the standards as compared to the INTASC standards and found the Virginia standards to be congruent. Descriptors of these standards were developed along with 4-point behaviorally anchored rubrics to rate effectiveness. To demonstrate student academic progress, teachers administer a pre-assessment at the beginning of the year, set progress goals for students, monitor progress throughout the year, and administer a post-assessment to determine attainment of goals. School-based evaluators then rate the candidates on their field performance using this rubric. We are using the rating on the student academic progress standard to evaluate our graduates’ impact on students’ learning and development, as well as their effectiveness on applying professional knowledge, skills, and dispositions.
To collect these data, W&M contacted a school division within Virginia. The school division agreed to provide these data anonymized and aggregated. Using a combination of Commonwealth and school division supplied data, we obtained the performance reviews for 18 W&M completers employed by the division during the 2020-2021 academic year. We chose the most recent evaluation from the 2021-2022 academic year from which to obtain data. Teachers are rated on a scale of one to four on seven performance standards, with a one representing unacceptable and a four representing exemplary. No completer scored unacceptable in positive impact on student growth. Results year on year are changed very little.
| Standard | Average Score | Standard Deviation |
|---|---|---|
| Student Academic Progress: The work of the teacher results in acceptable, measurable, and appropriate student academic progress. |
3 | 0.524 |
This teacher evaluation included information about completer effectiveness in apply professional knowledge, skills, and dispositions.
| Standard | Average Score | Standard deviation |
|---|---|---|
| Performance Standard 1: Professional Knowledge The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. |
3 | 0.40 |
| Performance Standard 2: Instructional Planning The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students. |
3 | 0.47 |
| Performance Standard 3: Instructional Delivery The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs. |
3 | 0.40 |
| Performance Standard 4: Assessment of and for Student Learning The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year. |
3 | 0.47 |
| Performance Standard 5: Learning Environment The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning. |
3 | 0.5 |
| Performance Standard 6: Professionalism The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning. |
3 | 0.57 |
In addition, we surveyed our graduates in the spring of 2022 and asked their perceptions of their own impact on student learning and development. This represents a repeat of data reported last year. We are currently changing the manner in which we collect data and will now be part of the VEAC group collecting data across Virginia. VEAC different timing for the collection of data and we will report the data obtained this spring in the next annual report. We administered a survey to 117 program completers. The completers’ names were derived from a list provided by the Commonwealth of Virginia from the initial teacher preparation programs for the years 2019 through 2021, as well as from a list of alumni that we have developed through outreach efforts. Of these 117 completers, we obtained 29 responses for a response rate of 25%. Program completers responded on a scale of one to four, with a one indicating No Effect and a four indicating a Significant Effect. Mean ratings from the survey indicate that our graduates perceive that they have a positive effect on academic/scholastic achievement (M =3.72, SD = 0.46), a positive effect on cognitive skills/intellectual development (M=3.52, SD=0.51), a positive effect on social/emotional development (M=3.48, SD=0.51), and a positive effect on psychomotor/physical development (M=2.9, SD =1.08).
| Perceived Effect on Student Learning and Development | Average Score | Standard Deviation |
|---|---|---|
| Academic/Scholastic Achievement | 3.72 | 0.45 |
| Cognitive Skills/Intellectual Development | 3.52 | 0.51 |
| Social/Emotional Development | 3.48 | 0.51 |
| Psychomotor/Physical Development | 2.9 | 1.08 |
Responses to these questions have been consistent year on year. Alumni perception of student impact remains high for academic, cognitive, and social emotional development. The lower score for psychomotor development is consistent with previous years’ results and represents the diversity of respondent subject areas. That is, if students are in the higher-grade levels, their impact on psychomotor development will be limited.
