Completer effectiveness
(R4.1)Data must address: (a) completer impact in contributing to P-12 student-learning growth AND (b) completer effectiveness in applying professional knowledge, skills, and dispositions.
Measure One (Initial)
In 2011, the Virginia Department of Education released requirements for teacher evaluation. To meet these requirements, evaluations of educators would have to include measures of student academic progress. The Commonwealth developed seven standards for evaluating teaching: professional knowledge, instructional planning, instructional delivery, assessment for/of learning, learning environment, professionalism, and student academic progress. The Commonwealth conducted an alignment analysis of the standards as compared to the INTASC standards and found the Virginia standards to be congruent. These standards were updated and approved by the Virginia board of education in 2021 to separate the measures into eight separate standards. Descriptors of these standards were developed along with 4-point behaviorally anchored rubrics to rate effectiveness. To demonstrate student academic progress, teachers administer a pre-assessment at the beginning of the year, set progress goals for students, monitor progress throughout the year, and administer a post-assessment to determine attainment of goals. School-based evaluators then rate the candidates on their field performance using this rubric. We are using the rating on the student academic progress standard to evaluate our graduates’ impact on students’ learning and development, as well as their effectiveness on applying professional knowledge, skills, and dispositions.
To collect these data, W&M contacted a school division within Virginia. The school division agreed to provide these data anonymized and aggregated. Using a combination of Commonwealth and school division supplied data, we obtained the performance reviews for 17 W&M completers employed by the division during the 2022-2023 academic year. We chose the most recent evaluation from the 2022-2023 academic year from which to obtain data. Teachers are rated on a scale of one to four on eight performance standards, with one representing ineffective and four representing highly effective. No completer scored less than effective, with a positive impact on student growth. Results year on year have changed very little.
| Standard Eight | Average Score | Standard Deviation |
|---|---|---|
| Student Academic Progress: The work of the teacher results in acceptable, measurable, and appropriate student academic progress. |
3.11 | 0.32 |
This teacher evaluation included information about completer effectiveness in applying professional knowledge, skills, and dispositions.
| Standard | Average Score | Standard deviation |
|---|---|---|
| Performance Standard 1: Professional Knowledge The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. |
3.05 | 0.41 |
| Performance Standard 2: Instructional Planning The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students. |
3.1 | 0.47 |
| Performance Standard 3: Instructional Delivery The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs. |
3.22 | 0.43 |
| Performance Standard 4: Assessment of/for Student Learning The teacher systematically gathers, analyzes, and uses all relevant data to measure student progress, guide instructional content and delivery methods, and provide timely feedback to students, parents/caregivers, and other educators, as needed. |
3.05 | 0.24 |
| Performance Standard 5: Learning Environment The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning. |
3.17 | 0.51 |
| Performance Standard 6: Culturally Responsive Teaching and Equitable Practices The teacher demonstrates a commitment to equity and provides instruction and classroom strategies that result in culturally inclusive and responsive learning environments and academic achievement for all students. |
3.15 | 0.38 |
| Performance Standard 7: Professionalism The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning |
3.27 | 0.32 |
In addition, we surveyed our graduates in the spring of 2024 through a partnership with VEAC. The Virginia Education Assessment Collaborative (VEAC) is a growing partnership between Educator Preparation Programs (EPP) in the Commonwealth of Virginia that allows for individual EPPs to obtain information about its graduates and their perception of the adequacy of their preparation. In addition, because this is a collaborative effort, the data allows individual EPPs to benchmark their performance against other institutions in Virginia.
Through VEAC, we supplied the survey to 69 completers, and received 19 responses for a 28% response rate. The results of the administration of this survey are in the table below.
The survey uses a four-point scale to gauge completer’s perceptions of their performance in InTASC and VUPS domains, and a five-point scale to measure the satisfaction of completers with their preparation. The wording of the performance perception items is: “Based on your preparation at William & Mary how would you rate your performance in each of these teaching areas.” Respondent can choose “Exemplary (4),” “Proficient (3),” “Developing/Needs Improvement (2),” or “Unacceptable (1).” To find the average rating, responses are coded, from 1 to 4. Higher values indicate more proficiency.
These results demonstrate that W&M completers perceive that their ability to apply their knowledge is consistent. The lowest score from completers was for item IE, where completers felt they needed additional preparation in Using resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.
Finally, the last item in the VEAC completer survey asks, “Overall, how satisfied are you with your preparation from W&M. Respondents could respond “extremely dissatisfied, somewhat dissatisfied, neither satisfied nor dissatisfied, somewhat satisfied, or extremely satisfied.” To find the average overall satisfaction, responses are coded, from 1 to 5. W&M initial program graduates indicated that they were somewhat to extremely satisfied with the preparation from W&M, with an average score of 4.36.
| VEAC Revised 2022 Item | InTASC | VUPS 2021 | W&M N | W&M Mean (SD) |
|---|---|---|---|---|
|
IA: Demonstrating your understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.* |
1, 2, 4 | 1 | 19 | 3.26(0.65) |
| IB: Planning using state standards, the school's curriculum, engaging and research-based strategies and resources, and data to meet the needs of all students.* | 1, 2, 7, 8 | 2 | 19 | 3.26(0.65) |
| IC: Effectively engaging students in learning by using a variety of research-based instructional strategies in order to meet individual learning needs.* | 1, 2, 8 | 3 | 19 | 3.36(0.68) |
| ID: Systematically gathering, analyzing, and using all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to students, parents, caregivers, and other educators.* | 6, 10 | 4, 8 | 19 | 3(0.88) |
| IE: Using resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.* | 3 | 5 | 19 | 2.89(0.93) |
| IF: Maintaining a commitment to professional ethics, collaborating and communicating effectively, and taking responsibility for and participating in professional growth that results in enhanced student learning.* | 1, 2, 9 | 7 | 19 | 3.36(0.59) |
| IG: Engaging in practices that results in acceptable, measurable, and appropriate student academic progress.* | 6, 7, 8 | 8 | 19 | 3.31(0.67) |
| IH: Using content-aligned and developmentally appropriate instructional technology to enhance student learning.* | 7, 8 | 3 | 19 | 3.26(0.73) |
| IJ: Demonstrating a commitment to equity by providing instructional practices and classroom strategies that result in culturally inclusive and responsive learning environments and academic achievement for all students.* | 2, 3, 8 | 5, 6 | 19 | 3.21 (0.63) |
| IL: Collaborating with the learning community (e.g. school personnel, caregivers, and volunteers) to meet the needs of all learners and contribute to a supportive culture.* | 3, 9, 10 | 7 | 19 | 3(0.57) |
| IM: Using assessment results to inform and adjust practice.* | 6 | 4, 8 | 19 | 3.26(0.65) |
| IN: Engaging in reflection on the impact of their teaching practice and adapts to meet the needs of each learner.* | 9 | 7 | 19 | 3.36(0.68) |
| Overall, how satisfied are you with your preparation from William and Mary?** | - | - | 19 | 4.36(0.50) |
* Items Range from 1-4
** Item Ranges from 1-5