Completer effectiveness
(R4.1) Data must address: (a) completer impact in contributing to P-12 student-learning growth AND (b) completer effectiveness in applying professional knowledge, skills, and dispositions.
Measure One (Initial)
In 2011, the Virginia Department of Education released requirements for teacher evaluation. To meet these requirements, evaluations of educators would have to include measures of student academic progress. The Commonwealth developed seven standards for evaluating teaching: professional knowledge, instructional planning, instructional delivery, assessment for/of learning, learning environment, professionalism, and student academic progress. The Commonwealth conducted an alignment analysis of the standards as compared to the INTASC standards and found the Virginia standards to be congruent. Descriptors of these standards were developed along with 4-point behaviorally anchored rubrics to rate effectiveness. To demonstrate student academic progress, teachers administer a pre-assessment at the beginning of the year, set progress goals for students, monitor progress throughout the year, and administer a post-assessment to determine attainment of goals. School-based evaluators then rate the candidates on their field performance using this rubric. We are using the rating on the student academic progress standard to evaluate our graduates’ impact on students’ learning and development, as well as their effectiveness on applying professional knowledge, skills, and dispositions.
To collect these data, W&M contacted a school division within Virginia. The school division agreed to provide these data anonymized and aggregated. Using a combination of Commonwealth and school division supplied data, we obtained the performance reviews for 20 W&M completers employed by the division during the 2021-2022 academic year. We chose the most recent evaluation from the 2022-2023 academic year from which to obtain data. Teachers are rated on a scale of one to four on eight performance standards, with one representing ineffective and four representing highly effective. No completer scored less than effective, with a positive impact on student growth. Results year on year have changed very little.
| Standard | Average Score | Standard Deviation |
|---|---|---|
| Student Academic Progress: The work of the teacher results in acceptable, measurable, and appropriate student academic progress. |
3.05 | 0.22 |
This teacher evaluation included information about completer effectiveness in applying professional knowledge, skills, and dispositions.
| Standard | Average Score | Standard deviation |
|---|---|---|
| Performance Standard 1: Professional Knowledge The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences. |
3.05 | 0.39 |
| Performance Standard 2: Instructional Planning The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students. |
3.1 | 0.45 |
| Performance Standard 3: Instructional Delivery The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs. |
3.2 | 0.41 |
| Performance Standard 4: Assessment of and for Student Learning The teacher systematically gathers, analyzes, and uses all relevant data to measure student progress, guide instructional content and delivery methods, and provide timely feedback to students, parents/caregivers, and other educators, as needed. |
3.05 | 0.22 |
| Performance Standard 5: Learning Environment The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning. |
3.15 | 0.49 |
| Performance Standard 6: Culturally Responsive Teaching and Equitable Practices The teacher demonstrates a commitment to equity and provides instruction and classroom strategies that result in culturally inclusive and responsive learning environments and academic achievement for all students. |
3.25 | 0.44 |
| Performance Standard 7: Professionalism The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning. |
3.1 | 0.31 |
In addition, we surveyed our graduates in the spring of 2023 through a partnership with VEAC. The Virginia Education Assessment Collaborative (VEAC) is a growing partnership between Educator Preparation Programs (EPP) in the Commonwealth of Virginia that allows for individual EPPs to obtain information about its graduates and their perception of the adequacy of their preparation. In addition, because this is a collaborative effort, the data allows individual EPPs to benchmark their performance against other institutions in Virginia.
Through VEAC, we supplied the survey to 40 completers, and received 13 responses for a 33% response rate. The results of the administration of this survey are in the table below.
The survey uses a four-point scale to gauge completer’s perceptions of their performance in InTASC and VUPS domains, and a five-point scale to measure the satisfaction of completers with their preparation. The wording of the performance perception items is: “Based on your preparation at William & Mary how would you rate your performance in each of these teaching areas.” Respondent can choose “Exemplary (4),” “Proficient (3),” “Developing/Needs Improvement (2),” or “Unacceptable (1).” To find the average rating, responses are coded, from 1 to 4. Higher values indicate more proficiency.
These results demonstrate that W&M completers perceive that their ability to apply their knowledge is consistent, and statistically equivalent to the perceptions of completers for other Commonwealth institutions, with the exception of item IG: Engaging in practices that results in acceptable, measurable, and appropriate student academic progress, for which, W&M completers score statistically higher than other institutions in Virginia.
Graduates rated their performance for questions IA, IF, and IG the highest, with a rating of 3.62. Graduates rated their performance for question IL the lowest, with a rating of 3.17. No rating average fell below three, indicating that survey respondents felt that their performance was at a minimum proficient.
Finally, the last item in the VEAC completer survey asks, “Overall, how satisfied are you with your preparation from W&M. Respondents could respond “extremely dissatisfied, somewhat dissatisfied, neither satisfied nor dissatisfied, somewhat satisfied, or extremely satisfied.” To find the average overall satisfaction, responses are coded, from 1 to 5. W&M initial program graduates indicated that they were somewhat to extremely satisfied with the preparation from W&M, with an average score of 4.38, a value that was not statistically different from other institutions in the Commonwealth.
| VEAC Revised 2022 Item | InTASC | VUPS 2021 | VEAC N | VEAC Mean (SD) | W&M N | W&M Mean (SD) | p-value |
|---|---|---|---|---|---|---|---|
|
IA: Demonstrating your understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.* |
1, 2, 4 | 1 | 1,079 | 3.30, (0.60) | 13 | 3.62, (0.51) | 0.050 |
| IB: Planning using state standards, the school's curriculum, engaging and research-based strategies and resources, and data to meet the needs of all students.* | 1, 2, 7, 8 | 2 | 1,070 | 3.24, (0.63) | 13 | 3.23, (0.60) | 0.92 |
| IC: Effectively engaging students in learning by using a variety of research-based instructional strategies in order to meet individual learning needs.* | 1, 2, 8 | 3 | 1,076 | 3.23, (0.65) | 13 | 3.46, (0.52) | 0.23 |
| ID: Systematically gathering, analyzing, and using all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to students, parents, caregivers, and other educators.* | 6, 10 | 4, 8 | 1,071 | 3.06, (0.70) | 13 | 3.23, (0.73) | 0.38 |
| IE: Using resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.* | 3 | 5 | 1,080 | 3.38, (0.68) | 13 | 3.38, (0.65) | 0.96 |
| IF: Maintaining a commitment to professional ethics, collaborating and communicating effectively, and taking responsibility for and participating in professional growth that results in enhanced student learning.* | 1, 2, 9 | 7 | 1,081 | 3.49, (0.59) | 13 | 3.62, (0.51) | 0.51 |
| IG: Engaging in practices that results in acceptable, measurable, and appropriate student academic progress.* | 6, 7, 8 | 8 | 1,079 | 3.28, (0.61) | 13 | 3.62, (0.51) | 0.046 |
| IH: Using content-aligned and developmentally appropriate instructional technology to enhance student learning.* | 7, 8 | 3 | 1,076 | 3.28, (0.63) | 13 | 3.54, (0.66) | 0.11 |
| IJ: Demonstrating a commitment to equity by providing instructional practices and classroom strategies that result in culturally inclusive and responsive learning environments and academic achievement for all students.* | 2, 3, 8 | 5, 6 | 1,078 | 3.38, (0.63) | 13 | 3.54, (0.66) | 0.31 |
| IL: Collaborating with the learning community (e.g. school personnel, caregivers, and volunteers) to meet the needs of all learners and contribute to a supportive culture.* | 3, 9, 10 | 7 | 1,075 | 3.24, (0.68) | 12 | 3.17, (0.83) | 0.80 |
| IM: Using assessment results to inform and adjust practice.* | 6 | 4, 8 | 1,070 | 3.19, (0.67) | 13 | 3.38, (0.51) | 0.35 |
| IN: Engaging in reflection on the impact of their teaching practice and adapts to meet the needs of each learner.* | 9 | 7 | 1,076 | 3.33, (0.62) | 13 | 3.31, (0.48) | 0.70 |
| Overall, how satisfied are you with your preparation from William and Mary?** | - | - | 1,089 | 4.43, (0.82) | 13 | 4.38, (0.51) | 0.34 |
* Items Range from 1-4
** Item Ranges from 1-5
