Hattie for over a decade has used the categories of Teacher Training & Education interchangeably. Then around 2018 he changed this category to Initial Teacher Training Programs and then around 2021 changed it to Teacher Qualifications.
The change in category names shows that Hattie has had a lot of trouble defining this category. More concerning is that Hattie mostly cites studies that are about Certification that often happens AFTER Teacher Training. Worse, juxtapose his denigration of Teacher Training with his promotion of Certification (see Teacher Certification).
A brief Summary of Hattie's presentations on Teacher Training/Education
In Hattie's 2008 Nuthall lecture he called Teacher Education a disaster!
Hattie continued to use these slides to 2012 & emphasized the low Effect Size,
Hacke (2010) found a low ES = 0.09 in comparing the achievement of students with NBC to non-NBC teachers.
Hacke (2010) Meta-analysis comparing student outcomes for National Board certified teachers and non National Board certified teachers.
Sparks reports a major problem with how student achievement is represented - a major issue is the use of school or state-level means in place of individual student data (p. 101). For example, in one of the 5 studies used,
Hattie continued to use these slides to 2012 & emphasized the low Effect Size,
"All of you who are in Teacher Education should shake in your boots" (@8mins).
At his 2011 Melbourne Graduate School address more vitriol @22mins,
"Teacher Education, as we most commonly know it, is the most bankrupt institution I know."
Hattie used the following 3 meta-analyses in Visible Learning (2009):
The Visible Learning for Teachers update (2012) - added 2 more
The average ES remained around 0.11 until about 2020 when it appears that all of these studies were removed without explanation.
Also, there are discrepancies in Hattie's referencing. In 2009, he cites Wu, Becker & Kennedy, but in 2012, Qu, Becker & Kennedy.
I tried to find the exact details of these studies in Hattie's bibliography in both 2009 & 2012. However, the only reference relating to authors Wu or Qu was (VL (2009), p. 354),
Likewise, Hattie's bibliography on Sparks (2004) is also wrong.
Visible Learning is littered with these type of errors & also more serious ones. Given the impact of this book on Australian Education, e.g., the HITS, a more thorough checking of Hattie's claims is in order.
These Studies are About Certification Not Necessarily About Teacher Training.
These studies are mostly about certification in the USA context, e.g., Qu & Becker (2003), which was the study referenced in Hattie's bibliography (but not in his table),
"There are three main types of teacher certificates-the traditional teacher certificate, alternative teacher certificate, and emergency teacher certificate. Within each are variations in program activities, program length, and duration of the certification. (p. 4)
Then, Hacke (2010) investigates the most popular traditional certification in USA, the National Board Certified Teachers (NBC). Note that Hattie was part of a small group commissioned to set the NBC up (see Teacher Certification). To gain this certification Teachers need a bachelor degree in Education, plus 3 years teaching experience. So this certificate can be only gained AFTER Teacher Training (details below).
Hacke (2010) found a low ES = 0.09 in comparing the achievement of students with NBC to non-NBC teachers.
The other studies looked at alternate certificates which generally involved a non teaching degree then some form of teacher training in classrooms. ES for comparisons between traditional and alternate certificates were also small.
Hattie then averaged all these ES from different studies into one ES and he rarely discussed the details or nuances of these studies.
This has lead to a number of scholars critiquing Hattie's method, e.g., Thibault (2017) in "Is John Hattie's Visible Learning so Visible?",
"We are entitled to wonder about the representativeness of such results: by wanting to measure an overall effect for subgroups with various characteristics, this effect does not faithfully represent any of the subgroups that it encompasses!
...by combining all the data as well as the particular context that is associated with each study, we eliminate the specificity of each context, which for many give meaning to the study itself!"
Further details of the studies I could find are documented below.
It is hard to understand how Hattie justifies these studies as a proxy for Teacher Training/Education given that he also wrote extensively in Chapter 11 of Visible Learning, about his own study to promote the USA's NBC (see Teacher Certification).
The Studies Hattie Cites in More Detail
This study is clearly about the same certification (NBC) as his own validation study in 2000 & his chapter 11 in Visible Learning. However, Hacke uses Student Achievement to compare NBC vs Non-NBC.
Hacke's major inclusion criteria are: studies must be done in the USA on year 3-12 students and,
"Student achievement is defined as end-of-year or end-of-instruction test score gains on standardised tests in reading and mathematics" (p. 20).
Certification costs between $18,000 - $31,000 per teacher and about 400 hours work (p. 113). One of Hacke's aims was to determine whether the cost and time of certification are worth the effort.
Certification consists of four components:
-written assessment of content knowledge,
-reflection on student work samples,
-video and analysis of teaching practice,
-documented impact and accomplishments as a teaching professional.
The certification is expensive and takes a lot of time to prepare, so many experienced teachers do not go through the process. Also, school districts in poorer areas do not require certification as there is major turnover and a shortage of teachers.
The low effect size indicates NO.
NBC is a teaching certificate for teachers with a teaching degree plus 3 years experience.
-written assessment of content knowledge,
-reflection on student work samples,
-video and analysis of teaching practice,
-documented impact and accomplishments as a teaching professional.
Hacke (2010) pinpoints the central issue to all of Hattie's work,
"identifying effective teachers hinges on how it is defined and measured" (p. 32).
Sparks (2004) The Effect of Teacher Certification on Student Achievement. As stated above, Hattie incorrectly references a different Sparks study.
This study compares certified versus alternatively certified or non-certified teachers. Sparks reports a range of correlational effect sizes, she advises against averaging them,
"The disparate definitions of certification do not permit effect size estimates to be combined" (p. 89).Yet, Hattie goes ahead to get an average of 0.11 in VL, then 0.12 in his update.
Sparks reports a major problem with how student achievement is represented - a major issue is the use of school or state-level means in place of individual student data (p. 101). For example, in one of the 5 studies used,
"student achievement was represented by a single state-level mean and paired with the proportion of certified teachers in the state" (p. 102).Kelley & Camilli (2007) is a study on early childhood (pre-school) teachers, it does compare Bachelor degrees with Diplomas & finds not a lot of difference or low ES in comparison. So this is a reasonable study to cite in the category of Teacher Training.
No comments:
Post a Comment