What Does It Mean To Be An Education Expert? Part 2
More of what I wrote in 2013 about Jo Boaler and education experts.
This is the second part of a two-part series. I wrote the original version of these posts in 2013. As I explained in Part 1, I didn’t post it then. This was probably because I feared readers would not understand why I thought Jo Boaler was influential. Now she has become well-known in the political debate about education in the US, and the standard of her scholarship has been widely discussed. So, I will now share what I wrote then.1
The story so far…2
In Part 1, I explained that I had read a chapter from a book by Jo Boaler, a professor of maths education at Stanford and probably the leading figure in the field. This chapter made me reflect on what is required of subject experts in education. I asked two questions:
1) Do you have to be an expert in a subject to be recognised as an expert in teaching that subject?
and
2) Does the work of an education expert have to be well-researched?
I did not think that the chapter showed a strong understanding of mathematics. In answer to my first question, I noted that it seems possible to be recognised as an expert in mathematics teaching without much expertise in mathematics. The chapter also suggested teachers were at fault for not communicating the importance of the Golden Ratio3 to their pupils. However, much of what was written about the Golden Ratio was an urban myth. In answer to my second question, I observed that poor background research appeared acceptable from experts in the teaching of mathematics. In this post, I will ask a couple of further questions about what might be required from experts in teaching subjects such as mathematics.
3) Does an education expert need to have done high-quality empirical research?
Just because someone’s background research is poor, does not mean they haven’t conducted high-quality empirical studies. Jo Boaler is most famous for empirical studies showing that mixed-ability teaching in mathematics is more effective than setting. However, these studies are almost an exemplar of how not to do research. Tiny samples; bizarre metrics; a muddling of quantitative and qualitative research; no attempt at objectivity, and unverifiable statistics are the main features of this work. It would take several blogs to explain everything I can find wrong with these studies. Fortunately, others have done the work. The obvious flaws in her early (and probably most famous study) are pointed out in more detail in these two blog posts:
The case of Amber Hill and Phoenix Park from the Mathematics Education Research blog.
Boaler’s Bias (or BS) from the Education Realist blog.
Boaler looked at a tiny amount of unrepresentative data and then cherry-picked which metrics to use for comparisons. A later study by Boaler, this time conducted in the US, resulted in a brutal critique from three mathematics professors. They also concluded that she had cherry-picked metrics to get the results she wanted. I find it impossible to see any rational argument for the validity of Boaler’s empirical work. To my knowledge, no one has defended her methodology. This brings us to how her work has been defended.
4) Do education experts have to defend the quality of their research?
I have just linked to two blogs and an unpublished article. I have done so because I think those sources contain strong arguments, rather than because the mere fact that somebody has made a criticism means that research is discredited.4 Of course, if you have faith in the authority of educationalists (which is the very thing I am arguing against) and ignore the quality of arguments, then the mere existence of criticism counts for little. After all, anyone can criticise academic work. Academic work should only gain authority because it stands up to criticism. It is here where education ceases to resemble a reputable academic discipline.5 How was Boaler’s work defended against these criticisms?
One of her allies, a professor of maths education at the University of Oxford, came up with this defence:6
...some academics in the USA accused Jo of inventing the data on which her study was based, as if the UK academic standards were so low that she would be awarded a PhD and a thesis prize for fictitious data. .
Boaler herself, when I managed to alert her to the second of the above blog posts came up with this response:7
You may have noticed a theme here. The work of education experts is good because it is the work of education experts, and is recognised by other education experts. Those who criticise it can be ignored because they aren’t education experts.
The mathematicians who wrote the paper criticising her work aren’t anonymous. They have studied how we learn mathematics. Can the substance of their criticisms be ignored? Amazingly, yes. Boaler’s response, can be found here (in the section “Original posting, from October 2012”). It is one of the most truly exceptional things I have read by any academic in any discipline.
Her argument is this:
By criticising her work, her critics have subjected her to persecution. Examples of criticism of her work are listed as evidence of persecution.
It is her political stance that they object to.
One of them has used racist language and made threats that have been investigated by the police.
It is against university policy and federal law for her critics to have identified the source of her data in order to check it.
She hasn’t been found guilty of misconduct.
The criticisms haven’t been peer-reviewed.
The other side to the most sensational allegations can be found here. Some elements of this, particularly the allegation of racism and terrorist threats, suggest a farcical situation. What is striking is that, once again, no defence has been made of Boaler’s actual research.
I am not cherry-picking here. I looked for a proper defence. That was all there was. I have yet to find any attempt by Boaler or her allies to directly defend the validity of her methodology. They only appeal to her personal authority and character and attack that of her critics. It seems possible to be recognised as an educational expert; based on work which can only be defended by appealing to one’s status as an educational expert. Ideas are accepted because they are promoted by experts, and experts are recognised because they are the ones promoting those ideas. Ideas that merely have evidence behind them are seen as far less important.
Don’t trust experts, trust evidence
I have chosen Jo Boaler as the best possible example of an educationalist who teachers have no good reason to listen to, who is nevertheless widely quoted and prominent. But there are other academics8 who have plenty of dodgy research behind them and who are held up as authorities. This does not leave teachers completely helpless. The question of when teachers can trust the experts is addressed in this book, which I recommend. But I raise this because it is often suggested that politicians and policymakers are failing us if they do not listen to “the experts”. It is also often suggested that teachers must follow “the research”. Of course, policymakers and teachers should look at academic research into policy and practice, but it should be to find arguments, not pronouncements. We should recognise those experts who are sources of good arguments, and ignore those who are not.
I have, where possible, tightened up the language and removed errors as the original post (or rather writing intended to become a post) was only a first draft. I have also changed links that no longer worked or were missing. The footnotes I have added are generally new.
In 2013, I had intended the contents of Part 1 and Part 2 to be a single post. This section was written for this post to split the 2013 material into two.
It should also be noted that I am not appealing to the personal authority of the authors of this material.
It seems that Andrew of 2013 had a touching faith in other academic disciplines that Andrew of 2024 cannot endorse.
As I recall, she blocked me for giving an honest answer to her rhetorical question.
I cannot remember who I was referring to here. I could come up with many examples now. My “Two Stars and a Wish” posts often highlight dubious research papers.