John Hattie: I’m a statistician, I’m not a theoretician

The text you are about to read is an interview with the key figure behind the concept of ‘visible learning’, Professor John Hattie. Hattie’s books, ‘Visible Learning’ (2009) and ‘Visible Learning for Teachers’ (2012), as well as the concept and basic ideas regarding ‘visible learning’, and the programme Visible Learning (distributed under licence from Cognition Education Limited), have a great impact on everyday school life in many different countries. Visible learning as an idea and programme was developed in New Zealand and Australia, and is currently being implemented in schools and municipalities in 23 countries around the world. The ideas and programme have been vigorously discussed, both in international research and in local settings like Denmark, where I am located. Some of the discussions accept the basic assumptions and ambitions of visible learning regarding efficiency, while others dismiss them as almost morally wrong. These two positions are strongly defended by teachers and researchers alike. For instance, from a teaching practice point of view it has been argued that learning is not visible and that the risk of learning targets is that they may demotivate the pupils (e.g. Frederiksen, 2015). In education research, some of the discussions have focused on whether the factors pointed out are actually the ones with the biggest effect (e.g. Snook, O’Neill, Clark, O’Neill, & Openshaw, 2009), whether it makes sense to drive educational policy decisions based on standardized effect size studies (e.g. Simpson, 2017), and whether efficiency is the lens through which education ought to be viewed (e.g. Biesta, 2010). I am an educational researcher myself, and I find it important to go beyond both the efficiency discussion and the more normative discussions in order to see how visible learning works and what it does. My interest is parallel to current studies of the translation processes and performative effects of educational programmes (e.g. Staunæs, 2018), and of the performative effects of non-state policy actors like edu-businesses (e.g. Lingard, 2018). I have struggled with some very basic questions while reading ‘Visible Learning for Teachers’ in particular and while observing various forms of visible learning practices in Danish schools and municipalities. These questions relate to the meaning of the core concepts, the theoretical points of departure for visible learning and translations into teaching and political practices. Consequently, the interview given here is neither a research article nor a journalistic interview, but perhaps more of a conversation between researchers. It may be a little naive to pose these questions to John Hattie himself, as no single person controls the reception and translation of educational ideas and programmes. You could say that it is difficult to say where ‘John Hattie’ begins, and where he, his thoughts, his authority and responsibility end. Nonetheless, I find it interesting to gain insight into his reflections and views on his work and his own role. After all, Hattie is actively involved in all parts of visible learning: the production of data, the analysis, the development of concepts and teaching programmes, dissemination (through contracts with companies and by giving presentations himself), the evaluation of the effects of the programme and the corrections of the programme and scripts. John Hattie works constantly to support visible learning as a globally valid concept. This makes it interesting to learn more about his awareness of context and his view of concepts, and in the interview I have asked for his reflections on the following questions:


Introduction
The text you are about to read is an interview with the key figure behind the concept of 'visible learning', Professor John Hattie. Hattie's books, 'Visible Learning' (2009) and 'Visible Learning for Teachers ' (2012), as well as the concept and basic ideas regarding 'visible learning', and the programme Visible Learning plus (distributed under licence from Cognition Education Limited), have a great impact on everyday school life in many different countries. Visible learning as an idea and programme was developed in New Zealand and Australia, and is currently being implemented in schools and municipalities in 23 countries around the world.
The ideas and programme have been vigorously discussed, both in international research and in local settings like Denmark, where I am located. Some of the discussions accept the basic assumptions and ambitions of visible learning regarding efficiency, while others dismiss them as almost morally wrong. These two positions are strongly defended by teachers and researchers alike. For instance, from a teaching practice point of view it has been argued that learning is not visible and that the risk of learning targets is that they may demotivate the pupils (e.g. Frederiksen, 2015). In education research, some of the discussions have focused on whether the factors pointed out are actually the ones with the biggest effect (e.g. Snook, O'Neill, Clark, O'Neill, & Openshaw, 2009), whether it makes sense to drive educational policy decisions based on standardized effect size studies (e.g. Simpson, 2017), and whether efficiency is the lens through which education ought to be viewed (e.g. Biesta, 2010). I am an educational researcher myself, and I find it important to go beyond both the efficiency discussion and the more normative discussions in order to see how visible learning works and what it does. My interest is parallel to current studies of the translation processes and performative effects of educational programmes (e.g. Staunaes, 2018), and of the performative effects of non-state policy actors like edu-businesses (e.g. Lingard, 2018).
I have struggled with some very basic questions while reading 'Visible Learning for Teachers' in particular and while observing various forms of visible learning practices in Danish schools and municipalities. These questions relate to the meaning of the core concepts, the theoretical points of departure for visible learning and translations into teaching and political practices. Consequently, the interview given here is neither a research article nor a journalistic interview, but perhaps more of a conversation between researchers.
It may be a little naive to pose these questions to John Hattie himself, as no single person controls the reception and translation of educational ideas and programmes. You could say that it is difficult to say where 'John Hattie' begins, and where he, his thoughts, his authority and responsibility end. Nonetheless, I find it interesting to gain insight into his reflections and views on his work and his own role. After all, Hattie is actively involved in all parts of visible learning: the production of data, the analysis, the development of concepts and teaching programmes, dissemination (through contracts with companies and by giving presentations himself), the evaluation of the effects of the programme and the corrections of the programme and scripts. John Hattie works constantly to support visible learning as a globally valid concept. This makes it interesting to learn more about his awareness of context and his view of concepts, and in the interview I have asked for his reflections on the following questions: I spent three months in Australia and New Zealand in the first part of 2017. One of the reasons for this trip was visible learning: I wanted to understand the society and educational system in which the programme is rooted, and I wanted to talk to John Hattie. A colleague in Australia read one of my emails to Hattie, and she was somewhat shocked by my approach because I stated quite frankly that I was interested in confronting him with some of the critiques of his work as they appear in a Danish context. My ambition was to be honest, and fortunately Hattie agreed to answer my questions. He has subsequently read this text and elaborated on some of the themes, especially on the concept of evidence. I had also hoped to follow the processing of Danish data carried out by the company Cognition Education, which has a license agreement with John Hattie. They collect data in Danish schools through school visits and questionnaires, and this data is processed by employees in Auckland, New Zealand. I would have liked to make observations and interview these employees about their work and reflections on data processing while I was in Auckland, but this was unfortunately not possible. John Hattie and I did not manage to meet while I was in Australia, and we had to conduct the interview via Skype between Brisbane and the south coast of Australia. I have abbreviated and edited the transcript in order to make it readable, and I have endeavoured to be as loyal and precise as possible.

Scaling up and translation
Hanne Knudsen (HK): Visible learning has had a big impact in Danish schools. I am curious about your view on this impact. Why do you think it has this big impact in a Danish setting? John Hattie (JH): I wish I knew the answer. When I wrote the book 10 years ago, I never imagined it would have this kind of uptake. As you probably know, I am a measurement person, I am a statistician, and the work with the concept and implementation was kind of a hobby I did on the side. I am not very good talking in schools. I am happy to go out and talk in schools, but then nothing happens. So when the book came out, I asked my team in New Zealand to demonstrate how it could make a difference in schools, focusing on the impact it has on the students. It fascinates me that in education we do not have a literature on scaling up programmes. I can only find five articles that talk about scaling up success. The other thing we don't have in education is many models on implementation. We have great policies, we have plenty of research, we have wonderful ideas, but we know that implementation is one of the biggest problems. So I asked the team to focus on this. If you ask me why Denmark, I really don't know. Of 23 countries, Denmark is one of the larger ones to implement visible learning. I know a book came out in Danish on why visible learning wouldn't work in Denmark which I find absolutely amusing because it is working (Bjerre et al. 2017, HK). When I look at the data from Denmark, I can see big effects on students. I know it is not easy to implement visible learning, because it focuses on listening to the students rather than listening to the teachers. I know there is a lot of resistance. Some teachers don't like it, because their argument is that they have been teaching like this for 30 years. Why should they change? And in some cases, VL does not ask them to change but gives permission to keep having their high impact. Why do you think it has caught up?
HK: I don't know, I don't know really. When you were in Denmark in November, you said that you used to produce a lot of knowledge, but it didn't really catch on until you surrounded yourself with translators. I find this notion of translators interesting: Who are these translators, and what do they translate? Is it between various languages, from research to practice or? JH: When I was in New Zealand, I was on the board of a commercial company, and I talked to the chairman about our problems with scaling up education. It turns out he was a lawyer specialized in licensing and franchising, and he said: 'Can you imagine setting up a coffee shop and not worrying on day one about how you spread the good word, how you increase your customers?' In economics, scaling up is huge business. I looked at the licensing model, and in licensing there are two things you worry about: One is quality control, and the other is to make sure that if your licensee does not deliver on quality control, you have to have very tight legal contracts so they can be fired overnight. I spend a lot of time worrying about quality control, and now I have 23 different licensees around the world. It's all done through Cognition in Auckland. We develop the concepts together. They develop the materials, and then we trial it. We trial it about five times, we improve it, we look at the impact on the students, and then, when we get it together, we write up the whole workshop. And everywhere around the world, including in Denmark, they have to deliver that workshop, almost word for wordbut allowing for local adaption. We hire people to deliver it, and only one in five passes. That is because almost every teacher wants to get up and talk about their stories, their anecdotes and their classrooms. We will not allow that, because as soon as you allow that, you legitimize every teacher in the room talking about their war stories, their views, their kids. We concentrate entirely on the impact on the students: How would you know that the programme has been implemented in high fidelity? It does mean that I am in control.
If you go to a workshop, you'll never know that it is scripted. To the word. You would never know it, and that's part of the success. HK: It must be important to you, this quality check.
You do it through scripting. How do you make sure that there is a quality in what the various consultants do? JH: I am a great fan of the notion of translation.
There has always been a debate on how research gets into practice. Vivian Robinson wrote an article in the 90s where she argued that most academics do not acknowledge that teachers have very strong theories of learning. Most academics go in and say: 'This is what I found, here is the evidence'. And they ignore the fact that the teachers are interpreting it through their lens, through their model of learning. If I translate into Danish, I should be aware of how you interpret it. We do the same in visible learning. Before we release writings for teachers, we run workshops with teachers to see the answers to two questions: Do teachers understand this? If they don't, we are wrong. And the second one is if there is a consequence or an action that the teacher takes. If there is not, we have wasted our time. That really means we have to listen to the interpretations. That is how I think the notion of translation can be a very, very powerful notion. Most researchers don't do that. We do as we work out the scripts for the workshops.
HK: So you see translation as a matter of you saying something and then testing whether they understood it the way you meant it? JH: Correct. HK: I also wondered about the language issue between New Zealand, Australia, Denmark and the UK. There must be a lot of language issues? JH: Oh, yes. I am not sure with Danish, but certainly in the Scandinavian countries, they don't have a word that distinguishes between teaching and learning. Japan doesn't. There are major language issues. And that's why in every country we have local people that are part of the translation. It is a major problem. The Danish critiques HK: One of the critiques is that visible learning is like an alien coming into the Danish school tradition, not allowing voices that have been built through a Danish school tradition to be heard. The word evidence-based is very powerful, and there is a risk that it will shoot down discussions among teachers, and that visible learning is not sensitive to what is going on in a Danish setting and tradition. JH: You are right, in some countries there is a history, a tradition and a dialogue, and something like this comes along from the outside. As I said before, it is scripted, and one of the things that are really critical is how it is interpreted in a local context. When I first started this in New Zealand, we probably lost 70% of our schools within the first 6 months because the resistance was too strong. When I first went into the Skanderborg (a Danish municipality) schools, there was resistance, there always has been. Now it is the opposite. We are very careful in listening to the local context, having local people at the board. But when it comes down to it, no one has to do this. And as you know, some of the Skanderborg schools have pulled out for all kinds of reasons, but most have stayed the distance because they see the impact of the work on the students. I'll be much more interested in the results you find on what effects it has in schools, both on the teachers and the students, so tell me more on what you are doing. HK: One of the major figures in a Danish context is called Grundtvig. Some of his ideas are difficult to combine with the visible learning philosophy, because it is the idea of meeting the student as an equal, as a human being, seeing education not as training but as developing the whole person. Visible learning and these ideas are incomparable; it's like apples talking to seabirds. One of my interests is how we can create a dialogue on school development which is actually a dialogue, because it has been extremely conflict ridden, very angry and frustrated. JH: I never wanted the image that this was evidence-based. children. If you ask me, I would just leave them. I mean, I would allow them to be interested in the subject and not in their own learning process. JH: The problem is that if you compare a child from a family like yours to a child from a less resourced family, then how many words do you think your child was exposed to as compared to a child from a lower-resourced family? 30 million! I don't think I want to leave those kids alone; I do worry about how they learn. I deeply worry about what those kids learn, and what they think learning is. Between the age of 1 and 2, kids develop a theory of mind, a way of thinking about the world. That's when we should get to them about the notion of learning. Those kids have incredible fascination with learning, which unfortunately they lose when they come to school. Maybe you are lucky with your kids, but many kids don't think about learning and we do have to intervene. HK: I would think they are interested in the world, and they are interested in learning about the world, but they are not interested in learning about learning. JH: Why is it that 3-year-olds are obsessed with why questions? HK: Because they are obsessed with understanding the world…? JH: That's right, and to learn about how they learn about the world. And discover how they make connections.
HK: But it's this extra learning about the learning I don't like. I am worried about our expectations of our children in terms of self-reflexivity. JH: I would still want children at that very young age to try different learning paths, because that's when they start to get the habits that make a dramatic difference later on. I don't want to go overboard, but I am certainly worried about the amount of resources we put into the 0 to 5-year-old with very little return from these resources. Some of those kids have no concept of learning. I don't want to go into preschools and teach them to read or write. That's not at all what I am saying, but I do think that the age of 0 to 8 years is probably the most critical time to get to them on the notion on what learning means. They should get inspiration, they should be learning through play. There are a tremendous amount of things we should be doing with the young children which we are not.
Visibilitybasic assumptions HK: I have wondered: What do you mean by visible? What is visible? JH: It did take me 6 or 7 years to come up with the idea of the visible. A researcher that for many years did lots of classroom observations, Graham Nuthall, showed that the teacher doesn't see or hear 80% of what happens in the classroom, and that's why I am not a big fan of the idea of teachers' reflections on what they think happened as opposed to reflecting on their impact. In a nutshell, the major idea in visible learning is switching from talking about how you teach to the impact of your teaching. What I try to do, as my political self here in Australia, is more than anything else to legitimize the expertise of the teacher in terms of their impact on kids, because we have some stunning teachers who have great impact. They should get that visibility of their impact right up front. HK: So what is visibility? I understand the idea, but what does visibility actually mean? Maybe I am very concrete in my thinking, but I don't really understand it. And the other part of it: What is learning? When can you say that learning has taken place? JH: One example of visibility is an iPhone app where teachers can get, almost immediately, in three seconds, a script of everything they say in the classroom. The app can code it automatically while the teachers are doing it.
If the teachers allow them, we can ask the kids to react too, so the teachers can see their impact on the kids through the kids' eyes. When we show teachers a script of what they say, and we have done that with about 7000 to 8000 teachers, they are shocked, because most teachers don't realize they talk 70-80% of the time. Some teachers don't realize what they look like through the students' eyes. If you ask teachers to reflect, they would think they only talk 20-30% of the time. No, they don't. So how do you get that visibility back to them? That is what we try with visibility: What does it look like through the students' eyes? As to learning: Over the last three years, we have had a well-funded science and learning research centre, and one of my projects was to look at learning strategies. We focus on what that notion of learning means. We have identified 400 different strategies. We have narrowed it down to a top ten. If you are doing surface learning it works quite differently than if you are doing deep learning. One example is problem-based learning, which comes out with a very low effect size. The reason for that is that problem-based learning only works for deep learning; it doesn't work for surface learning. And 90% of the schools introduce problembased learning for surface learning, so of course it doesn't work. Learning means moving from surface to deep to transfer. I am also very interested in what students think learning is. One of the first things we do in schools is that we ask the teachers and the students what learning means. We put that up and show the contrast which it often is. When we ask them what it means to be a good learner at this school, they say things like 'come on time, sit up straight, be quiet'. But none of that is learning. And that contrast is really important.

Evidence and theory
HK: Your work comprises a huge number of studies. But there aren't many references to psychological, philosophical or historical literature within education. Or you could say theory in general. And I wondered how you see theory.
What is the reason for the absence of theoretical references, and do you somehow see a contradiction between evidence and theory? JH: When I wrote Visible Learning, I remember my wife saying: 'No one will ever read this, it's boring, it's full of numbers, and it's full of research studies and evidence'. I had no idea that it would have the impact it had; it has sold more than half a million copies. But if I had known that it would go to an audience larger than just researchers, I probably would have had a whole lot more of theory in it. I had never dreamed it would catch on like this, so you are right, it is quite devoid of theory in terms of how it is written. I am a measurement researcher, I am a statistician, I am not a theoretician, so I haven't written a lot of theory. But of course I have a very strong model of teaching. I have worked for many years with some of the more well-known people on the theoretical side of teaching around the world. But you are right, it did not come through, and sometimes I think I should write a book about teaching as a professionwhich indeed I am doing right now. I get accused more of not taking into account sociology, and the background of kids. Of course I think that is very critical, but the book was never written for that kind of general notion. You are right, it is a criticism, and maybe I should write something more theoretical, but it is not really my strength. I do have some very strong theories on how theory works and the concept of teaching. It just didn't come through in the book. HK: I also ask because in the Danish foreword to 'Visible Learning for Teachers', Niels Egelund and Lars Qvortrup, two Danish education professors, write that finally education has become evidence based. It has taken several hundred years for medicine to become evidence based, and now finally education is evidence based, and we can get rid of theory, meaning abstract thinking and speculative thinking. JH: Obviously, I wouldn't have said that. Evidence does not mean there should be no theory. I think evidence is, and should be, the most contested word of the business. My view on evidence is that evidence is interpreted; evidence is interpreted through a lens, and that lens is theory. Theory dominates. Yes, I do have strong theory, and yes, it is not as clear and articulate as it should be. But medicine worries me as a parallel, because evidence in medicine is very new, and to say that doctors don't have theories is just absurd. I would like the evidence that teachers use to be about their impact on students, not whether they read thousands of articles. HK: Could you elaborate on your concept of evidence? When can you claim that something is evident? Or that something is evidence based? JH: Evidence is contested, as I mentioned. The key is to understand how different people in the classroom (teachers, students, peers, and maybe also parents, fellow teachers, school leaders) make their interpretations about evidence. Evidence is not neutral. A key of the VL theory is that we can optimize our impact if we understand the impact of our actions and teaching through understanding how the students see learning. The research studies in VL offer probability statementsthere are higher probabilities of success when implementing the influences nearer the top than bottom of the chart. But then it depends on your fidelity, your adaptation and, of course, the evidence of the impact you have on your studentshence the claim 'Know thy impact'. Evidence is a key but only a part of this chain. Evidence can also be related to experienceand the extensive experience of many teachers is legitimate evidenceto be contested, to be examined, and to be evaluatedin terms of the best impact on the learning lives of students. When there are differences between the evidence from the research and from experience, then there is a need for examination, for reflection, for seeking more avenues of evidenceand I want this to be via the effects on the students. couldn't be if you were not a researcher. JH: Absolutely correct. And the other beauty is that I do get all the data. I get all the data, e.g. from Denmark where I get data from all those schools, obviously without the kids' names. Data is my main interest. Like yesterday, I got all the mindframe data from around 10,000 teachers. That's the excitement of being an academic. The playing with data. The interpretation of the evidence, the discovery of new ways of seeing the world; that's the fun part of the job.

Disclosure statement
No potential conflict of interest was reported by the author.

Funding
This work was supported by Aarhus University Research Foundation [12609].