Title: Misunderstanding and misrepresentation: a reply to Hutchison and Schagen
Abstract: Click to increase image sizeClick to decrease image size Notes 1. Whenever I have written previously about MLM, its advocates have resorted to amending what I wrote, quoting the incorrect text and then arguing with it in a way that relies for any efficacy on readers not caring enough about the truth of the matter to read the full accounts. Presumably, this is easier for MLM advocates than actually discussing what I did write. I have described this process of distorting evidence, by writers such as Plewis and Fielding (2003 Plewis, I. and Fielding, A. 2003. What is multi‐level modelling for? A critical response to Gorard (2003). British Journal of Educational Studies, 53(4): 408–19. [Taylor & Francis Online] , [Google Scholar]) and Noden and Goldstein (2007 Noden, P. and Goldstein, H. 2007. A brief response to Gorard and Fitz. British Educational Research Journal, 33(2): 273–4. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), as ‘the antithesis of intellectual endeavour’ (Gorard and Fitz 2006 Gorard, S. and Fitz, J. 2006. What counts as evidence in the school choice debate?. British Educational Research Journal, 32(6): 797–816. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). Noden and Goldstein (2007 Noden, P. and Goldstein, H. 2007. A brief response to Gorard and Fitz. British Educational Research Journal, 33(2): 273–4. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), for example, attempted to deflect criticism of their faulty arithmetic in a previous paper where they calculated a national average by adding together scores for each LEA without regard to the size of each LEA. They did so by claiming that I had made the same mistake in another paper (Taylor and Gorard 2001 Taylor, C. and Gorard, S. 2001. The role of residence in school segregation: Placing the impact of parental choice in perspective. Environment and Planning A, 30(10): 1829–52. [Crossref] , [Google Scholar]). I invite all interested readers to check the facts out for themselves. They will see that nowhere in the paper do I find a national average for anything. I do use unequal size enumeration districts and unequal size schools as organizational units but their size is automatically taken into account when aggregating to LEA‐level. Therefore, I did not, indeed could not, make the same serious arithmetic error as Noden and Goldstein. Hutchison and Schagen (2008 Hutchison, D. and Schagen, I. 2008. Concorde and discord: The art of multilevel modelling. International Journal of Research & Method in Education, 31(1): 11–18. [Taylor & Francis Online] , [Google Scholar]) appear to approve of such misquotations and distortions by citing a paper by Plewis and Fielding (2003 Plewis, I. and Fielding, A. 2003. What is multi‐level modelling for? A critical response to Gorard (2003). British Journal of Educational Studies, 53(4): 408–19. [Taylor & Francis Online] , [Google Scholar]) who used this dishonest and anti‐intellectual technique of mis‐quoting widely, despite the fact that Hutchison and Schagen also cite my response to Plewis and Fielding (Gorard 2003a Gorard, S. 2003a. In defence of a middle way: A reply to Plewis and Fielding. British Journal of Educational Studies, 51(4): 420–6. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and so must have read about these inaccuracies. In case any readers are in any doubt about how blatant these examples are, please follow the references in Gorard and Fitz (2006 Gorard, S. and Fitz, J. 2006. What counts as evidence in the school choice debate?. British Educational Research Journal, 32(6): 797–816. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and read the original papers. I have to assume that Hutchison and Schagen either have not done so, or cannot see what Plewis and Fielding (2003 Plewis, I. and Fielding, A. 2003. What is multi‐level modelling for? A critical response to Gorard (2003). British Journal of Educational Studies, 53(4): 408–19. [Taylor & Francis Online] , [Google Scholar]) did wrong. Perhaps, this is because, like them, Plewis and Fielding were seeking to defend the widespread use of MLM. If such apparent leaders in their field have to resort to misrepresentation to defend their position, and if peer‐reviewers and editors cannot spot it when they do, then education research faces a poor future. This is partly why I say that the issue underlying this paper is important, and about more than whether MLM produces better results than OLS. 2. On the topic of clustering, Hutchison and Schagen say: ‘He [Gorard] also states that: “a girl in one class may be much more similar to a girl in another class than to a boy in another class. And so on”. Again this is a misunderstanding. As an analogy of why this is incorrect, for example it is known that boys are on average taller after puberty than girls. This statement of averages does not imply that all boys are taller than girls’ (2008 Hutchison, D. and Schagen, I. 2008. Concorde and discord: The art of multilevel modelling. International Journal of Research & Method in Education, 31(1): 11–18. [Taylor & Francis Online] , [Google Scholar], 16). There are many problems with this passage. First, the quotation is wrong (see note 1). What I actually wrote was ‘A girl in one class may be more similar to a girl in another class than to a boy in her own class. And so on’ (Gorard 2007 Gorard, S. 2007. The dubious benefits of multi‐level modelling. International Journal of Research & Method in Education, 30(2): 221–36. [Taylor & Francis Online] , [Google Scholar], 227–8). Second, presumably both versions – what I said and what they misquote me as saying – are actually perfectly reasonable. A girl might, in many respects, be more similar to another girl of the same age in a different teaching group than to a boy whether in the same or a different teaching group. How can Hutchison and Schagen call this ‘incorrect’? Third, what is their ‘analogy’ intended to show? It reminds me that boys might tend to be taller than girls of the same age. This would increase the probability that, in terms of height at least, two girls might be more similar than a girl and boy – whatever teaching groups they were in, since height is not usually a criterion for allocating pupils to classrooms. This would seem to me to agree both with what I actually said and with what they thought I said. I cannot imagine how they think this any kind of argument for the use of MLM.