NAPLAN to NOWHERE – An Ineffectual Test

It is disturbing to find government ministers claiming that schools don't teach to the test and children and teachers don't become stressed about NAPLAN when it is clear that across the country they do ("Brands cash in on pupil test fear", 11/5). It is even more concerning that an ineffectual test, which measures so little in a narrow and meaningless way for individuals, pretends that it can capture the essence of what is happening across a county as diverse as Australia.

So it begs the question ... why does the Federal government insist on placing such emphasis and credence on the NAPLAN? Particularly given the compelling research, what the experts say and what the school communities know about the limitations and the negative ramifications in terms of learning and practice.  The answer lies in the Government having good intention - to raise educational standards and to measure these standards so that they as a government can support schools that need support, reward excellent teachers and provide the best opportunity as possible for our children to learn.  Unfortunately there is massive contradiction between intention and practice.

The first error of their ways lies in the measurement tool being used (NAPLAN); all researchers know that a measurement device needs to be both accurate and precise.

Accuracy relates to how close can you get to the bulls eye with your measurement tool - with NAPLAN that means how accurately does NAPLAN measure children's learning and teachers performance?  The answer is that NAPLAN fails to provide an accurate tool to measure authentic literacy and numeracy.  Furthermore it completely misses other critical skills, knowledge and understandings we need to give to our children for them to be successful learners such as;

  • self empowerment
  • decision making
  • organisation skills
  • ownership of their learning
  • self esteem
  • authentic contributions
  • having a voice and lateral
  • creative thinking

If we think the education of children is about being able to rote learn, to be competent in numeracy and literacy and nothing else ... and then to be compared to every other child across the country then NAPLAN is the way to go.

If we think of education as the development of the skills required for our children to become successful in life (including numeracy and literacy) then NAPLAN fails dismally as an assessment tool.

Precision relates to the degree to which repeated measurements under unchanged conditions show the same results (the degree of scatter of the shots around the bulls eye) - with NAPLAN this means if you were to test the same children using NAPLAN over a period of months or years would they achieve a similar result?  The research reports that the NAPLAN has very poor precision.

Figure 1: Some assessment tools have high accuracy but low precision (left hand panel) (eg accurately measures what is claimed, but there is a large scatter around the measure) while others assessment tools have high precision but low accuracy (right hand panel) (eg not measuring what is claimed, but the scatter around the measure is very narrow).  NAPLAN has low accuracy and poor precision (eg fails to provide an accurate tool to measure authentic literacy and numeracy and there is a very large scatter around the measure)!

The agency responsible for the My School website cautioned against using national literacy and numeracy test results to award financial bonuses to schools, warning the data was likely to be unreliable. The Australian Curriculum, Assessment and Reporting Authority reported that overseas experience showed that data that attempted to track the improvement made by students over time was volatile.  That the authority board cast doubt on the reliability of the information and that caution was urged against using this data set, as the error margins (i.e. the precision) are likely to be too great to infer any meaningful conclusions. International experience shows that data tend to bounce from year to year.

All of this provides compelling evidence that while the government may have good intention their method to fulfill this intention is completely flawed and runs a high risk of achieving an outcome that is the opposite; poor teaching and learning; disenfranchised teachers; and disengaged children.  It is incomprehensible how the Gillard government could waste millions of dollars in bonuses to schools and teachers on such meaningless and flawed data;

The Governments insatiable desire to test, measure and perpetuate failed systems leaves us all in a state of intense frustration, anger and disbelief.  For how much longer do we need to go down this path before Julia Gillard's advisors get what so many others in the western world know - that standardised testing is a complete waste of everyone's time, misleads and misdirects where the emphasis of measurement of quality education should go and serves to compromise our teachers but more importantly is a total injustice to our children.

Shona Bass & Kathy Walker