Wednesday, September 19, 2012

NAPLAN data

The next round of NAPLAN data is being released and the issues with summarised statistics arise.  In low socio-economic schools this data is damaging and will close schools - not because of poor teaching but because of cohort changes.

Let's take a sample school.

General assumption:
NAPLAN scores have dropped over four years.  Obviously something wrong with the teaching staff.

Examining raw data:
Increase in students with little or no schooling (refugee intake)
Opening of new school nearby attracting higher performing students
Half cohort was generally a weak group due to many students (and siblings) moving to private schools in yr 7 (entry to secondary schooling early was a significant factor in parents choosing schools in yr 7 coupled with aggressive marketing by private schools to maintain student numbers)
High turnover in experienced staff
Decrease in general school attendance (and students not attending at all) - increase in overseas holidays in yr 8, truancy, mental health issues
Issues with changing curriculum and yr 7 content not being taught to the level required by NAPLAN in public primary schools
Inability to move on students with little or no interest in schooling
Strong increase in performance of high school ready students (what high school teachers are trained to do) and low levels of improvement of students that are at primary levels during yr 7/8 (area of improvement for the school).

The issues make it hard to compete with local private schools.

None of these factors are taken into account by a one number summary, nor does it take into account the lead-in required to cater to a new circumstance that the school is experiencing (in this case a much higher number of low ability students).  Even if the school diagnosed the problem, reacted and implemented cohort specific solutions (including structural changes to better cater to low ability students), it takes lead time and strong leadership to identify and implement actions that have significant impact on NAPLAN statistics and student learning.  Yet in many cases a lower NAPLAN score will be seen as a teacher issue, comments driven by the misuse of statistics.

Furthermore, little analysis is done to see where systems are working and where changes in the pipeline have caused a significant positive change in student results.

Lastly, by releasing this data to parents (rather than aggressively seeking the problems and rectifying it within schools) a downward spiral commences.  A school with a low NAPLAN score does not attract good students, thus the score continues to drop each year and student numbers fall.  Senior school offerings reduce as student numbers are not sufficient to sustain courses.

2 comments:

  1. So true Russell. And sadly they've let the genie out of the bottle by publishing results to the public and I don't think they'll ever rescind that decision no matter how negatively it affects schools.
    NAPLAN is a good tool for INTERNAL use in schools to adapt to student needs and identify whole school and specific area weaknesses. Making results public only weakens the schools that need supporting the most (like you I am at a low socio-economic school - primary though - that takes a hammering from NAPLAN because we aren't addressing entrenched issues affecting our students that take years to overcome.
    Just wait til NAPLAN becomes the determinant of teacher bonuses and salary progression :(

    ReplyDelete
  2. Statistics exist that show that progress is being made (a look at ears and firstcut properly exposes this information). If "progress" rather than "performance" data is used to identify underperforming teachers then it is one indicator of where improvement can be made in a school. I don't have a problem with this.

    I think my issue is that it takes time and skill to properly analyse and there are too few people that can do it properly. If this analysis was done centrally (with a bit of thought applied) by those experienced in statistical analysis in conjunction with schools then (perhaps) NAPLAN data analysis would have some validity.

    I would share your concern if NAPLAN "one number" statistics became the be all and end all of teacher performance. Given not all teachers teach years 3,5,7,9 I imagine this is unlikely.

    ReplyDelete

Hi, thanks for leaving a comment.. it's good to hear what people think!