It is the very nature of humans to search for patterns and meaning across all aspects of their lives, including work. When used well, and in tandem with other factors, the use of data and statistics can guide strategic and operational development across many levels of an organisation, including schools.
I still remember my excitement in first year university statistics when I was introduced to a range of probability distributions. It still seems fascinating to me that patterns of outcomes in large groups can be modelled using relatively simple mathematical forms.
Improving the learning experience
Educators use data to guide their teaching every time they meet with their class, but they may be unaware of how they are using it or even that they are collecting it. Teachers will speak of “reading the room”. This might involve unpacking a student’s response to an assessment task in terms of their probable cognitive skills or estimating a student’s position along a learning continuum through observation and teacher judgement.
In addition to using ‘individual student’ data, teachers and, more broadly, schools also consider ‘large group’ data when considering their practices. Large group data can include results from common tests across a few classes or whole year level data such as NAPLAN results.
In the wider education sector, researchers can use data from thousands of students to seek out broad system patterns. Some work at the meta-analysis level by combining various pieces of research to find overarching generalisations and effects. These ideas can help inform policy and strategy but understanding the stories and shaping the pathways for individual students can only happen using personal, small group and cohort data.
However, when doing so, caution is required on three fronts.
Firstly, the data normally captures some instant of time about human response. A core tenet in statistical measurement is that, if the measurement is repeated on the same item, the same result is observed (within the random errors of the measuring device). The repeatability of human response is questionable as so many circumstances (such as mood, time of day, blood sugar level) can affect the response.
Secondly, the size of the data set in most schools is relatively small. Typical groups are class level (20-25) up to year level (100-200). This can cause sizeable confidence limits when interpreting collated results.
Finally, the core variable of human difference is almost impossible to control. Comparing data sets across years (such as Year 3 NAPLAN results over the past five years) becomes quantitatively complex as each cohort has different characteristics.
Individual student data is exposed to even deeper numerical problems due to the changes occurring in the learner. The sample size of one means we cannot use statistical smoothing to identify particular trends.
Why data is important
The above seems rather gloomy, suggesting so many problems in collecting valid data that one wonders why schools invest time and resources in curating multiple data streams.
The answer is simple. Data about students provokes us to ask questions. We seek to understand the story that underpins the information the data presents.
Data can deepen the understanding of how a student is thinking or developing specific skills. It can inform (rather than drive) future action by helping develop hypotheses or confirm the usefulness of certain interventions.
We can leverage the true power of data within our school by understanding the limitations and exploring the patterns we see by challenging the assumptions and probing the circumstances. Triangulation of indicators (usually more than three sides, so not really a triangle!) helps us understand which are core and which are transitory, behaviours or outcomes.
We can never fully understand what is happening within someone else’s head, so we use observations and data to inform teacher judgement. The more sources of information we have, the deeper the questions we can ask, and the more solid is our understanding of the learner.
Data plays a critical part in the educator’s toolkit, and like any tool, it needs to be used properly. Data literacy has become a necessary part of all teachers’ suite of skills.
Andrew Baylis (OM 1979)
Director of Learning and Research
About Andrew Baylis
Andrew Baylis (OM 1979) has held the position of Melbourne Grammar School’s Director of Learning and Research since 2014.
Prior to this, his appointments included Executive Director of the Crowther Centre, Director of Teaching and Learning at Brighton Grammar School, and Head of Physics at both Henry Box School in the UK and St Bernard’s College.
