Computers ‘do not improve’ pupil results, just like wood ‘does not improve’ houses

Here we go again. There’s an OECD report on “Students, Computers and Learning” that is doing the rounds. @palbion points to one media report

A report that starts with the observation

Investing heavily in school computers and classroom technology does not improve pupils’ performance, says a global study from the OECD.

The same OECD report has been mentioned in Moodle research discussion forum.

Almost 30 years ago Seymour Papert (1987) wrote “Computer Criticism vs. Technocentric Thinking” in which he asked people to

Consider for a moment some questions that are “obviously” absurd. Does wood produce good houses? If I built a house out of wood and it fell down, would this show that wood does not produce good houses? Do hammers and saws produce good furniture? These betray themselves as technocentric questions by ignoring people and the elements only people can introduce: skill, design, aesthetics. Of course these examples are caricatures. In practice, hardly anyone carries technocentrism that far. Everyone realizes that it is carpenters who use wood, hammers, and saws to produce houses and furniture, and the quality of the product depends on the quality of their work. But when it comes to computers and LOGO, critics (and some practitioners as well) seem to move into abstractions and ask “Is the computer good for the cognitive development of the child?” and even “Does the computer (or LOGO or whatever) produce thinking skills?” (p. 24)

Beyond asking an “obviously absurd” question, there are other problems I have with this study.

What is meant by “use tablets and computers”

The BBC report quotes the OECD’s education director as saying

Those students who use tablets and computers very often tend to do worse than those who use them moderately.

But what does “use tablets and computers” mean?

Papert (1987) again

Stated abstractly, the two studies have the same explicit intention: the children are to be given “programming”– and the purpose of the experiments is to see what happens. But there is no such thing as “programming-in-general.” These children are not given “programming.” They are given LOGO. But there is no such thing as “LOGO-in-general” either. The children encounter LOGO in a particular way, in a particular relationship to other people, teachers, peer mentors, and friends. (4) They don’t encounter a thing, they encounter a culture. (p. 27)

Now read that quote again and replace “LOGO” with “tablets and computers”.

The culture in schools impacts on how students are using the tablets and computers. At best the report has found a correlation and not the cause. The cause would seem likely to arise from the culture and how it impacts that nature and quality of students’ use of “tablets and computers”.

Given that the 2015 Horizon Report for K-12 identifies “Integrating technology in teacher education” as one of the problems facing the use of “tablets and computers” in K-12 education, might not this say something about the culture influencing the way “tablets and computers” are used in schools. Especially given that the 2015 Horizon Report for K-12 suggests that

the most important finding is that the level of a teacher’s digital competence directly correlates with students’ learning outcomes when technology is used

The OECD’s education director understands that how “tablets and computers” are used are important. The BBC report again

But Mr Schleicher says the findings of the report should not be used as an “excuse” not to use technology, but as a spur to finding a more effective approach.

But don’t start blaming the teachers and the teacher educators (at least not solely).

Warning: reliance on anecdote.

Every year I teach a course in “ICT and Pedagogy”. A course with the aim of helping them be able to design effective ways to use tablets and computers for student learning. Each year around 400 of these students go out into schools through Australia and abroad. Each year there are positive and negative stories.

There are stories of classrooms with little or no technology that actually works. Stories of where the only technology available is in a computer lab that the class can access for an hour or two each week. Stories of a full curriculum and a focus on standardised tests. Stories of teachers and school leaders that are only now starting to use technology in their everyday life. What might these stories say about the culture in these schools?

Both the negative and positive stories feature the above elements. Typically the only difference between the negative and positive stories is effort put in by a mentor teacher and/or the pre-service teacher involved. Very rarely are the positive stories a result of the school culture as a whole.

How is “learning” measured?

The BBC report reports the OECD’s education director again

He said making sure all children have a good grasp of reading and maths is a more effective way to close the gap than “access to hi-tech devices”

Which brings me to the question of “learning” is measured?

The summary of the report suggests that learning is being measured in this report “Based on results from PISA 2012”. PISA 2012 “assessed the competencies of 15-year-olds in reading, mathematics and science (with a focus on mathematics) in 65 countries and economies”.

Okay, so learning is being measured by a test on “reading, mathematics and science” and results indicate that “a good grasp of reading and maths is a more effective way to close the gap”.

Using the available data, not the meaningful data

Lastly, it appears that report seems to have taken the available data (i.e. PISA data showing performance on the PISA tests and ICT use) and sought to identify patterns and draw conclusions from that data.

Data about how students actually use ICT. Data about the culture within schools around learning, teaching, and the use of ICT doesn’t seem to have been available and hence it wasn’t considered.

Disclaimer

The comments/argument above is based on the BBC report. I have not read the full report. Hence I’m liable to be committing the following offence.

References

Papert, S. (1987). Computer Criticism vs . Technocentric Thinking. Educational Researcher, 16(1), 22–30.

One thought on “Computers ‘do not improve’ pupil results, just like wood ‘does not improve’ houses

  1. Computers that are falling apart, put together with masking tape, with missing keys and a broken mouse, plus horrible access to the Internet, do not improve student results. Yet, I see a lot of this in secondary Education Qld! Many non-IT specific teachers just give up on the idea of integrating digital tech in authentic ways; it’s hard not to sympathise with them.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s