An overview of learning analytics

The following is a summary and some reflections upon

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, (August), 1–13. doi:10.1080/13562517.2013.827653

A good overview of the learning analytics field as it stands. A good call to arms for teachers to engage with the fad and use it to improve learning and teaching.

Abstract

Learning analytics, the analysis and representation of data about learners in order to improve learning, is a new lens through which teachers can understand education. It is rooted in the dramatic increase in the quantity of data about learners and linked to management approaches that focus on quantitative metrics, which are sometimes antithetical to an educational sense of teaching. However, learning analytics offers new routes for teachers to understand their students and, hence, to make effective use of their limited resources. This paper explores these issues and describes a series of examples of learning analytics to illustrate the potential. It argues that teachers can and should engage with learning analytics as a way of influencing the metrics agenda towards richer conceptions of learning and to improve their teaching.

An abstract that has me very interested in the paper. Too much of analytics tends to lean toward the “management approaches that focus on quantitative metrics” and not on teaching staff engaging with learning analytics. Given the journal this is published in, the focus on teachers is not surprising.

Introduction

Identifies the tension teachers and learners face between economic activity and actual learning.

Quantitative metrics are increasing due to

  1. “theoretical framings” i.e. management etc.
  2. big data

    Like the idea of “big data” – the availability of data and the rise of computational techniques to handling that data – as an enabler.

Describes the scale of data in the sciences – e.g. 23 petabytes of data in 2011 generated by the Large Hadron Collider. Would like to compare the size of this data with the data available to a university on learners as a comparison of just how “big” big data is in tertiary education.

Makes an important point which I think most traditional researchers don’t get as a difference between traditional approaches to data and big data

The volume and scope of data can be so large that it is possible to start with a data-set and apply computational methods to produce results and to seek an interpretation or meaning only subsequently

Frames the rise of business intelligence and its focus on performance metrics as something encroaching into higher education through the “education as an economic activity”

What about the learners?

Moves onto learning analytics. Offers the following distinctions/definitions

  1. Academic Analaytics – the application of business intelligence to education.
  2. Educational Data Mining – development of methods for analysing educational data focusing “more on the technical challenges than on the pedagogical questions (Ferguson, 2012)”.
  3. Learning analytics – “first and foremost concerned with learning”.

But doesn’t mention teaching analytics – the “proposed subfield of learning analytics that focuses on the design, development, evaluation and education of visual analytics methods and tools for teachers in primary, secondary and tertiary educational settings”.

But does make the point that a key concern is generate “actionable intelligence” which informs appropriate interventions. And brings up the Campbell and Oblinger (2007) 5 step cycle:

  • capture
  • report
  • predict
  • act
  • refine

Good description of the “field”

Learning analytics is not so much a solid academic discipline with established methodological approaches as it is a ‘jackdaw’ field of enquiry, picking up ‘shiny’ techniques, tools and methodologies

Positioned as both strength (“failitates rapid development and the ability to build on established practice and findings”) and weaknesses (“lacks a coherent, articulated epistemology of its own”).

Predictive modelling

Defined as

a mathematical model is developed, which produces estimates of likely outcomes, which are then used to inform interventions designed to improve those outcomes.

Techniques: factor analysis and logistic regression

Applied to data including

  • prior educational experience and attainment
  • demographic information
  • information arising from the course – use of tools, assessment etc.
  • Whether the student went on to complete the course.

Similar to the teaching noticing a struggling student, but some practical differences

  1. Output is in the form of estimated probabilities, which many people struggle to understand.
  2. Output available to more than just the teacher.
  3. Interventions can be triggered without teacher involvement.

Predictive power of these models is not perfect. But they can be more often right than wrong and they can be used to improve student completion.

Pudue’s Course Signals project is the example used. Makes the point that

Course Signals are not used in a decontextualised
environment: the teacher is central to the process and uses their judgement to direct students to appropriate existing resources within the university.

Mentions other examples – OUUK’s Retain and S3 from D2L. Mentions the OUUK’s finding that level of activity was not a predictor, but a fall-off in activity was a clear indicator.

SNA

Use of network analysis to analyse connections between people. Can be interpreted both by eye and thorugh mathematical analysis.

SNAPP as the example. Mentions the trade-off. SNAPP is easy to use, but not as flexible/powerful as more general tools.

Mentions other more complex work, including an ANT informed analysis of Tapped-In.

Usage tracking

Tracking of student activity in computer-based environments. Raises questions about value of this for student learning, what sorts of feedback is helpful, and ethical questions.

Content analysis and semantic analysis

A move away from quantitative data generated by students toward the analysis of qualitative textual data.

Example of “Point of Originality Tool” that helps to “track how students develop originality in their use of key concepts over the course of a series of writing assignments”.

Also the OU work on exploratory talk, a more speculative example

These methods could be used to analyse the students’ contributions to an online forum, giving them feedback about the degree to which their online talk is exploratory (or matches other criteria for constructive educational dialogue), and offering suggestions for ways in which they might contribute more effectively.

Automated assessment not currently within learning analytics. But does appear to be a form for learning analytics.

Recommendation engines

Analyse behaviour to provide suggestions to individuals for items that may be of interest.

Could be used to suggest learning resources to a student. But may be difficult in the context of a set curriculum.

This is an approach I’d like to implement in BIM recommending to students either

  • Other class mates blogs they may wish to follow.
  • Useful links that have been mentioned in student posts.
  • Or perhaps specific posts from students that may be of interest.

Discussion

Engages in some implications for teacher in higher education

Ethics and privacy a bit one. Mentions common points.

Being open about learning analytics with students can improve perceptions.

The student knows more about their own learning situation than even the best teacher. Make it available to them.

Teachers have a professional responsibility to use appropriate means to improve student learning. Analytics raise ethical questions – do you allow a student to enrol if the predictive algorithm suggests they’ll fail.

Student feedback has to be handled well.

Learning analytics is often atheoretical or not explicit about its theoretical basis. Some authors have tried to ground it.

Makes the point that if assessment doesn’t reflect learning, then learning analytics that improves performance on assessment, doesn’t necessarily improve learning.

Conclusions

The promise of learning analytics is the empowerment of teachers and students to
understand the wealth of data that relates to their learning….to achieve institutional change, learning analytics data need to be presented and contextualised in ways that can drive organisational development (Macfadyen and Dawson 2012).

Makes the point that data and mathematics (and learning analytics) can be used to reinforce the status quo. And leads into this risk of learning analytics being driven by the world view of managers and the economic framing of education.

References

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, (August), 1–13. doi:10.1080/13562517.2013.827653

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s