Learning analytics and study behaviour: A pilot study

It looks like we’re going to start playing around with the Indicators project again. So it’s time to start reading up a bit on the learning analytics literature and see where the interesting work is. First up is

Phillips, R., Maor, D., Cumming-Potvin, W., Roberts, P., Herrington, J., Preston, G., Moore, E., et al. (2011). Learning analytics and study behaviour: A pilot study. In G. Williams, P. Statham, N. Brown, & B. Cleland (Eds.), ASCILITE 2011 (pp. 997-1007). Hobart, Tasmania.

The following is a summary/reflection of reading through it.

Abstract

Describes a study that looks closely at analytics data of four students – interacting mainly (it appears) with lecture capture software and then interviewed them to find check out the assumptions from analytics. It found that the analytics had some limitations which was supplemented nicely by the qualitative data. Makes some suggestions for further research around the methodology.

Aim is to find out how students engage with and study in e-learning environments. But this study focuses on the lecture capture tool

Background/intro

Makes mention of “Much e-learning research over the years has been based on quantitative data largely derived from the perceptions of students” which has some limitations. Other work is qualitative around individual contexts. This work is seeking to combine descriptive/qualitative with learning analytics.

Analytics has a history, but the meaning of the data is not always clear. usage logs record behaviour, without explaining why suggesting taking care to analyse and interpret the data.

Prior work

Some mention of the lecture capture work. Which is seen to focus on the technology, not the learning environment as a whole.

This was the impetus for our current work, which holistically examines a unit of study, and uses learning analytics to gain a richer understanding of what students actually do in a technology-enhanced learning environment.

The focus being on learning processes used by students, rather than learning outcomes.

Lectopia usage patterns

Summarises prior work around analytics from lecture capture system and identifying “types” of student users: conscientious, good-intentioned, repetant, bingers, crammers, one-hit wonders, and random users.

This work reports on a pilot study interviewing students with diverse usage patterns to find out what’s going on.

Method

..pragmatic, mixed-methods paradigm of inquiry using a modified design-based research approach..

Data sources: their analysis tool, SNAPP, standard LMS usage reports, assessment results, attendance logs for lectures, interviews with teachers, and semi-structured interviews with students.

Received ethics approval for an approach that included identifying students and handling this appropriately. Some difficulties getting a good sample of students to interview – became a “convenience sample”

Students drawn from a 3rd year sociology of education course. ~150 students on main campus, ~50 students on regional campus, ~100 DE students. 1 hour lecture and 2 hour workshop for internals. Lectopia plus discussion forum activities for DEs. Numerous readings.

109 students accessed lecture recordings

Recording usage appears to match the expected peaks (lead up to assessment) and troughs (other times).

Interviewed students were all internal.

Discussion

Interview data from 4 students is summarised. And shows very different approaches to study.

the four cases reported here start to illustrate some of the complexity of the modern, technologyenhanced learning environment.

A range of limitations of the study are identified and then

two major implications arose from this analysis: a need for the broadening of the mechanisms for identifying student behaviour patterns; and the application of the methodology to other contexts.

Student don’t visit the lecture capture system enough to give useful data. Broader LMS usage needs to be examined.

Suggests the addition of a mid-term survey of students about their perceptions of technology use and wider range of data.

Talks about selecting more units to include by reviewing courses for alignment.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s