I’m currently thinking about a potential contribution to the SoLAR Southern Flare Conference on Learning Analytics in about a month or so. The early shape of that contribution is online and my last few posts have been summarising some explorations through various areas of the literature. Not the learning analytics literature, but broader literature on which much of the learning analytics literature should be based, but of which much isn’t.
That reading has me increasingly pessimistic about the end result of learning analytics moves within Australian universities. It will be just another fad. Does anyone feel more positive?
Sure, there will be some neat examples of where it is used. But unless some lessons are learned quickly, I don’t see it being widely used beyond a few motivated folk and the odd senior manager who makes a right balls up because of it.
Some illustrative quotes
A field of research within the Information Systems discipline with a history stretching back to the 1970s. It’s a field that includes business intelligence, data warehousing and a range of other explorations of the use of technology to aid organisational decision making. Arnott and Pervan (2005) provide a critical analysis of the fields and have some nice comments.
As a result data warehouse development is dominated by central IT departments that have little experience with decision support. A common theme in industry conferences and professional books is the rediscovery of fundamental DSS principles like evolutionary development (Keen, 1997).
Business intelligence (BI) is a poorly defined term and its industry origin means that different software vendors and consulting organizations have defined it to suit their products; some even use ‘BI’ for the entire range of decision support approaches
Hosack et al (2012, p. 321) add this
Additionally, the best DSS cannot overcome poor managerial decision making.
Some more from Houghton and Mackrell (2012)
Specifically we found that existing patterns of sensemaking hindered the data quality of the BI system because of how key people made sense of their work. We argue that because there was divergence in sensemaking patterns in the social systems, the data collected may not represent a true picture of ‘business
More recently there is this article on “Big Data Fail” which starts with the claim
Much of the great promise of business intelligence (BI) goes unrealized because decision makers aren’t using the decision support systems in any meaningful way. The vast majority of big data and business analytics projects implemented by normal companies suffer from chronic underuse.
And then proceeds to explain the traditional problem with information systems. The IT folk building systems no-body uses because the systems don’t do what the users need them to do.
Talk about deja vu all over again.
Not to mention Macfadyen and Dawson’s (2012) experience
the reality that the institutional planning process was nonetheless dominated by technical concerns, and made little use of the intelligence revealed by the analytics process.
“dominated by technical concerns”, sounds like a good prediction for the roll out of learning analytics in at least some Australian universities.
Arnott, D., & Pervan, G. (2005). A critical analysis of decision support systems research. Journal of Information Technology, 20(2), 67–87.
Hosack, B., Hall, D., Paradice, D., & Courtney, J. F. (2012). A Look Toward the Future : Decision Support Systems Research is Alive and Well. Journal of the Association for Information Systems, 13(Special Issue), 315–340.
Houghton, L., & Mackrell, D. (2012). The impact of individual, collective and structural Sensemaking on the usefulness of business intelligence data. MCIS 2012.
Macfadyen, L., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Educational Technology & Society, 15(3), 149–163.