Some findings from initial exploration of Moodle Book usage

I recently gave a talk at the Moodlemoot’AU 2016 conference in Perth. The talk was titled “How and why do people use the Moodle Book module?”  It reported on analysis of data from one University’s Moodle isntances to explore how courses, learners, and teachers made use of the Moodle Book module (aka the Book) from 2012 through 2015. The slides, abstract, and links to various interactive graphs used in the presentation are available via the presentation webpage.

This talk is part of a broader project that is looking more into post-adoption usage of e-learning systems, with an initial focus on the Book module.

The following provides a summary of some of what was found.  The links are to web pages that contain relevant interactive graphs.

Most books aren’t completely read

The percentage of students who completely read all of the Book resources appears quite low. In 2012, half the books were completely read by less than 47% of students. In 2015 half the books were completely read by less than 2% of enrolled students.

More integrated design appears to increase completion

I teach a course where Moodle book resources are integrated into the course and contribute a small part toward the final result. Not unexpectedly this produces better completion rates. However, questions remain the quality of the “reading” and some ups and downs in the data.

Courses using the Book tend to be bigger

Courses that use the book tend to have more students (median course enrolment of 85 in 2015) than those that don’t. (median of 11 in 2015). Raising questions about what is the difference with larger courses that seems to drive the use of the Book module.

..and have tended not to be for online only students

The majority of courses that use the Book do not include (m)any online students, especially when compared with all courses.

Again raising the question of the motivation/purpose for using the Book.

However in 2015 there was an increase in the number of courses with online students using the Book module.

This appears to be a result of the discontinuation of an institutional system that was used to transform more traditional distance education study guides into web-based resources.

Size of books and number of books

In 2015, the median number of Book resources per course was 3, however, there were two courses that had almost 80 Book resources.

In 2015, the median number of chapters per Book was 4, however, there were 3 that had more than 80 chapters. In 2014, there was a Book resource that contained over 74,000 words.

Most people are not using the Book import functionality

The Book module provides two broad methods to create resources:

1. create; and
   Use a web-based editor to type (or perhaps copy and paste is the most popular method) text into the Book resource.
2. import.
   Given a HTML file (or collection of such files) created externally, you can then import that file and it will be broken up into chapters.

It appears that very few people are using the import facility.  The following graph shows that roughly 10% of chapters, books and course offerings at this institution involved the use of the import method.

After the presentation, someone from the institution’s central L&T group came up for a chat. Indications are that between he (supporting another teacher) and I (in my course), we potentially account for all use of the import functionality.  Further discussions reveal issues arising from people copying and pasting from Word into the Book module.

It appears that content authoring might remain an unresolved problem.

Advertisement

Exploring Moodle Book usage – part 9 – Strange courses

Time to explore some of the strange courses that have been identified.  There are currently two types:

1. courses with many individual Book resources; and
2. courses with huge Book resources.

Strange books

Courses with many books

Back in part 2 there appear to be a number of courses that have more than 50 individual book resources.  That seems a bit excessive.  Wonder what that indicates? What are these courses (one of mine might be part of this group)? Is there something wrong hidden in these figures (e.g. are some of them hidden)?

There are 15 course offerings with more than 30 Books.  7 of these are offerings of the course I teach.  The remaining 8 are split between 3 different bridging/preparation courses.

If the line is drawn at 20 books, then there are 23 course offerings drawn from another 2 education course , a nursing course, and another 2 bridging courses.

Courses with huge books

In part 6 it was discovered that there are books with 100 chapters (individual web pages). Most of the books had less than 20.

There are 51 offerings with books with greater than 26 chapters (25 is the upper limit for 2015).  This converts into 20 courses with a number being offered multiple times.

Further stats about these courses – as per the Word doc

Imported books

Part 6 in the series also outlines details of the number of books that were imported – only 9.8% of chapters imported, from 10.2% of books, from 11.8% of courses.

Comparisons

The aim now is to take a closer look at these strange course to learn more about them.  The following graphs will report different stats about the courses that fall into the three different categories

1. IMPORT – courses that have used the Book import facility
2. BIG – courses that include big books (greater than 26 chapters)
3. MANY – courses that have many books (greater than 20 books)

% of online only students

The institutions has a number of different types of student, including online only. The following graph shows the % of online students in each course.

It shows that the IMPORT courses tend to have a higher concentration of online students, with the MANY courses next. The BIG courses, tend to include more courses that have no online student.  There is online 1 MANY course with no online only students. Only 3 IMPORT courses with no online students.

# of revisions per book

On the other hand, the following graph shows that the BIG courses, tend to have more revisions, including one that has 701 revisions.  That has to be explored a bit more – most of the BIG courses with high revisions are from the same allied health discipline. They are all – except from 2 (100% and almost 20%) – on-campus courses.

When they are read by students

The following heatmaps aim to represent when these books are read.

Import books

The following heatmap shows when books in courses that used the IMPORT facility were read by students. It seems to suggest that the IMPORT facility was really only heavily used in 2015.

Big Books

The next images shows that courses with BIG books started a bit earlier and tend to be first semester occurrence.  Perhaps suggesting – like my course – a big offering in the 1st semester and a smaller repeat in the 2nd?

It also seems to suggest a tendency for the books in these courses to be read more earlier in the semester and also to be read on week days. In the first half of 2015, there’s even a trend for reading more on Monday and Tuesday.

Many Books

When are they modified

The next set of heatmaps are for the same collection of courses, however, these show when there were modify/create events. How often and when were they changed.

Import

The import courses align somewhat with the above.  Really only seeing action in 2015.

And much of that importing in 2015 is taking place on the weekends – I have a suspicion that this might be my course having an influence.

Big

The BIG courses have modify/create spread over the years. However, it does appear that there’s a tendency to modify/create happening earlier in the semester

Many

The MANY courses are – overall – showing a bit less activity. The pattern in the first half of 2015  suggests that a single course is having a fair impact. If this is my course, then it might be worthwhile taking it out and running these again.

Remove EDC3100 – students viewing and printing

The next 6 heatmaps repeat the 6 from above, but with any of the offerings of the course I teach removed. This appears to radically change the picture.

Import

Reveals the drawback of the combined heatmap approach. It appears that my course offering has a great deal of activity, which in the above Import heatmap “overwhelmed” some of the other courses.  This map shows that use of the import facility starting earlier and being used. It shows in S1 2015 quite a lot of activity toward the end of semester.

I was interested in the balance between viewing and printing.  The following heatmap is for the same set of courses as the above, but shows only the events associated with printing out a chapter or book.  It is suggesting that these courses are rarely printed

Big

As mentioned below, this map appears no different with my course removed. Identifying that my course isn’t a BIG course.

The following heatmap is for the same courses, but only the print actions. When compared to the other groups of courses, it suggests that the BIG courses have more print actions. Suggesting the bigger the books, the more likely students are to print them.

Many

However, my course is one of the MANY.  Removing my course here allows the activity of the other courses to come through, in particular the small number of courses in first semester 2012 that had fairly consistent student usage throughout semester.

And the following again shows just the print actions. It suggests that the MANY courses print a bit, but not as much as the BIG.

Remove EDC3100 – when modified

Import

It appears that the modifications occur toward the start of the semester and rarely continue during semester (Late Feb – S1; Late Jun/Jul – S2; Late Oct – S3)

Big no 3100

This appears to be pretty much the same map as above – indicating that perhaps my course is not one of the BIG courses.

Many 3100

A more major change here. Major reduction in modifications during semester, limiting modifications largely to the start of semester. With an exception in S1, 2012. Is this the course that had students doing it?

Student updates – no edc3100

There are 2 course offerings that have students modifying Book resources. The following heatmaps foll

Exploring Moodle Book usage – Part 8 – linking to and from

Natalie writes about how she’s working a new practice into how she responds to student queries. It’s a process in which she attempts to model an approach to answering the query and including links to relevant sites. This is a practice that I use a fair bit, especially with the Moodle Book resources in my undergraduate course. This post seeks to explore my own practice, but also how wide spread that practice is in others.

This post picks up on work and ideas from an earlier post. The Moodle Book module helps create/manage collections of web pages. My interest is to explore how much people are using them as web pages, rather than just dumping grounds for print material. One of the main affordances of the web is links. The prior post found that around 15% of the book resources contain no links. It also found that the median number of links per book has grown from 11 through 17. It also shows that there are some books with hundreds of links. It also showed how the number of links in the book resources I produce has grown with the median hitting about 25.

This post will seek to refine and expand this exploration a bit, including

• Looking more closely at what the book resources are linking to: other book resources, multimedia, other institutional resources etc.
• Whether or not book resources are being linked to from course forum posts — this will become the topic for another post.

Link break down by destination

First, lets break down links by three destination categories:

• LMS – to within the institutional LMS
• USQ – other non-LMS links for the institution; and,
• OTHER – everything else.

EDC3100 2015 2

The following graph shows the breakdown between the three categories for each book resource in the most recent offering (for which I have data) of the course I teach. It shows that typically links to the broader web tends to be the largest category, followed by other links within the LMS, and finally links to the institution.

But there is also some variety depending on the purpose of the specific book.

The book with the largest number of links (almost 100) is also one of the longer books.  It is also the book that contains all of the assessment related information (what is required, where to submit, how to request an extension, how to query marking etc). It includes 25 links to other parts of the study desk and 70 links to other sites.

The book with the most LMS-based links is titled Conclusions Week 1. It provides a summary of what was learned that week and includes links back to the specific pages in the various books for that week.

The book with the second most number of links, also only has external links.  This book aims to show folk how to use resources out on the web and literature to learn how to use a new digital technology. Hence it has a large number of links out onto the broader web and that’s even before Diigo widgets that contain the most recent collection of links shared to the course Diigo group are rendered.

Evolution over time – EDC3100

The next graphs show the evolution of links from the EDC3100 books over time.For each offering of the course it shows the number of links per book of each type: LMS, institutional, and other.

There is no graph for institutional links (NOT the LMS) because from 2013 S2 it flat lines indicating – apart from one – none have links to non LMS institutional resources.  I imagine this may well be very different in other courses.

The first graph shows the evolution of LMS links. The median starts and remains at about 4 LMS links per book. With a slight growth at the top end in recent years.

The next graph is for links outside the institution. The growth in these links is a bit more evident. The median growing from around 9 to around 13 and the upper from 40 to 50

Evolution over time – All courses

The following graphs show how many links of each type (LMS, USQ and other) found in all book resources in each year. These graphs do not include EDC3100, the course I teach.

There is a broad common trend in all three.  The number of books with large number of links increases over the years. However, that number is largely insignificant as the vast number of books contain much fewer links.

The first graph shows the number of LMS related links in each book. It shows that in 2012 almost all of the books had no such links. 75% of books in 2012 has less than 5. 50% had 0. As the years progress there are a growing number of books with quite large numbers of links, with the maximum reaching 400. This corresponds to the appearance of some books that are very large. By 2015, 75% of books has less than 7. 50% less than 2.

The next graph focuses on number of links to institutional resources (not in the LMS) in each book. The basic shape is much the same. Starting quite low and then having a number of books added in 2014/2015 with quite large numbers. However, the numbers involved fewer numbers of links than the LMS graph (e.g. the maximum gets to just over 200, rather than 400). It also shows that the overall trend is a bit down.

In 2012, 75% of books has less than 4 USQ links. By 2015, that had reduced to 3.

The following graph focuses on links onto the broader web.  The numbers are higher.

In 2012, 75% of books had less than 17 links, 50% had less than 7. 2015 was largely the same.

What Moodle links exist?

Next step is to take a closer look at the Moodle links. What type of activities and resources are being linked to.

Will people be linking to anything? Mostly resources? Activities?

EDC3100 2015 2

Start with the latest version of the course I teach.

It shows that most of the links in the latest offering are to other Moodle book resources. Over time I’ve made an effort to link between books to show the interconnection of ideas.

Surprisingly, it also shows links to the discussion forums.  These are going to be offering specific since each offering of a course uses different forums.  Interestingly, take away the book links, and the forum links make up almost 61% of these links.

Would an analysis that divides links between activities and resources indicate anything interesting about learning design?

All 2015 S2 – but EDC3100

So how do all the other courses use of Books form 2015 S2 compare?

Lots of resource focus – book, pluginfile, printing the book, equella.  But also links to the quizzes and forums.

Interestingly for me it highlights 3 other courses using BIM. 2 of which I don’t teach.

But how widespread is this?

As discovered above

In 2012, 75% of books has less than 4 USQ links. By 2015, that had reduced to 3.

Meaning that those 1000+ links to Moodle books above were found in a fairly small number of books. Perhaps a couple of the quite large books. More to find out

• How widespread are these links? How many books?
• What type of links are they?

Something for later.

Exploring Moodle Book usage – Part 7a) – when are they modified

In a previous post I generated various representations of when Moodle Book resources were being used and some indications of when they were being created. What I didn’t do in that post was generate a calendar heatmap of when the Book resources were being create and modified. This is of interest because I’m wondering whether or not these resources (web pages) are being modified throughout the semester, or just at the beginning.

The following corrects that. It starts with calendar heatmaps showing when I’ve edited/created the Book resources in my course. I’ve tended – or least eventually developed – a practice of developing and changing the books as the semester progresses. I think I’m strange – turns out that I’m apparently not that strange at all.

EDC3100

Each of the following shows some level of change prior and during semester. Some even show changes after the end of semester.

For most of the semester, the weekend are the days that tend to be busiest in terms of edits. Showing an unhealthy practice of using weekends to catch up.

In S1 I also teach on-campus students, which is typically done during the week. Perhaps that limits the edits that happen during the week in S1.

S1 typically starts early March and finishes late June/July. S2 typically starts late July and finishes early November.

2012 S2

Fair bit of work before semester and on-going.  Fair bit of work on saturday and sunday.

2013 S1

Lot of work in the leadup. Not so much during the early part of the semester.

2013 S2

More front ended activity before and early in semester.  Late in the semester not much.

2014 S1

More weekend editing.

2014 S2

A generally lighter collection of updates.

2015 S1

More before semester, lightish during.  Much of the work during is occurring late in the week.

2015 S2

A more even spread across the week.

Courses other than EDC3100

So what about updates in all the other courses?

Well, that is a surprise.  Indications are that at least someone is modifying a Book resource most days throughout the year.  Even in some circumstances well before or well after the year.

The question with these now is whether this spread is due to the number of book resources or number of courses using the book.  A topic for further exploration.  Perhaps by doing a heat map showing the % of courses that have books being modified?

2012

2013

2014

2015

Exploring Moodle Book usage – Part 7 – When are they used?

The last post in this series looked briefly at the contents of Moodle Book resources. This post is going to look at when the book resources are used, including:

• What time of day are the books used?
• When in the semester are they used?

By the end I spent a bit of time exploring the usage of the Book resources in the course I teach.

What time of day are they used?

This is a fairly simple, perhaps useless, exploration of when during the day. More out of general interest and laying the ground work for the code for the next question.

Given the huge disparity in the number of views versus print versus updates, there will be separate graphs for each. Meaning 3 graphs per year.  For my own interest and for the sake of comparison, I’ve included a fourth graph which is the same analysis for the big 2015 offering of the course I teach.  This is the course that perhaps makes the largest use of the Book and also the offering in which  I did lots of updates.

The graphs below show the number of events that occurred in each hour of the day. 12pm to 1am, 1am to 2am,…and so on.  Click on the graphs to see expanded versions.

There is no graph for prints per hour for 2012 as there were none in the database. This appears likely to be a bug that needs to be addressed.

Overall findings from time of day

Growth – The maximum number of events has grown each year (as expected given earlier indications of growth).

• max views per hour: 2012 just less than 35K to 2015 over 150K
• max prints per hour: 2013 just over 400 to 2015 over 1500
• max updates per hour: 2012 just over 500 to to 2015 over 6000.

Similarity – The overall of shapes of the graphs stay the same, suggesting a consistent pattern in interaction.

This is especially the case for the viewing events. Starting with a low number from midnight to 1am, a on-going drop in events until 5am when it grows until the maximum per hour between 11am and midday. Then there is a general drop away until 7pm to 8pm when it grows again until dropping away after 9pm

Views per hour each year

2012

2013

2014

2015

EDC3100 2015 S1

Prints per hour each year

2012

2013

2014

2015

EDC3100 2015 S1

Updates per hour each year

2012

2013

2014

2015

EDC3100 2015 S1

Calendar Heatmaps

A calendar heatmap is a fairly common method of representing “how much of something” is happening each day of the year. The following aims to generate calendar heatmaps using the same data shown in the above graphs. The plan is to use the method/code outlined on this page.

It requires the generation of a two-column CSV file. First column the date in YYYYMMDD format and the 2nd column the “how much of something” for that day. See the example data on the blog post.  Looks like it might be smart enough to figure out the dates involved.  Let’s see.

It is, but doing all of the years together doesn’t work all that well given the significant increase in numbers of courses using the Book as time progresses and the requirement for the heatmap to use the same scale for all years. As a result the 2012 usage doesn’t show up all that well. Hence each of the years were mapped on separate heatmaps.

The following calendar heatmaps show how often the Book resources were viewed on each day. The events counted are only those for Book resources from courses offered in the given year. In 2012, 2013 and 2014 this means that there is a smattering of views of a books early in the following year (semester 3 stretches from Nov to Feb). There is no similar usage for the 2015 books because the data does not include any 2016 events.

The darker the colour the greater the use. In the 2012 image below you should be able to see a tool tip showing a value of 81 (out of 100) that is quite dark, but not the darkest.

2012

The 2012 map seems to establish the pattern.  Heavy use at the start of semester with a gradual reduction through semester. A few upticks during semester and toward the end of semester.

I no longer have easy access to specific dates for 2012 and 2013. The 2014 heatmap has some specific dates which should broadly apply to these earlier years.

2013

2014

The institution maintains a web page that shows the important dates for 2014, it includes:

• March 3 – Semester 1 starts.
  Course websites open 2 weeks before this date – 17th Feb
• June 16 – Semester 1 exams start.
• July 21 – Semester 2 starts
  Course websites open 2 weeks prior – 7th July.
• November 3 – Semester 2 exams start.
• November 17 – Semester 3 starts.

2015

The semester 1 2015 offering of my course had the following due dates for its 3 assignments

1. 30th March – which appears to coincide with a heavy usage day.
2. 4th May – also a slightly heavy usage day, but not as heavy.
3. 15th June – two somewhat heavy usage days before and on this date.

Raising the question of what the heatmap for that course might look like – see below

EDC3100 – S1, 2015

Focusing just on my course the increase in usage just before the due date for the assignments is more obvious. One of the reasons for this is that all the Assessment information for the course is included in a Moodle Book resource.

Other time periods relevant to this course are:

• April 6 to 17 – the two week mid-semester break; and,
  Which correspond to two of the lightest periods of usage of book resources.
• May 18 to June 5 – a three week period when most of the students are on Professional Experience within schools.
  Which also corresponds to a light period of usage.

The two heaviest days of usage are the 9th and 10th of March. The start of Week 2 of semester. It’s a time when the pressure is on to get a blog created and registered and start completing learning paths.

After the peak of the first three weeks, usage of the Book resources drops to around 50% per day.

Questions to arise from this

• Does the learning journal assessment item for EDC3100 change when students interact with the course site?
• Is the pattern of usage (down to 50% a day) indicative of students turning off, or becoming more familiar with the approach?
• Does the high level of usage indicate

It also begs the question about whether particular offerings of the course show any differences.

2012 – S2

The 2012 S2 pattern is quite a bit different. It is a bit more uneven and appears to continue well after the semester is finished.  This is due to this being the first semester the course used the Book module and also because there was a semester 3 offering of the course for a few students that used the same resources.

The 2012 heatmap also shows a trend that continues. i.e. usage of the Book resources continue well past the end of semester. It’s not heavy usage, but is still there.

Question: is that just me, or does it include students?

2013 – S1

2013 S1 is a bit different as well. Lighter use at the start of semester. A bit heavier usage around assignment due dates. My guess is that this was still early in the evolution of how the Book was being used.

2013 – S2

This map seems to be evolving toward the heavy use at the start of semester.

2014 – S1

And now the pattern is established. Heavy use at the start of semester and in the lead up to Assignment 1. A slight uptick then for Assignments 2 and 3. With the light usage around Professional Experience evident.

2014 – S2

2015 – S2

 
What about just the students?

The following shows just the student usage for the 2013 S1 offering. Not a huge difference to the “all role” version above suggesting that it is students who are doing most of the viewing. But it does confirm that the on-going usage of the Book resources past the end of the semester are students who appear to have found some value for the information post the course.

Exploring Moodle Book usage – Part 6 – What do they contain?

Part 6 of this series diverges a bit from the last post and moves away from what people are doing with the Book resources to focus on the contents of the Book resources themselves.  Questions I’m hoping to explore in this post include:

• How long are the Book resources?
  Measured perhaps in number of chapters, bytes, and perhaps textual word count.
• Are the Book’s web or print documents?
  Do they include links? To other books in the course? To external sites? Which sites? Do they include multimedia?
• What does one book with 500+ links actually link to?
• How readable is the text?

NOTE: Click on the graphs below to see larger versions.

How long are the Book resources

A Moodle Book resource is a collection of “chapters” and “sub-chapters”, which are essentially web pages. The following starts looking in more detail at these chapters and their contents.

Where did they come from – import or create?

Looking more closely at the chapters provides an opportunity to find out how they were created.

Each chapter has a field importsrc which specifies the name of a file from which the content was imported.  Indicating that the chapter was created by uploading a already written file, rather than using the Book online editing interface.

Analysis shows that only

• 9.8% (2397 out of 24408) of chapters are imported;
• these belong to 10.2% (287 out of 2801) of books; and,
• 11.8% (44 out of 374) of courses.

i.e. ~ 90% of chapters, books and courses are created by using the online Book interface.  Not a great way to create.

How many chapters per book?

The next step is to have a look at how long each book is based on the number of chapters. This isn’t a great indication of length because each chapter is simply a web page, it could be quite short, or quite long.

The following graph shows the number of chapters in every book grouped by year. Overall the number of chapters stays pretty much the same.  However, there are a couple of strange outliers tending toward 100 chapters in a book. The median number of chapters per book has increased from 6 in 2012 to 8 in 2015.

The total number of books shown in the above graph for each year is a bit out from earlier data. I will need to come back to these analysis and nail down what courses/books are counted in each analysis.

How many words in each book?

To get a better idea of the size of books the aim here is to convert the chapter content to plain text and do some analysis of the text.  This is where the beauty of Perl (confirmation bias) comes to the fore.  There’s a module for that.

The following graph maps the number of words for each book by year.  It shows that in 2014 and 2015 the number of words per chapter/book was certainly getting longer.  The median went from 1157 words per book to 1718 per book (with a dip in 2013 back to 1004 words per book). The upper limit moved from 5282 words in a book to 6930 words per book. Scarily, there are outlier books that are approaching (and in some cases bypassing) 60,000 words in length.

To give you some idea of read time, I’ll use Medium’s method for calculating read time (ignoring images) to convert the numbers into minutes to read:

• Around the median word count – 1700 words – equates to about 6.1 minutes.
• The maximum upper word count – 6930 words – equates to about 25.2 minutes.
• The outliers – around 60,000 words – equates to about 218.2 minutes, which is approaching 4 hours.

Adding to this is that I’m not sure the typography and design of your typical Moodle Book is going to match what you might expect on Medium. Not to mention that Medium don’t mention if their average adult reading spead (275 words per minute) is for words on print or screen.

Readability?

The module that calculates words also does readability tests, including the Flesch reading-ease test. The following graph shows the results on that test for each of the books grouped by year.

Grain of salt – The graph does exclude a number of books that achieved negative results on the test. Initially, it appears that this may be due to the conversion to text only not handling some special characters which worsen the readability.  (Apparently it is possible to get a negative value on the test). This may also be decreasing the “reading ease” of other books.  This will be examined more closely later.  But then again, quoting Wikipedia

While Amazon calculates the text of Moby Dick as 57.9,^[9] one particularly long sentence about sharks in chapter 64 has a readability score of −146.77.

The median moves between 43.7 and 47.3, which is apparently around the 45 that Florida law requires for life insurance policy (thank you again Wikipedia).  However, the lower bound loiters around 5 suggesting very difficulty to read.  Wikipedia suggestions 30 to 50 as being the range for “college” and being difficult to read.

And my books?

Which has me wondering about mine. I think I’ve developed a tendency to reading difficulty.  The following graph shows the distribution for the latest offering of my main course that is contained in the data set.

That’s a nice-ish surprise.  Median at 60. Worst is 40 and best is 77. With better than 75% of the books above 50 which is the lower bound of the 10th  to 12th grade boundary.

However, I believe these results may be a little padded by the fact that I write most of my books in straight HTML. Meaning there’s no increase in complexity because of the difficulty of converting it into clean text.

Which has me wondering about the evolution of readability.  The following graph shows the results from all offerings of the course that use the Book. A bit of a dip at the start with a small upward trend over time.  Not bad – but then of limited use given the limitations of this type of thing.

What about links – links per book?

One of the questions I’d like to answer is whether or not the people using the Book are using it as a poor-man’s replacement for a collection of paper, and how many are using it as a collection of web pages.  First exploration of this question is the rough indicator of how many links per book?

The following graph shows the number of links per Book per year. “Link” is defined here as any type of link, excluding a link to a style sheet. That means links to images, youtube videos etc are all counted as links.

As the graph shows there are a large number of books that have no links.  The median number of links is increasing each year. Starting at 11 in 2012 and moving through 13, 14, and finally 17 in 2015.  As the graph shows there are some major outliers with some Books having hundreds of links, including some with over 500 links.  These might include some of the very long books included above, but it might also include other books that contain huge numbers of links

In terms of books with very few links in 2012, 15.4% of the books had less then 3 links (remember that includes images, links, embedded videos etc) with 2014 having 16.1% and 2015 having 15.3%

Links per book in EDC3100?

For a quick comparison, the following graph shows the number of links per Book for EDC3100 (the main course I use the Book in). Over time I  have been trying explicitly to think of the Book resources as collections of web pages.

The median # of links per book for all courses moved from 11 to 17. In EDC3100, the median has moved from 14 at its lowest (2013 S2 – a bad semester for links) up to 30 in 2015 (both semesters).  Similarly, the upper range for all courses ranged from 46 to 74 (driven by some truly large link numbers), for EDC3100 the upper range went from 43 in (2013 S2) up to 111 in 2015.

Exploring types of links a bit more

The above couple of link graphs are limited because I really haven’t yet explored the diversity of link types that are included.  I had removed CSS links, but not script links.  I also haven’t split apart the different types of links. An examination which might shed some light on those strange books with 500+ links. Time than to explore.

Will try to identify the different types of links, generate stats for all the types, but when counting links, limit to more standard types (img/a)

Types of link to exclude from the count of links: iframe, embed, object, meta – handle link better.

The presence of <tag meta=”generator” looks like being one way of identifying chapters coming from Word.

Cleaning up the links does bring the numbers down a bit. e.g. the media for 2015 goes from 17 to 15, but the other medians stay the same. The upper for 2013 onward comes down by 1 to 3.

What about the 500+ books? What are those links

I’m interested in the books that have 500+ links.  What are they linking to?

One with 517 links has 510 <a links and 7 <img links. What are those 517 <a links?

Lots of internal links and all sorts of other links – other book chapters, readings. Looks like it might be a large book, is it?

29 chapters and 32,871 words – so a big, all in one book.

Exploring Moodle Book usage – Part 5 – more staff and student use

Continuing the exploration of how the Moodle Book module is being used, this post picks up from the last and will

• Revisit the who is updating/creating posts, including data from the second half of 2015.
• Explore the balance of all actions (print/view/update) by staff.
• Explore the balance of all actions by students.

Who is updating/creating posts

The last post included a graph that showed generally (apart from two course offerings) that the core teaching staff appear to be doing the creation of books.  That graph had a few problems, including

• Limited data from the 2nd half of 2015.
  Due to the switch in how Moodle logged events.  Need to handle the new log format.
• Didn’t handle all roles.
  Appears there are some non-standard Moodle roles that the previous query didn’t handle.
• Handling deleted books and chapters.
  I believe this is an issue for the new logging process which has connections back into the book and book chapters table. Which works nicely until books/chapters are deleted.

With those changes fixed, the following graph emerges show how many times each of the roles updated a Book resource in every course.  The changes between the following and the same graph in the last post, includes:

• Significant increase in the number of updates for most roles (e.g. examiner up from 21968 to 31343; assistant examiner has almost doubled from 5144 to 10708)
• Addition of the UNKNOWN role not in the previous graph

It should be noted that the following graphs do not include ~20K updates that I did in one course in one semester.

And I thought it would be interesting to break down the updates by year to see what if there was any growth. Given the growth in the number of courses using the Book (17 in 2012 to 152 in 2015) there should always have been some growth.

The graph above shows examiners making 2152 updates in 2012 and 13649 in 2015.  That’s a 6.3 times growth in number of updates for 12.6 times growth in the number of updates. Or, alternatively in 2012 a course examiner (on average) made 179 updates. In 2015 a course examiner (on average) made 90 updates.

Suggesting that the examiners are making less updates. Perhaps farming out the updating to other staff. The growth in edits by moderator and assistant examiner roles in 2014 and 2015 suggest that.  But more exploration is required.

Role balance of actions

Updating/creating is not the only action that can be done with a Book, you can also view and print parts or all of a Book resource. This step aims to explore what balance of actions each of the roles are involved with

For this purpose I’ve grouped log events into the following actions someone can perform on a Book

• view – view a chapter or the entire book online
• print – print a chapter or entire book
• modify – delete or update a chapter/book
• create – create or add a chapter or book
• export – use the export to IMS option

The above updating/creating graphs including both modify and create actions.

The table shows the total events on all books by all roles from 2012 through 2015. It shows how viewing the book is by far the most prevalent action, accounting for 97.6% of actions.

Interestingly, at least for me, is that the percentage of modifications (1.1%) exceeds the percentage for printing (0.9%). I assume this is due to my outlier behaviour in 2015 in modifying a huge number.  Indeed it does.  The numbers in brackets in the table indicate a recalculation taking out that outlier.

Action	# actions	%
View	5040285	97.6  (98)
Print	46162	0.9 (0.9)
Modify	56754 (35867)	1.1 (0.7)
Create	18537	0.4 (0.4)
Export	1	1.9373E-05

Given the preponderance of viewing, the graphs tends to be a little less than useful by role. But the following look at usage by students and examiners.

 

Student usage

The graph below shows the spread of actions by students with the books. It shows that the most common action performed by students is viewing books. The table following the graph provides the raw data for the graph.

Both this table and the one below for examiners show no print actions.  This suggests a bug in the analysis.

Another interesting point is the dip in printing between 2014 and 2015.  Even though the number of courses using books, and the number of views by students on books increased from 2014 to 2015, the number of print actions dropped. I wonder if this has anything to do with the large number of modify/create actions by students in 2015. Were the students creating the books/books created by students less likely to be printed?

Year	View	Print	Modify	Create
2012	386101		41	2
2013	812133	4487
2014	1447190	20310
2015	1967047	15198	1335	28

 

Examiner usage

The graph below shows the spread of actions by examiners with the books. The table following the graph provides the raw data for the graph.

The relative increase of modify/create actions by examiners between 2014 and 2015 is another indication of the 20000 updates I performed in 2015.

The views and prints by examiners drop between 2014 and 2015

Year	View	Print	Modify	Create
2012	7193		2072	80
2013	26774	105	4850	495
2014	35855	647	8364	1833
2015	35185	452	26790	7746

 

Further questions to explore

• What are the UNKNOWN roles?
• How are the updates and other actions shown above distributed between users? Are there a small number of users making up the lion share of the actions (e.g. me and updates in 2015; and the one or two courses that had students updating books).
• How many chapters do each student read? What about printing? Do they print and read online?
• What is happening with print actions in 2012? Was there really no-one printing books?
• Were the books created by students less likely to be printed? Did this account for the drop in print actions by students between 2014 and 2015? If not, what did?
• Remove my 2015 outlier actions from the examiner actions graph and see what changes are made.

Exploring frameworks to understand OER/OEP

Some colleagues and are re-starting an exploration of OEP in Initial Teacher Education (ITE). A first task is an attempt to get a handle on what has been done/is known about OEP/OER. Yes, we’re looking for spectrums/frameworks/models etc that help map out what might be done with OEP/OER.  We’re interested in using this to understand what’s been done around OEP within ITE and also what we’ve already done.

The following is a summary of a quick lit review. No real structure and includes a range of strange notes.

OER adoption: a continuum for practice

Stagg (2014) offers the following continuum of practice

The proposed model seeks to acknowledge the complexity of applied knowledge required to fulsomely engage with open education by examining practitioner behaviours and the necessary supporting mechanisms. This conceptual model aims to be of use to both practitioners and also those responsible for designing professional development in an educational setting.

A Google Scholar search reveals some use this continuum.

Including Falconer et al (2016), which includes

We view our fourth category, enhancing pedagogy, as fundamentally different to that of producing high quality materials efficiently or cost effectively, in that it is underpinned by altruistic positions rather than a business model approach. It puts its emphasis on the value of the OER development process, rather than on the value of the OER content produced. (p. 99)

Through our analysis, some fundamental tensions have become apparent that will need to be resolved if the purposes of OER release are to be realised. (p. 101)

This limits imposed by a reputation-building motive are exacerbated at present as higher education institutions are encouraged to become increasingly competitive, elevating the importance of brand recognition. The consequence is a move away from risk-taking, towards a demand for predictable quality outcomes. This discourages innovation unless direct benefits can be proven in terms of new markets, student numbers, or shared costs of development and teaching. The benefits of OER in terms of institutional showcasing and attracting potential students, may prove attractive to institutional managers and gain institutional support for OER, but unless culture changes, they place inherent limitations on efficiency gains and the adoption of more open practices which are ultimately founded on a commitment to academic commons. (p. 102)

And develops some frameworks/continuums

and

Assessing the potential for openness

Stagg (2014) is also cited by Judith and Bull (2016)

While this literature has been significant in driving forward the open agenda, there has been relatively little published about the practicalities of implementing openly licensed materials in higher education courses (p. 2)

which raises the question of just how much more difficult the idea of implementing open educational practices are going to be. i.e. if sharing materials is hard enough.

OER engagement ladder

Masterman and Wild (2013) bring in the OER engaement ladder, which is talked more about in this blog post. (Interestingly the institutional repository URL for the full research report is now broken, but blog posts and slideshare resources remain)

References

Falconer, I., Littlejohn, A., McGill, L., & Beetham, H. (2016). Motives and tensions in the release of Open Educational Resources: the JISC UKOER programme. Australasian Journal of Educational Technology, 32(4), 92–105. doi:10.14742/ajet.2258

Judith, K., & Bull, D. (2016). Assessing the potential for openness: A framework for examining course-level OER implementation in higher education. Education Policy Analysis Archives, 24(42). doi:10.14507/epaa.24.1931

Masterman, L., & Wild, J. (2013). Reflections on the evolving landscape of OER use. Paper presented at OER13: creating a virtuous circle, Nottingham, UK

Stagg, A. (2014). OER adoption: a continuum for practice. Universities and Knowledge Society Journal, 11(3), 151 – 164. doi:10.7238/rusc.v11i3.2102

Exploring Moodle book usage – part 4 – students and initial use

Yesterday’s part 3 in this series of posts continued the higher level examination of book usage. i.e. what types of courses use the Moodle Book module (the Book). This post is going to continue that a little and then start to make some forays into looking more closely at how resources produced using the Book are actually used. In particular, it’s going to look at the following:

• Compare the number of online students in courses that use the Book, versus those that don’t use the Book.
• Who is actually creating and revising the Book resources?

At this stage, I’m not sure if I can answer these questions with the data I have to hand.

Yep, that worked.  Still a fair bit to do, the next post(s) will

• Revisit the staff usage of the Books to include more recent data and fix some of the other limitations of what’s below.
• Start exploring how (if?) students are using the Books.

Identifying type of students in courses

The last post identified that the Book is generally used in larger courses. A possible implication of this is that the Book is more likely to be used if the course has distance education/online students. The thinking here is that such courses have historically had print-based study guides, which could be converted into the Book module. Also, that on-campus courses are more typically going to rely on lectures and tutorials as the primary form of teaching method. This links directly back to the idea of horsey, horseless carriage thinking.

To explore this further I need to identify whether or not the current data set will allow me to identify the types of students….turns out group allocation allows this.

Plotting the number of online students enrolled in courses using the Book gives the following graph.  It shows that the number of online students in courses using the Book was initially quite low. For example, in 2012 50% of courses using the Book had less than 4 online students. Many of that 50% had no online students. In fact, the only courses using the Book in the first half of 2012 had no online students.

However, over time the number of online students in courses using the Book increased. In 2015, though there remained a large number of these courses that few if any online students.

Rather than focusing on the number of online students in courses using the Book, the following graph focuses on the percentage of online students in those same courses. It shows that in 2015 there was a significant increase in courses with higher percentages of online students starting to use the Book. Before that a majority of courses using the Book had less than 20% online students. 2012 appears have included only 1 course that had online students – the big outlier with 100%.

For me this raises a couple of interesting questions

• How and why did the courses with 0% online students use the Book?
  The use of the Book by these courses challenges my assumption.
• Why does 2015 appear to have been a turning point for using the Book by courses with higher percentage of online students?
  My current guess is that this correlates with the cessation of the previous method for placing traditional print-based study guides online. That tool stopping meant the courses had to look for an alternative.

 Who is creating the book resources?

My experience is that creating resources using the Book module is not necessarily a straight forward process. I’ve kludged together various tools and practices to reduce the difficulty, but I’ve heard other staff give up on using the Book because they couldn’t. This has me wondering who and how these Book resources have been created in other courses.

Answering this question requires taking a closer look at who is doing what with the Book resources, which requires a bit of work. It’s also the foundation for most of the subsequent interesting analysis.

As a result of getting this working, an interesting question suggested itself

• For each course, how many “events” happen around the books in those courses?
• What percentage of events for the whole course, do those book events represent?
• What about breaking those events down into read, change, and print?

The first rough cut at answering the question is given in the following graph. It shows the number of update events associated with Book resources grouped by each user role. It apparently shows that the core teaching staff (examiner, moderator and assistant examiner) are making most of the updates.  Interestingly, the student role is next in line in terms of number of updates. But there are some insights/limits/caveats to this graph.

The insights/limits/caveats include

• The 1376 updates made by students are from two courses only. One course with 1335 (~97%) of the updates. Indicating a specific pedagogical choice for that course.
• There was one course offering where the idiot examiner (i.e. me) almost doubled the number of updates by examiners. This offering has been excluded from the above graph.
• The notion of an “update” event doesn’t provide any indication of how much was updated/created.  It might be as simple as deleting a character, or perhaps importing a whole new book.
• The above data (so far) does not include data from second half of 2015 when the new Moodle event logging was implemented.
• The mapping between old and new style logging needs to be smoothed out
• There are events that aren’t logged for the book (e.g. this tracker item).
• The mapping of logged events to changes to the book need to be rechecked.

 

Exploring Moodle book usage – part 3 – who and how much

Continuing on from yesterday’s post this post seeks to explore a bit further how the Moodle Book module (the Book) is being used at my current institution.

The plan is that this post will explore:

• What percentage of all courses are using the Book?
• How big are the courses using the Book?

Follow up posts should look to explore

• Compare the number of online students between courses using the Book and those that don’t.
• Which staff are using/creating resources with the Book?
• How many students actually use the Book – leading into questions of how they use it?
• Exploring the content of the Books might also be useful.

Percentage of courses using the Book

Yesterday’s post showed the number of courses using the Book per year has grown from 17 in 2012 to 152 in 2015. To put that into perspective what percentage does that represent of all courses offered.

There’s a difficulty in calculating the percentages. I’m not entirely certain what makes an “official” course. The following is based on the criteria of being a course with at least one student enrolled, that also has a Moodle course site.

Year	% courses using the Book
2012	2.02%
2013	3.01%
2014	6.36%
2015	10.35%

 How big are the courses using the Book?

Flowing from that, I’m wondering whether the courses using the Book are big courses or small courses?

The next graph shows the number of students per course, for each course using the Book. The following graph shows the number of student per course, for all courses. The aim is to see if there is any difference in the size of the courses using the Book.

The indication is that the courses using the Book tend to be larger than the typical course. Overall the median course has between 11 and 13 students. Compared to a median between 85 and 142 for the courses using the Book.

Possible implications might include:

• The larger courses see the need to make use of the Book, perhaps as a source of information distribution.
• The larger courses are more likely to have existing study guide material.
• The larger courses are more likely to have significant numbers of online/distance education students.
• The larger courses may be more likely to have assistance in using the Book.