Category Archives: bim2

BIM testing and fixes

A journal of fixes and testing of BIM. Aim here is to address some minor issues with integration with my current institution’s Moodle instance thereby providing a minimum working version for installation. As per yesterday’s planning the hope is to make further changes based on this foundation.

Result is a slightly tweaked version released via Moodle contrib. This will be the foundation for some tweaks, though I can feel time slipping away.

Latest version of BIM and PostgreSQL

The institutional Moodle instance uses Postgresql. Thanks to playing with MAV I know have a version of Moodle running with Postgresql (aka fred). The plan here is to install BIM on that instance and test it

  1. What’s the latest BIM?

    MOODLE_25_STABLE is the latest, but MOODLE_24_STABLE is what I need for this work, institutional Moodle version still at 2.4.

  2. Install it on fred.

    Get the source

    git clone https://github.com/djplaner/moodle-mod_bim/
    mv moodle*bim bim
    cd bim
    git branch MOODLE_24_STABLE
    git pull origin MOODLE_24_STABLE

    Visit notifications as the admin user on fred and install of BIM successful.

    BIM not appearing in the list available in a course. A setting? No, there is an error? What error? Change ownership on the directory and all good.

  3. Do some basic tests with that version of BIM.
    • Create BIM activity in old EDC3100 course. – DONE
    • Do some work as administrator.
      • Register a blog – DONE
      • Create a question – DONE
    • Create some teaching staff – fred already has some details for users. – DONE

      Need to address the absence of the auth plugin – my laptop doesn’t have the institutional auth plugin, can I work around this?

      Need to create some new users.

      • examiner – david
      • marker – vick, rick
      • students – nerf, abe
  4. Do a BIM restore from the S2, 2013 version of BIM – this will be complex given usernames? – DONE

    This worked surprisingly well. Taken a bunch of data from real life S2, 2013 and placed it into the institutional version of the course and it’s worked all good.

  5. Check the known institutional problems
    • Bulk email – fixed.
    • User search.

      Stalling for some users. Works for others – having a registered feed may be a distinction?

      Having trouble identifying the cause. Wonder if it’s purely a Postgresql problem. Try with another version of Moodle with MySQL.

      Works, but generates an error about curl:$count in lib/filelib.php – there is a call to SimplePie. – suggesting that the problem isn’t Postgres, but the proxy configuration on the other Moodle server. Confirmed. This raises an issue with the timeout situation with curl (changed). But also about where this is being called – showing student details I imagine.

    • All teaching staff are coordinators – DONE

      Maybe due to how institutional roles are mapped to Moodle archetypes – examiner/teacher/moderator – editing teacher; tutor/non-editing teacher/marker – teacher.

      Vick Far – teacher (archetype editingteacher) – gets the coordinator view. Rick Nerf – marker (archetype teacher) – gets the marker view.

Some new issues

As doing the above testing, am adding issues into GitHub associated with a milestone. What follows is a record of dealing with those.

Undefined property warnings in locallib.php – 435 – Fixed.

Ugly about messages – Fixed. Raises some potential to offer better support to folk around BIM.

Share this with the world

These fixes need to be shared more broadly.

  • Back to github
  • Up to Moodle contrib

Done.

Identifying some immediate changes to BIM

I have until the 21st of February to get BIM tested and ready for installation into the institutional Moodle instance. The following is some initial planning of what I’d like to get done in that time frame. A list that will then need to be further whittled away to what I can get done in that time frame. There are three categories of changes

  1. Changes to better support the pedagogy I’m currently using.
  2. Changes from the BIM issues list.
  3. Changes to ensure correct functioning.

Better support the pedagogy

The pedagogy/learning design that informed the initial design of BIM is fairly limiting. The learning design/pedagogy I’m currently using isn’t directly supported by BIM. I found myself last year doing a range of programming kludges to get it to work. This won’t work in the second half of this year when a non-technical academic takes over as course coordinator. BIM better fitting the current learning design saves me time and enables other people to use this approach.

  • Allow more than one post to be allocated to a question

    Already had this as an issue. Would allow “questions” to be though of as modules (in EDC3100 speak) or time periods.

  • Allow student allocation of posts to a question.

    Mentioned in this issue (amongst other extensions).

  • The student interface would also need to be changed to handle the display of multiple posts to a question.
  • Have BIM generate statistics about length of posts, number of links and number of links to other student posts. Display this to the student and generate a CSV file for the marker.
  • Also seems to suggest some sort of auto mark generation based on the statistics.
  • Have a default allocation of posts to questions/topics based on the date OR just have it set to a particular value (i.e. all posts should now be allocated – by default – to Module 1).

The basic idea behind these changes is that students are required to make a number of posts per Module in the course (one module = about 3 weeks). Students mark for the posts for each module is based solely on the number of posts and that the posts meet a certain set of statistics (length of posts, links etc). These marks are added as part of the bigger assignment that is associated with the module.

Students need to be able to see their progress. Markers need to be able to access the statistics to mark assignments.

Changes required would likely include

  1. Changes to student interface
    1. Show question allocation as a row (not a column).
    2. Replace the “Status” column with a “Statistics” column that includes the number of words, links etc statistics
    3. What options exists with parsing HTML and extracting links in PHP and in Moodle.
    4. Show unallocated posts together in a separate set of rows – perhaps even a special form – where the questions allocation drop box is available.
  2. Changes to the coordinator interface

    1. Configuration option for multiple questions

      database change required

    2. Change the default allocation of posts.
    3. Some how deal with questions that aren’t questions.
    4. How to specify statistics for auto marking.
    5. What if any changes are required for the “Manage Marking” page.

      May not need it. As this page simply shows counts. With multiple posts to a question, the count should still work.

    6. What to do about the workflow and the idea of marked/suspended/released if posts aren’t being marked, simply analysed.
  3. Changes to the marker interface
    1. “Mark posts” will still need to be used to allocate posts (or should this feature be added to “View student details”) even though “marking” may not make sense.
    2. “Mark posts” cell with a question would need to show the number of posts for that question implying no way to mark directly. Or perhaps list each post? Needs thought here
    3. “Allocate posts” page will need to retain all question names in the “Choose one” drop box.
    4. “Allocate posts” page should also have an additional heading to group multiple posts to the one question — this may enable doing without the database change for the new configuration option.
    5. May need to add to “Allocate posts” a link to “mark this post” so that “Mark posts” page can point to a list of posts and one can be chosen for marking. Mmmm.
    6. One of these pages should have a link to export a CSV file containing marks for students against posts.

From the BIM issues list

The BIM source code is hosted on github and I’ve been using the associated issue list to record any ad hoc improvements/fixes. The following are the issues that would be nice

  • No response to find student – a bug, has it been fixed yet?

    Seems to specific to Moodle 2.5, so not directly applicable to the institutional context.

  • Warn of summary feeds

    Problem from last year, blogs configured to just showing the first few lines of posts, not the whole post.

  • A couple of issues about updating posts – allow students/markers to update posts stored in BIM (mostly to fix errors or recent changes).

Correct functioning

  • Check out any and all warnings being generated by BIM now.
  • Bulk email

    Used in a number of places. This is not working. Either on mootest or my box. A missing parameter. Not sure when this cropped up.

  • User search

    Search for a student within BIM isn’t working on mootest, it does work on my box. My initial guess is some SQL type queries in BIM that are MySQL specific.

  • All teaching staff are coordinators

    The distinction between coordinator and marker isn’t kicking in as it should.

Visualising posting frequency: BIM and EDC3100

The following describes an attempt to develop a visualisation of the frequency with which students in the course I teach posted to their blogs during two offerings of the course in 2013. A few reasons for doing this

  1. Confirm my gut feel that some students (for a variety of reasons) treated blogging pragmatically and only posted just before due dates.

    Only some evidence of this.

  2. Explore if there’s any visual correlation between this behaviour and their final grade in the course and other factors.

    Some suggestion of this.

  3. Help think about whether or not a visualisation like this might be something to include in the next version of BIM.

    That might be difficult given time-frames and other constraints, but I’ll be exploring how to create some of these manually and use them during the coming semester.

Cut to the chase (you may have to be patient for these visualisations to come up in your browser – and I can’t be sure they’ll work for everyone)

More description of what and how follows.

The first plan

A table. Columns represent days of the semester. Each row is a student. Colour the cells where a post occurs. Show the due dates by running a line through the table. Group the rows via various external factors including: GPA, result in the course, sector.

Google Chart’s magic table looks like a good tool for a first run. So here goes.

Test out magic tables

Well, the example code for Data::Google::Visualization::DataTable is truly broken. But using it will generate Javascript array that will slot straight into the MagicTable visualisation.

Doing it with BIM data

So bring in some of the BIM code, read the database and generate the data structure required by the MagicTable code and you get the following as a first stab (click on it to see a larger version). Each row of this figure represents an individual student blog. Each column represents a day of the semester. Any cell that is shaded a particular colour represents a day when some posts were added to the student’s blog.

The fish-eye view in the above is overing over one student who definitely seems to have left things to the last minute – 16 posts on the one day. A day that happens to be toward the end of the semester.

The prevalence of green squares earlier in the semester represents days with 6 to 8 posts.

If you want to be more interactive, you can see the full working HTML page here. The table itself has a size of 1403×1315.

What’s missing/wrong from this view?

  1. The first few columns don’t seem to be getting set correctly – NaN?
  2. Need more signposting of the various semester dates. e.g. when are the 3 assignments due? When’s the Professional Experience period? When are the semester holidays?

    On the HTML page it’s possible to see a pattern where there are fewer posts later in the semester. As it happens, the last 5 weeks of the teaching period for this semester included 2 weeks of holidays and 3 weeks of Professional Experience (where the students are out teaching in schools).

  3. Include some indication of student results – e.g. final grade, GPA, assignment result or even the result for the learning journal (the marks given for the blog).

    This example of a magic table uses the idea of column/row headers which could be useful in doing this.

Next step

If I move directly to generating the Javascript from Perl for magic table it appears I have a bit more control. I can set a row and column header so that you can see some detail about the column (e.g. the data) and row (e.g. GPA, student name, grade etc). I have that working. Now I need to get some extra student data into the database.

It is moments like this that I detest working in an environment where there is not decent data infrastructure. Having to waste time to fix up some of the data I’m working with.

Ok, now let’s sort by grade and here is the finished product. The page itself offers some more description of what is shown.

I don’t have the time to explore what’s going on here in detail. So, some quick observations, none of which tell you very much without further exploration

  • There is an early student with “no grade” (indicating a withdrawal from the course) that has apparently posted consistently throughout the semester.
  • All of the other “no grade” students don’t post after the 15th August (when the first assignment is due).
  • There’s a group of 5 students with the lowest passing grade who blogged consistently throughout the semester but started later than other groups of students.
  • There are a few students with the top grades who are quite obviously blogging just before the assignment due dates.

However, it does reinforce an existing opinion that the blogging in this course is not yet introduced sufficiently well for the students to get real value from it.

Semester 1

The semester 1 offering of the course had three times as many students and was the first time the blogs were used in this course. The semester 1 offering also included 3 groups of on-campus students as well as online students (semester 2 was only offered online). I’m thinking that the semester 1 map should show evidence of more “pragmatic” blogging.

Let’s find out.

A rough first run at the semester 1 view.

Some quick comments

  • The magic table widget breaks down with the a table this size. Difficult to see the row/column labels as the page scrolls.
  • At least for me, you can see the shape of the semester. Assignment 1 due – break. Assignment 2 due – Professional Experience (no blogging), then assignment 3 due.
  • THere’s at least one C student who appears to be blogging just about every day and often more than one post. What’s with that?
  • There’s a group of C students who appear to be very pragmatic.
  • There seems to be a general increase in posting regularity and intensity as you get to the HD students.

    At least this is how it looks to me.

Would be interesting to explore further. Does mode in semester 1 make a difference? Amongst many more.

What’s next?

Wonder how difficult this would be to incorporate into BIM in a usable way?

At this stage, I’m thinking I’ll produce a couple of these visualisations during the coming semester for students to think about where they are with their practice.

May also prove an interesting indication of how changes in the scaffolding and support of the blogging process works out.

The visual representation is interesting, but needs easier ways to manipulate and explore the data more. e.g. some representation of the size, content, number of links, and quality (?) of the blog posts etc.

Blogs, learning analytics, IRAC and BIM

In 2014 I am hoping to make some changes to BIM that will enhance the course I’ll be teaching. The hope is to leverage various learning analytics to enhance student learning. The following is an attempt to use the IRAC framework to think about what might be done. Essentially a bit of brainstorming about possible future development.

Each of the headings below link to the IRAC framework. First off the content and the purpose of this use of learning analytics is described. Then each of the four components of the IRAC framework – Information, Representation, Affordances and Change – are considered.

I’ve just learnt about the proceedings from the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning, will need to read through that content for any additional insights.

Context

The course is a 3rd year course in a Bachelor of Education. It’s taken by folk hoping to become teachers at every level from prep, through Grade 12 and into the VET sector. The focus is the students be able to use Information and Communication Technologies to enhance/transform the learning of their students. During the course the students complete a three week practical in a school setting. The course is offered twice a year. The first offering has average around 300 students spread over three campuses and online. The second offering averages around 100 students all online. The students in the course are not necessarily all that ICT literate.

The students are required to maintain an individual blog that they use as a learning journal. The learning journal is intended to be used for capturing experiences, feelings and reflections. Contributions to the learning journal contribute 15% of the final mark. There is no formal marking of blog posts. Marking is done on the basis of the number of posts per week, the average word count, and the number of links to both external resources and blog posts from other students.

2013 was the first year the learning journal assessment was used. All 2013 student blog posts are archived in two instances of BIM. The plan is to use learning analytics to explore this data and test out various approaches that could be integrated into BIM and the course’s operation in 2014.

Purpose

At a high level, improve student learning while keeping the staff workload appropriate. Briefly, the pedagogy in the course is trying to encourage students’ self-regulation, reflection and building a PLN/making connections. I want the students to take ownership of their learning around ICTs and Pedagogy and for them to create and share a range of artefacts, insights and perhaps knowledge. The purpose that learning analytics can play in this is helping both the students achieve this and help the teaching staff support this.

Some high level aims for harnessing learning analytics.

  1. Provide students with some idea of how they are going and perhaps more importantly how to improve.
  2. Increase the diversity, quantity and quality of the connections between students and their posts and blogs.
  3. Allow teaching staff to identify who is struggling, who is doing well and who is in between and then help support staff in engaging appropriately.

A quick skim of 2013 course evaluation responses reveal some comments (emphasis added) from the semester 1 offering

The blog is a good idea to ensure the students are trying new ICTs during the course however the assessment was pointless. There was no real reason for us to be writing a certain amount of blogs per week. I found it a nuisance to maintain (on-campus student)

Probably the blogging was unnecessary but I still didn’t mind that. (on-campus student)

The blogs were very time consuming – and considering they were marked without being reviewed/marked then I am concerned that we could have done what ever we wanted! (on-campus student)

I also found blogging to be very beneficial in building my PLN (online student – most effective aspects of the course)

being forced to blog was actually great as it brought online students together as we shared resources and got to know each other. (online student – most effective aspects of the course)

the blog we had to keep, it had no purpose to it (online student – least effective aspect of the course)

the amount of blogs expected (online student – least effective aspect of the course)

The blogging, although I can see why we had to do it, I found it was hard to keep to the time frames as an online student (online student – least effective aspect of the course)

I don’t believe that I gained from blogging 3 times per week. I would rather have been assess on the quality of 1 blog per week and professional feedback that I could have provided to another student (rather than just links in the 3 blogs) (online student – least effective aspect of the course)

There were other comments on the blogs, common themes so far seem to be

  • The purpose of the blogs was non-existent to some, especially given that they weren’t marked based on quality by a human.
  • Blogs were potentially seen as more problematic in Semester 1 because of other issues with the course.

Change

Change is actually the last part of the IRAC acronym, but I’ll put it first. Mainly because it is the IRAC components that is least considered in learning analytics related projects (IMHO) and the one that I think is the most important.

In this case, I can see there needing to be three types of change considered: going outside of Moodle, using features inside of Moodle, and insidie BIM.

Outside Moodle

In short, thinking about, designing and implementing the type of changes to BIM and pedagogy outlined below is inherently a learning experience. I’m not smart enough to predict what is going to happen prior to implementation. I always gain insight when engaged in these activities that I want to leverage straight way into new approaches and new technological capabilities. i.e. I want to be able to make changes to BIM during the semester.

Not something I can do with the standard processes used for supporting a Universities institutional LMS. Hence the need to look at how I can do changes to BIM outside of Moodle and the institutional installation. In 2013, I did this via a kludge, essentially some Perl scripts and a version of Moodle/BIM running on my laptop.

Beyond the constraints of the institutional LMS processes, there’s the question of information and resources other than what is typically available to a Moodle module. Some examples include

  • Activity completion.

    Currently a small part of the 15% for the learning journal assessment in this course is based on students completing the activities for set weeks. This is in Moodle, but a module like BIM will typically not be able/expected to access this information.

    QUESTION: Can/how a module access information from other parts of a Moodle course site?

  • Student demographic and academic data.

    e.g. GPA of a student, how many times they’ve taken the course, might be used to help identify those at risk. Typically not information in Moodle.

  • Student dispositions.

    Data about students dispositions and self-regulation may be useful (see below) in providing advice. This would have to be gathered via surveys and would not be normally in Moodle.

  • Computationally heavy analytics.

    It is likely that a range of natural language processing and other potentially computationally heavy algorithms could be used to analyse student posts. Most enterprise IT folk are not going to want to run these algorithms on the same server as the institutional LMS.

All of this combined, means I’ll likely explore the use of LTI mentioned in this post from earlier in the year. i.e. use LTI to enable the version of BIM used in the course to be hosted on another server. A server only used for BIM in this course so that change can happen more rapidly.

In addition, that other server is also likely to run a range of other software for the computationally heavy analytics – rather than try and shoe-horn it into a Moodle module.

Inside Moodle

There’s a line of though – with which I agree – that learning analytics are most useful when supporting a specific learning design. The more specific, the more useful. This is in tension with the tendency of LMS tools to being generic. For example, much of what I’m talking about here moves BIM away from it’s original pedagogy of students answering questions to be marked by markers toward a more connectivist approach. Becoming more specific may limit the people who can use BIM. Not a big worry at the moment, but a consideration.

Moodle 2.0 has evolved somewhat in its ability to support change. For example, the introduction of renderers separates out the representation of BIM from the data and allows different themes to override a rendered. In theory, allowing other people to modify what is shown. However, the connection with a theme is potentially a bit limiting.

Task: Explore the concept of renderers more often.

Inside BIM

There is much that could be done to the structure of BIM to enable and support rapid development. e.g. Moodle is now supporting unit tests, BIM needs to move toward supporting this.

Information

To scaffold this look at the information that could be drawn upon, I’ll use the DAI acronym. i.e. the information to be used in learning analytics can be listed as

  • Data – raw data that is the starting point (e.g. blog posts for BIM).
  • Analysis – what method/algorithm is going to be used to analyse and transform the source information into….
  • Insight or perhaps Information – something that potentially reveals something new (e.g. how good is the reflection in this blog post)

Source

Information we currently have access to

  • All student blog posts from 2013.

    As part of the BIM database tables in the Moodle database.

  • The date and time when posts were made.
  • Student performance on assignments and in the course.

    Currently in a database in another non-Moodle assignment submission system. Pondering if this needs to move to the Moodle assignment submission system and thus the Moodle gradebook. But which raises a question..

    Question: Can/how would a module like BIM get access to Moodle gradebook data in the same course?

  • Some student demographic data.

    Currently as a CSV file manually downloaded from Peoplesoft by someone else. Includes age, postcode, sector, GPA.

  • Course and institution related dates.

    e.g. assignment due dates, semester start and end dates etc.

Information that we don’t have access to, but which might be useful

  • Comments on student blog posts.

    There’s no really standard way between different blogging engines of tracking and archiving the comments made on blog posts. So we don’t record those. Anecdotal observations suggest that many of the “connections” between students occur as comments. EduFeedr did some work around this.

  • Student perceptions of the learning journal assessment.

    Might be some mention in the 2013 course evaluation results.

    TASK: Take a look at the 2013 course evaluation results and see what mention is made.

  • Student dispositions and mindsets – e.g. this work.

Analysis

A very limited list of possible forms of analysis on the information we currently have

  • Link and social network analysis etc.

    Who is linking to who? etc.

  • Natural language processing, computational linguistics etc – which might open up possibilites such as

Combining the above with student demographic information and dispositions could also reveal interesting correlations and relationships.

I need to become more aware of what possible forms of analysis might exist. At the same time, the list of affordances (see below) may also suggest forms of analysis that are required.

Representation

Early suggestions for representation might include

  • Social network diagrams of various types.

    For students and teachers to see the structure and evolution of the social network of posts/blogs. e.g. this EduFeedr scenario

  • “My progress”

    Allow students to see a collection of stats about their blog and to see it in connection with others.

  • Student posting

The work reported in this paper on using badges gives on possibility for representation and also in terms of affordances for students to compare what they’re doing with others.

Affordances

The actual definition of affordances in the IRAC framwork – like the IRAC framework itself – is still in the early days of refinement. Here I’m going to use affordances as functionality that BIM might provide. Obviously influenced by the purpose from above.

  • Help students find interesting and relevant posts from other students.
  • Help students find interesting and relevant external links.
  • Allow students to see how “good” their blog is.
  • Show students how their blog compares to other students.

    There are reservations about this.

  • Allow all participants to get some idea of the important topics being discussed each week and over other time periods.
  • Show staff a progress bar/heat map/visualisation of some sort of student progress against expected milestones/questions.

    The EduFeedr progress visualisation below (click on it to see it bigger) is an inspiration.

  • Help staff to intervene and track interventions with all students.
  • Support staff in creating auto-marking approaches.

EduFeedr Progress

Measuring impact and improvement

If we ever get around to doing something in 2014, how will we know what’s changed? Alternatively, what might be useful to learn about the use of the student blogs in 2013?

Some possibilities

  • When did student post?

    Students were expected to have a number of posts each week, however, it was only assessed over a 3 or 4 week period.

    • How many students posted consistently each week and how many did the mad dash toward the end of the 3 or 4 week period?
    • Was there any correlation between when posts were made and the content of the posts, the students performance in the course, their GPA or anything else?
  • How did (if at all) did student posts change over the semester?
    • Is it possible to tell when holidays, professional experience, other assignments were due etc. from the student posts?
    • Did the emotions in posts change over semester?

      The course is quite heavy going. Especially in the first few weeks. I would expect some great nashing of teeth in the early weeks and perhaps in the leadup to assessment.

    • How did the connections between posts/students change over the semester?
  • Is it possible to develop indicators that might identify certain types of students/posts?
    • Indicators to identify students who are about to drop out?
    • Indicators to identify popular posts?
    • Indicators of students at all levels?

      e.g. what does a “good” student write about that an “ok” student writes about?

  • What were the most mentioned concepts during the semester?

To do

Some tasks left to do include, in no particular order

  • 2013 blog posts
    • Do some analysis of the 2013 blog posts.
    • Test out some of the planned analytics on these posts.
  • BIM
    • Explore the transition to renderers.
    • Explore unit tests.
    • Explore the “Moodle way” for assignments, marking, rubrics, outcomes etc.
    • Develop the “automated” marking feature.
    • Explore how the select “analytics” features will be identified.
  • LTI
    • Identify a good external hosting service.
    • Confirm that an LTI version of BIM will work with the course.
  • Purpose
    • Clarify exactly what pedagogical aims are going to be valuable.
    • Explore the self-regulated learning literature.
    • Look at the course evaluation responses from 2013 and see if there’s anything important to address.
    • Eventually identify a specific set of outcomes I want to work toward.
  • Information
    • Explore the various analysis methods that could be useful.
    • Explore how the analysis is best done with BIM, Moodle and PHP.
  • Representation
    • Explore how/if badges might be a possibility? USQ Moodle version and capabilities.
    • What PHP support is there for visualising social network diagrams?
  • Affordances
    • Get more into the literature around affordances, especially any work people have done on how to design affordances for learning/teaching.

#moodle, blogs, Moodle blogs and #bim

BIM is a Moodle activity module that I developed and use. BIM evolved from an earlier system called Blog Aggregation Management (BAM). BIM’s acronym is BAM Into Moodle. As the name suggests, BIM is essentially a port of all of BAM’s functionality into Moodle. Both BAM and BIM are designed to help with the task of managing students in a course writing and reflecting on their own individual web blogs. In particular, it was designed to do this for courses with hundreds of students.

The aim of this post is to explore and explain a comment that often arises when BIM is first mentioned. i.e. doesn’t Moodle already offer blog integration? The following tweet from @tim_hunt is an example of this.

The aim here is to answer the question, “What does BIM offer that Moodle’s existing blog integration doesn’t already provide?”

In short,

  • Blogs in Moodle are focused at providing a way for authors to create a blog inside of a Moodle instance.
  • BIM is focused on supporting teaching staff in managing a course where all students are expected to write on their own externally hosted blog.

Blogs in Moodle

Each user in Moodle has their own blog. i.e. the user’s (student, teacher or other) blog resides in Moodle. The functionality used to create and edit blog posts is provided by Moodle.

Each user’s blog can have an RSS feed if configured (by default this is turned off). However, standard advice appears to be to have RSS feeds secured (i.e. only people who can login to Moodle can access the feed).

There is support for “course tags” which allow particular posts to be associated with a course. Posts associated with courses in this way are still visible elsewhere.

If the Moodle administrators have enabled it, users can register their external blog with their Moodle blog. For example, if I registered this blog with a Moodle blog, then anything I post to this blog would also appear in my Moodle blog. Posts from an external blog can be deleted from a Moodle blog, but can’t be edited.

Summary

Moodle’s blog functionality is focused on helping users create and maintain a blog that sits within a Moodle instance.

It is user-focused, not course-focused. e.g. it appears to offer no functionality for teaching staff to find out which students have blogged or haven’t, and no functionality to mark blog posts.

The problem here (at least for some) is that

Reflective learning journals are demanding and time-consuming for both students and staff (Thorpe, 2004, p. 339)

Blogs with BIM

BIM doesn’t provide any functionality for students or teachers to create a blog. Instead, BIM relies on the author creating a blog on their choice of blogging platform (e.g. I always recommend WordPress.com). This means that the students’ blogs (it’s almost always student blogs that BIM works with) are hosted external to the LMS. Each student’s blog is their individual blog.

What BIM does is

  • Make a copy of all the posts students make on their blog within the LMS just in case the dog eats it.
  • Provide a couple of aggregated views that shows you who has blogged, how much they’ve blogged and how recently they’ve blogged.
  • Allows different teaching staff to see these aggregated views for the students they are responsible for (while the “in charge” teacher can see all).
  • Shows which students haven’t registered their blogs yet and provides a mail merge facility to remind them to do it.
  • Provides an interface so students can check what BIM knows about their posts.
  • If you really want to, allows you to mark student posts.

    This is done by specifying a set of questions that student posts should respond to, and the provision of a marking and moderation interface. Finally, the marks will integrate into the Moodle gradebook.

Summary

BIM functionality is focused on managing (and marking) of student blog posts. It aims to reduce the time-consuming nature of reflective journals implemented using blogs.

What functionality BIM currently provides for this task remains essentially the same as was designed into BAM in 2007. I’m hoping 2014 will see some long overdue evolution in functionality.

Moodle blogs and BIM?

The Moodle blog functionality is all about helping authors produce blogs. BIM is currently all about helping teachers manage and mark the student use of blogs. It is possible to argue that neither do an overly fantastic job.

This means that it should be possible for the two to work together. i.e. a student could register their Moodle blog with BIM, rather than using WordPress or some other external service. Indeed it is. I’ve just successfully registered a Moodle user blog in BIM.

This is of potential interest in situations where what the students are reflecting on might raise privacy concerns (e.g. nursing students – or just about any other profession – reflecting on their placement experiences). In this situation, the students could create their blog within Moodle and register the RSS feed with BIM.

However, the privacy of this approach depends on the blog visibility settings within Moodle and their impact on the generation of the RSS file. There appear to be three relevant settings for “blog visiblity” in Moodle

  • “The world can read entries set to be world-accessible”
  • “All site users can see all blog entries”
  • “Users can only see their own blog”

The question is what effect this visibility setting will have on the RSS file required by BIM. i.e. If visibility is set at “Users can only see their own blog” will this stop generation of the RSS file? A quick test seems to suggest that the RSS file is still generated.

This begs another question about privacy. The “security” or “privacy” of the RSS file generated by a Moodle blog is an example of security through obscurity. i.e. if you know the URL for the RSS file, you can view. The “security” arises because the URL includes a long list of hexadecimal numbers that make it hard to guess.

References

Thorpe, K. (2004). Reflective learning journals : From concept to practice. Reflective practice: International and Multidisciplinary Perspectives, 5(3), 327–343.

BIM for Moodle 2.5

Earlier this week @sthcrft asked

Talk about good timing. My shiny new Mac laptop arrived the same day and I’d been waiting on its arrival to explore whether or not BIM was Moodle “2.5ish happy”. It turns out that there are a few tweaks required and some improvements made possible. The following is records those tweaks.

Current status

BIM seems to be working on Moodle 2.5.

I have made a minor change so that there is now a branch of BIM specific to Moodle 2.5. Will probably become the master branch in coming days.

Tested the changes with my current course’s use of BIM – about 100 students – but have yet to add this to the Moodle plugin database.

Crashing on tabs

It was looking quite good for BIM on Moodle 2.5. Installed without a problem and appeared to be basically working. Some of the interface tweaks helped the unmodified BIM look a bit nicer.

But then I tried to “Find a student”. At which stage it appears to crash/stall/hang. Sit’s there never completing (or at least not for a very long time).

A bit of exploration of what’s happening suggests that the problem is with print_tab which appears to be deprecated from Moodle 2.5 onwards. A quick translate to the new alternative still left the same problem. The tabs work for all of the pages, but not on the submission of “Find Student”.

And back to this on the next day.

After a lot of wasted time – you idiot – I haven’t setup the http proxy on my server and that’s causing the delay. And again, you idiot.

Other tests

Tests as other users all seemed to work fine.

Layout issues

Some of the more “busy” pages for the coordinator (some overlap with the marker) don’t display very well. Never have really, but the current default theme emphasises those problems. Let’s change to another theme and see.

  • The text editor for comments on MarkPost overlaps a bit

These are minor issues and after a quick look, can’t see any quick way to solve it beyond a broader re-working of the interface.

Nested tabs

The move to tab tree apparently gives scope for nested tabs, that could solve one of the (many) uglies in BIM. i.e. the coordinators ability under “Your students” to view details and mark posts. Implementing these as nested tabs could be useful. An exploration.

That seems to work surprisingly easily. Now to remove the old kludge.

Neither strategy nor “space” to innovate is enough

Read this piece – Why CIO’s, IT and Faculty need to find common ground on technology – by David Wiley yesterday as it did the rounds. The article argues that the focus of the CIO/IT on highly reliable systems is a mismatch for the needs of innovation in learning and teaching. It brings up the tension between standard systems and rogue systems (aka shadow systems). The solution to this is a space – a policy space – that enables innovation.

In general, I am sympathetic to this argument, but I also don’t think it truly captures what is required for innovation in contemporary learning and teaching.

I then had a real laugh when I read the comments. The comments suggested that there was a misconception underpinning Wiley’s argument. In particular, that it’s not the responsibility of the CIO to maintain uptime. That’s the CTO’s job. The CIO should be innovative, they should be focused on strategy and on how to enable innovative ways to use technology.

I’m not convinced that either side has found the solution.

The idea that strategic thinking by a CIO will result in innovation in learning and teaching is, based on the literature and research I’ve seen, essentially wrong. That’s not where innovation originates. There are a variety of reasons, but one of the main ones is that being strategic inevitably leads to large scale projects and projects are based on incorrect understanding of the world. This essay – The seemingly peculiar property of projects – explains these problems and offers the solution – Tinkering.

And tinkering is where the “space” for innovation faces a challenge. I haven’t gotten to the stage of abstract principles, so instead I’ll talk about specifics. I’m currently using BIM in my teaching. To tinker with BIM while I’m using it, BIM needs to both have access to the institutional data/systems and yet be in a space where I can play. Without the institutional data I can’t use it effectively within the rest of the organisation. But without the space to play, I can’t tinker. There’s more, but taxi duty awaits.

BIM, Moodle, Simplepie, curl and HTTP proxy issue

Good news this week. BIM got into the institution’s testing site for Moodle. One step close to going live. The bad news is that there were a couple of issues to resolve. This post is a record of the attempt to address the big one (successfully as well).

The problem

When you attempt to register the blog for a student, BIM/Moodle generates this error

Unable to access the URL you provided

https://davidtjones.wordpress.com

The error created was
cURL error 28: connect() timed out!

It appears that it doesn’t play nicely with the institutional HTTP proxy. I had noticed this same problem with the development install of Moodle on my laptop, but had thought that was simply my bad practice.

Seems the problem may be a little more than that.

The plan

The rough plan is

  1. Find out if this a known problem?
  2. Does this problem effect other Moodle tools that rely on SimplePie?
  3. Is there an identifiable difference between what BIM and those other tools?

A known problem

A search for “moodle simplepie proxy” and similar doesn’t reveal a lot. (Simplepie is the 3rd party library that used to search for, parse and generally work with feeds.

You get this from GitHub which shows the Moodle modified version of GitHub. It includes evidence of modifications to SimplePie to

make sensible configuration choices, such as using the Moodle cache directory and curl functions/proxy config for making http requests in line with moodle configuration

There is also this closed issue on the Moodle tracker where there was a problem with a proxy that requires authentication. It’s been fixed and the fix should be in the versions of Moodle we’re using here. Also, I don’t believe the institutional proxy fits this problem. In fact, the error is very different.

Does it effect other Moodle tools?

There’s a “register an external blog” facility in Moodle. It connects the external blog to the users Moodle blog (I believe).

I do find it interesting that this asks the user to enter the Feed URL and not the blog URL. SimplePie does a good job of finding feeds from a blog URL (in my experience). Have just checked the code and it does use SimplePie.

Using this to register a URL without having configured the HTTP proxy results in a long period of waiting and then the error “This feed is invalid”. Seems to suggest some limitations of the code. Wish I had the time to look at this more.

Configure my box with the proxy details and try again. Oops, that didn’t work. Ahh, “Some settings were not changed due to an error” an error message when saving the HTTP proxy that didn’t exactly leap out at me. Not immediately obvious what the error was.

Checking the database (mdl_config) reveals that the proxyhost wasn’t set, apparently you don’t need the “http://” and the error doesn’t identify that. Fixed.

Okay, that works. External blog registered. And posts from the blog showing up in my Moodle “blog”.

Let’s try BIM now. Nope. The timeout problem again. Implying there’s something different going on here.

Is there a difference?

Yes, eventually tracked down one of the calls to SimplePie is using the normal SimplePie class and not the moodle simple pie class. Hence not using the proxy setup.

Tested it with the student registering process and that works as well.

A quick search for a Google reader alternative

Update: A student from last semester has shared her experiences from last semester using Feedly. The big limitation with Feedly is the absence of a search facility. But then it appears that this is a limitation of all the competitors as well (for now).

The second offering of The ICTs and Pedagogy course I teach starts next week. Last semester the course made a move to each student maintaining an individual blog on their choice of service. To encourage connections between students I generated a collection of OPML files and showed them how to use Google Reader to track what people are posting.

Of course, Google reader is now dead and an alternative recommendation is needed. The plan is to mention that there are a range of alternatives and that folk can choose their own, but then show some detail of how to use a particular feed reader. So I need to figure which one I’ll show the detail of. Due to time constraints, this will be a quick, rather than comprehensive search and test.

Here’s one list of alternatives and another more focused on education.

The planned test is

  1. Import an OPML of my subscriptions from NetNewsWire (my current reader).

    This is quite a large list so will be a reasonable test of the import process. Who knows, I might change from NetNewsWire.

  2. Use the read a bit.
  3. Check out mobile options.

BazQux

Was going to explore BazQux on a recommendation, but it appears to require payment. $9 a year is not that much, but not something I’m comfortable with requiring from the students.

Liked the ability to login with OpenID.

Imports the OPML with no worries.

Display is ok, but not brillant. And navigation works.

Feedly

Login goes straight to Google account. Wonder if this works with the students limited Google account that forms the basis of their institutional email?

OPML is imported okay. Presents an Organize page for moving the feeds and groups around. Apparently does drag and drop of feeds.

Interface is nicer. Has some missing bits, but looks alright.

Cost? Apparently, free for now and a paid version later.

There are versions for various platforms.

Given time constraints, that will probably do.

The most liked from this list of alternatives, but detested by the person who recommended BazQux.

Comparing Automatically Detected Reflective Texts with Human Judgements

The following is a summary and some thoughts on the following paper

Ullmann, T. D., Wild, F., & Scott, P. (2012). Comparing Automatically Detected Reflective Texts with Human Judgements. 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. 7th European Conference on Technology-Enhanced Learning (pp. 101–116). Saarbruecken, Germany.

My interest in this paper is as an addition to BIM to provide scaffolding for students in their reflections and also as part of the preparation for my Moodlemoot’AU 2013 talk next week.

Of course, it also allows me to engage with one of the current fads

The automated detection of reflection is part of the broader field of learning analytics, especially social learning content analysis [13]. (Ullman et al, 2012, p. 102)

where [13] is

Ferguson, R., Shum, S.B.: Social learning analytics: five approaches. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. p. 23–33. LAK ’12, ACM, New York, NY, USA (2012), http://doi.acm.org/10.1145/2330601.2330616

In working through this I’m pointed to the proceedings of a 2011 workshop on “Awareness and Reflection in Learning networks”. Which is something I’ll need to return to, not to mention the 2nd and 3rd workshops.

Would appear, that just yet, this work isn’t quite ready for BIM. But who knows what has happened in the last year or so. Possibilities exist.

A distributed cognition aside

At the moment, I’m thinking that that “Moving Beyond” part of that presentation – which will show off what I’m thinking of working on with BIM – is going to be scaffolded with the following which is from Bell & Winn (2000) and is their description of Salomon’s (1995) two broad “classes” of distributed cognition as applied to artefacts in a learning environment

  1. An individual’s cognitive burden can be off-loaded onto the artefact, which may not necessarily help the individual learn about what the artefact is doing.
  2. An artefact is designed to reciprocally scaffold students in specific cognitive practices, it helps them in a process and in doing so can help them be able to perform this task without the artefact.

Ullman’s et al (2012) work would seem to fit neatly into the second of those classes. I’m hoping it (or other work) will provide the insight necessary to scaffold students in learning how to reflect.

I’m also thinking that there’s another dimension to think about in the design of BIM (and e-learning tools in general), the identity of the individual. I’m thinking there are at least three/four different identities that should be considered, they are:

  1. Student – as above, the person who’s using the tool to learn something.
  2. Teacher – the individual there to help the student. (My thinking is rooted in a formal education environment – it’s where I work – hence there is a need for a distinction here and my context also drives the remaining identities).
  3. Institutional support and management – the folk who are meant to help and ensure that all learning is of good quality.
  4. Artefact developers – the folk that develop the artefacts that are being used by the previous three roles.

I’m thinking that a tool like BIM should be concerned with providing functionality that addresses both “distributed cognition classes” for all roles.

Abstract – Ullmann et al (2012)

  • Paper reports on an experiment to automatically detect reflective and non-reflective texts.
  • 5 elements of reflection were defined and a set of indicators developed “which automatically annotate texts regarding reflection based on the parameterisation with authoritative texts”.
  • A collection of blog posts were then run through the system and an online survey used to gather human judgements for these texts.
  • The two data sets allowed a comparison of the quality of the algorithm versus human judgements.

Introduction

Reflection is important – “at ‘the heart of key competencies’ for a successful life and a well-functioning society” (Ullman et al, 2012, p. 101).

Methods for assessing reflective writings are recent and not full established. Some quotes about how most focus has been on theorising reflection and its use, with little on how to assess reflection.

Issues with manual assessment of reflection

  1. Time-consuming.
  2. Feedback comes a long time after of reflection.
  3. Reluctance to share reflective writing given the nature of reflection.

Situating the research

This fits within learning analytics and social learning content analysis

Two prominent approaches for identifying automatically cognitive processes

  1. Connection between cue words and acts of cognition
  2. Probabilistic models and machine learning algorithms.

References to earlier work with both approaches provided.

Elements of reflection

Points out that no model of reflection is currently agreed upon. Presents 5 elements based on major stream of theoretical discussion

  1. Description of an experience.

    Sets the stage for reflection. A description of the important parts of the event. Might be a description of external events or the internal situation of the person. Common themes include: Conflict; self-awareness; and, emotions.

  2. Personal experience.

    Still some debate. But self-awareness, inner dialogue indicators.

  3. Critical analysis

    critical questions of content, process and premises of experience to correct assumptions beliefs etc.

  4. Taking perspectives into account

    Using a frame of reference based on dialogue with others, general principles, theory.

  5. Outcome of reflective writing

    New understanding/transformative and confirmatory learning. Sums up what was learned, concludes, plans for the future, new insight etc.

Acknowledges overlap between these elements.

A set of indicators were developed for each element.

Reflection detection architecture

a set of annotators – “bits of software” – developed and combined to do the analysis. Analysing the text and identify certain elements (roughly) based on keywords or other analysis.

  • NLP annotator – highlighting elements of natural language.
  • Premise (assuming that, because, deduced from) and conclusion (as a result, therefore) annotator
  • Self-reference (I me mine) and pro-noun other (he, they, others) annotators
  • Reflective vern (rethink, reason, mull over).
  • Learning outcome annotator (define, name, outline ) and Bloom’s taxonomy (remember, understand, apply, analyse)
  • Future tense (will, won’t)

An analysis component aggregates and tries to infer knowledge from the annotations. The creation of IF THEN statements used to chain lower level facts together.

This goes on until a high level rule(s) are used to connect with an element of reflection. Ended up using 16 such rules allocated to the five elements of reflection.

Method

Indicators were developed iteratively with sample texts.

Big question, what “weight should be given to each indicator to form a reflective text” (p. 108).

Used 10 texts marked as prototypical reflective writings from the reflection literature to parameterise the analytics. All this ended up with the following definition of the conditions for a reflective text

  • The indicators of the ”description of experiences” fire more than four times.
  • At least one self-related question.
  • The indicators of the ”critical analysis” element fire more than 3 times.
  • At least one indicator of the ”taking perspectives into account” fires.
  • The indicators of the ”outcome” element fire more than three times.

(p. 109)

Goes onto describe the questionnaire used as a comparison. Blog posts shown in random with questions. Human judgements reduced to 202. Data gathered by Mechanical Turk. Led to some problems, handled by filtering some.

Text corpus

Experiment was done using the “Blog Authorship Corpus” a collection of 681,288 posts, 140 millon words from 19320 bloggers from blogger.com in August 2004. Took the first 150 blog files. Short blog posts (less than 10 sentences) and foreign language posts removed.

5176 blog posts were annotated, 4m+ annotations and 170K+ inferences.

Results

Some value. Human folk more likely to agree with those identifies as reflective by the system.

One of the limiting factors is the parameterisation – the text used to do this was limited and there is no large body of quality reflective text available. Important because the parameterisation influences quality detection.

Doing more work on this.

Closes with

One possible application scenario especially useful for an educational setting is to combine the detection with a feedback component. The described reflection detection architecture with its knowledge-based analysis component can be extended to provide an explanation component, which can be used to feedback why the system thinks it is a reflective text, together with text samples as evidences.

References

Bell, P., & Winn, W. (2000). Distributed Cognitions, by Nature and by Design. In D. Jonassen & S. Land (Eds.), Theoretical Foundations of Learning Environments (pp. 123–145). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Ullmann, T. D., Wild, F., & Scott, P. (2012). Comparing Automatically Detected Reflective Texts with Human Judgements. 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. 7th European Conference on Technology-Enhanced Learning (pp. 101–116). Saarbruecken, Germany.