Category Archives: bim

BIM testing and fixes

A journal of fixes and testing of BIM. Aim here is to address some minor issues with integration with my current institution’s Moodle instance thereby providing a minimum working version for installation. As per yesterday’s planning the hope is to make further changes based on this foundation.

Result is a slightly tweaked version released via Moodle contrib. This will be the foundation for some tweaks, though I can feel time slipping away.

Latest version of BIM and PostgreSQL

The institutional Moodle instance uses Postgresql. Thanks to playing with MAV I know have a version of Moodle running with Postgresql (aka fred). The plan here is to install BIM on that instance and test it

  1. What’s the latest BIM?

    MOODLE_25_STABLE is the latest, but MOODLE_24_STABLE is what I need for this work, institutional Moodle version still at 2.4.

  2. Install it on fred.

    Get the source

    git clone https://github.com/djplaner/moodle-mod_bim/
    mv moodle*bim bim
    cd bim
    git branch MOODLE_24_STABLE
    git pull origin MOODLE_24_STABLE

    Visit notifications as the admin user on fred and install of BIM successful.

    BIM not appearing in the list available in a course. A setting? No, there is an error? What error? Change ownership on the directory and all good.

  3. Do some basic tests with that version of BIM.
    • Create BIM activity in old EDC3100 course. – DONE
    • Do some work as administrator.
      • Register a blog – DONE
      • Create a question – DONE
    • Create some teaching staff – fred already has some details for users. – DONE

      Need to address the absence of the auth plugin – my laptop doesn’t have the institutional auth plugin, can I work around this?

      Need to create some new users.

      • examiner – david
      • marker – vick, rick
      • students – nerf, abe
  4. Do a BIM restore from the S2, 2013 version of BIM – this will be complex given usernames? – DONE

    This worked surprisingly well. Taken a bunch of data from real life S2, 2013 and placed it into the institutional version of the course and it’s worked all good.

  5. Check the known institutional problems
    • Bulk email – fixed.
    • User search.

      Stalling for some users. Works for others – having a registered feed may be a distinction?

      Having trouble identifying the cause. Wonder if it’s purely a Postgresql problem. Try with another version of Moodle with MySQL.

      Works, but generates an error about curl:$count in lib/filelib.php – there is a call to SimplePie. – suggesting that the problem isn’t Postgres, but the proxy configuration on the other Moodle server. Confirmed. This raises an issue with the timeout situation with curl (changed). But also about where this is being called – showing student details I imagine.

    • All teaching staff are coordinators – DONE

      Maybe due to how institutional roles are mapped to Moodle archetypes – examiner/teacher/moderator – editing teacher; tutor/non-editing teacher/marker – teacher.

      Vick Far – teacher (archetype editingteacher) – gets the coordinator view. Rick Nerf – marker (archetype teacher) – gets the marker view.

Some new issues

As doing the above testing, am adding issues into GitHub associated with a milestone. What follows is a record of dealing with those.

Undefined property warnings in locallib.php – 435 – Fixed.

Ugly about messages – Fixed. Raises some potential to offer better support to folk around BIM.

Share this with the world

These fixes need to be shared more broadly.

  • Back to github
  • Up to Moodle contrib

Done.

Identifying some immediate changes to BIM

I have until the 21st of February to get BIM tested and ready for installation into the institutional Moodle instance. The following is some initial planning of what I’d like to get done in that time frame. A list that will then need to be further whittled away to what I can get done in that time frame. There are three categories of changes

  1. Changes to better support the pedagogy I’m currently using.
  2. Changes from the BIM issues list.
  3. Changes to ensure correct functioning.

Better support the pedagogy

The pedagogy/learning design that informed the initial design of BIM is fairly limiting. The learning design/pedagogy I’m currently using isn’t directly supported by BIM. I found myself last year doing a range of programming kludges to get it to work. This won’t work in the second half of this year when a non-technical academic takes over as course coordinator. BIM better fitting the current learning design saves me time and enables other people to use this approach.

  • Allow more than one post to be allocated to a question

    Already had this as an issue. Would allow “questions” to be though of as modules (in EDC3100 speak) or time periods.

  • Allow student allocation of posts to a question.

    Mentioned in this issue (amongst other extensions).

  • The student interface would also need to be changed to handle the display of multiple posts to a question.
  • Have BIM generate statistics about length of posts, number of links and number of links to other student posts. Display this to the student and generate a CSV file for the marker.
  • Also seems to suggest some sort of auto mark generation based on the statistics.
  • Have a default allocation of posts to questions/topics based on the date OR just have it set to a particular value (i.e. all posts should now be allocated – by default – to Module 1).

The basic idea behind these changes is that students are required to make a number of posts per Module in the course (one module = about 3 weeks). Students mark for the posts for each module is based solely on the number of posts and that the posts meet a certain set of statistics (length of posts, links etc). These marks are added as part of the bigger assignment that is associated with the module.

Students need to be able to see their progress. Markers need to be able to access the statistics to mark assignments.

Changes required would likely include

  1. Changes to student interface
    1. Show question allocation as a row (not a column).
    2. Replace the “Status” column with a “Statistics” column that includes the number of words, links etc statistics
    3. What options exists with parsing HTML and extracting links in PHP and in Moodle.
    4. Show unallocated posts together in a separate set of rows – perhaps even a special form – where the questions allocation drop box is available.
  2. Changes to the coordinator interface

    1. Configuration option for multiple questions

      database change required

    2. Change the default allocation of posts.
    3. Some how deal with questions that aren’t questions.
    4. How to specify statistics for auto marking.
    5. What if any changes are required for the “Manage Marking” page.

      May not need it. As this page simply shows counts. With multiple posts to a question, the count should still work.

    6. What to do about the workflow and the idea of marked/suspended/released if posts aren’t being marked, simply analysed.
  3. Changes to the marker interface
    1. “Mark posts” will still need to be used to allocate posts (or should this feature be added to “View student details”) even though “marking” may not make sense.
    2. “Mark posts” cell with a question would need to show the number of posts for that question implying no way to mark directly. Or perhaps list each post? Needs thought here
    3. “Allocate posts” page will need to retain all question names in the “Choose one” drop box.
    4. “Allocate posts” page should also have an additional heading to group multiple posts to the one question — this may enable doing without the database change for the new configuration option.
    5. May need to add to “Allocate posts” a link to “mark this post” so that “Mark posts” page can point to a list of posts and one can be chosen for marking. Mmmm.
    6. One of these pages should have a link to export a CSV file containing marks for students against posts.

From the BIM issues list

The BIM source code is hosted on github and I’ve been using the associated issue list to record any ad hoc improvements/fixes. The following are the issues that would be nice

  • No response to find student – a bug, has it been fixed yet?

    Seems to specific to Moodle 2.5, so not directly applicable to the institutional context.

  • Warn of summary feeds

    Problem from last year, blogs configured to just showing the first few lines of posts, not the whole post.

  • A couple of issues about updating posts – allow students/markers to update posts stored in BIM (mostly to fix errors or recent changes).

Correct functioning

  • Check out any and all warnings being generated by BIM now.
  • Bulk email

    Used in a number of places. This is not working. Either on mootest or my box. A missing parameter. Not sure when this cropped up.

  • User search

    Search for a student within BIM isn’t working on mootest, it does work on my box. My initial guess is some SQL type queries in BIM that are MySQL specific.

  • All teaching staff are coordinators

    The distinction between coordinator and marker isn’t kicking in as it should.

Visualising posting frequency: BIM and EDC3100

The following describes an attempt to develop a visualisation of the frequency with which students in the course I teach posted to their blogs during two offerings of the course in 2013. A few reasons for doing this

  1. Confirm my gut feel that some students (for a variety of reasons) treated blogging pragmatically and only posted just before due dates.

    Only some evidence of this.

  2. Explore if there’s any visual correlation between this behaviour and their final grade in the course and other factors.

    Some suggestion of this.

  3. Help think about whether or not a visualisation like this might be something to include in the next version of BIM.

    That might be difficult given time-frames and other constraints, but I’ll be exploring how to create some of these manually and use them during the coming semester.

Cut to the chase (you may have to be patient for these visualisations to come up in your browser – and I can’t be sure they’ll work for everyone)

More description of what and how follows.

The first plan

A table. Columns represent days of the semester. Each row is a student. Colour the cells where a post occurs. Show the due dates by running a line through the table. Group the rows via various external factors including: GPA, result in the course, sector.

Google Chart’s magic table looks like a good tool for a first run. So here goes.

Test out magic tables

Well, the example code for Data::Google::Visualization::DataTable is truly broken. But using it will generate Javascript array that will slot straight into the MagicTable visualisation.

Doing it with BIM data

So bring in some of the BIM code, read the database and generate the data structure required by the MagicTable code and you get the following as a first stab (click on it to see a larger version). Each row of this figure represents an individual student blog. Each column represents a day of the semester. Any cell that is shaded a particular colour represents a day when some posts were added to the student’s blog.

The fish-eye view in the above is overing over one student who definitely seems to have left things to the last minute – 16 posts on the one day. A day that happens to be toward the end of the semester.

The prevalence of green squares earlier in the semester represents days with 6 to 8 posts.

If you want to be more interactive, you can see the full working HTML page here. The table itself has a size of 1403×1315.

What’s missing/wrong from this view?

  1. The first few columns don’t seem to be getting set correctly – NaN?
  2. Need more signposting of the various semester dates. e.g. when are the 3 assignments due? When’s the Professional Experience period? When are the semester holidays?

    On the HTML page it’s possible to see a pattern where there are fewer posts later in the semester. As it happens, the last 5 weeks of the teaching period for this semester included 2 weeks of holidays and 3 weeks of Professional Experience (where the students are out teaching in schools).

  3. Include some indication of student results – e.g. final grade, GPA, assignment result or even the result for the learning journal (the marks given for the blog).

    This example of a magic table uses the idea of column/row headers which could be useful in doing this.

Next step

If I move directly to generating the Javascript from Perl for magic table it appears I have a bit more control. I can set a row and column header so that you can see some detail about the column (e.g. the data) and row (e.g. GPA, student name, grade etc). I have that working. Now I need to get some extra student data into the database.

It is moments like this that I detest working in an environment where there is not decent data infrastructure. Having to waste time to fix up some of the data I’m working with.

Ok, now let’s sort by grade and here is the finished product. The page itself offers some more description of what is shown.

I don’t have the time to explore what’s going on here in detail. So, some quick observations, none of which tell you very much without further exploration

  • There is an early student with “no grade” (indicating a withdrawal from the course) that has apparently posted consistently throughout the semester.
  • All of the other “no grade” students don’t post after the 15th August (when the first assignment is due).
  • There’s a group of 5 students with the lowest passing grade who blogged consistently throughout the semester but started later than other groups of students.
  • There are a few students with the top grades who are quite obviously blogging just before the assignment due dates.

However, it does reinforce an existing opinion that the blogging in this course is not yet introduced sufficiently well for the students to get real value from it.

Semester 1

The semester 1 offering of the course had three times as many students and was the first time the blogs were used in this course. The semester 1 offering also included 3 groups of on-campus students as well as online students (semester 2 was only offered online). I’m thinking that the semester 1 map should show evidence of more “pragmatic” blogging.

Let’s find out.

A rough first run at the semester 1 view.

Some quick comments

  • The magic table widget breaks down with the a table this size. Difficult to see the row/column labels as the page scrolls.
  • At least for me, you can see the shape of the semester. Assignment 1 due – break. Assignment 2 due – Professional Experience (no blogging), then assignment 3 due.
  • THere’s at least one C student who appears to be blogging just about every day and often more than one post. What’s with that?
  • There’s a group of C students who appear to be very pragmatic.
  • There seems to be a general increase in posting regularity and intensity as you get to the HD students.

    At least this is how it looks to me.

Would be interesting to explore further. Does mode in semester 1 make a difference? Amongst many more.

What’s next?

Wonder how difficult this would be to incorporate into BIM in a usable way?

At this stage, I’m thinking I’ll produce a couple of these visualisations during the coming semester for students to think about where they are with their practice.

May also prove an interesting indication of how changes in the scaffolding and support of the blogging process works out.

The visual representation is interesting, but needs easier ways to manipulate and explore the data more. e.g. some representation of the size, content, number of links, and quality (?) of the blog posts etc.

Reflective Blogging as part of ICT Professional Development to Support Pedagogical Change

I am planning to do some more work on BIM in preparation for using it in teaching this year, including finishing some analysis of how the blogging went in last year’s two offerings.

As luck would have it, I skimmed one of my feeds and came across Prestridge (2014). What follows is a summary and some thoughts. It’s nice to be reading an open access journal paper after a few very closed off articles.

Aside: I am wondering whether or not in the new world order being someone that reads feeds and has students blog is become somewhat old fashioned.

Abstract

The abstract for Prestridge (2014) is

Reflection is considered an inherent part of teacher practice. However, when used within professional development activity, it is fraught with issues associated with teacher confidence and skill in reflective action. Coupled with anxiety generally associated with technological competency and understanding the nature of blogging, constructive reflection is difficult for teachers. This paper focuses on the reflective quality of school teachers’ blogs. It describes teachers’ perceptions and engagement in reflective activity as part of an ICT professional development program. Reflective entries are drawn from a series of blogs that are analysed qualitatively using Hatton and Smith’s (1995) three levels of reflection-on-action. The findings suggest that each level of reflective action plays a different role in enabling teachers to transform their ICT pedagogical beliefs and practices. Each role is defined and illustrated suggesting the value of such activity within ICT professional development, consequently reshaping what constitutes effective professional development in ICT.

This appears to be relevant to what I do as the course I teach is titled “ICTs and Pedagogy” and reflection through blogging is a key foundation to the pedagogy in the course. Of course, this appears to be focused on in-service, rather than pre-service teachers.

Introduction

In the Australian education context various government policies illustrate that ICTs are important. There’s a move to 1-to-1 student/computer ratios. However, “success with regard to technology integration has been based on how extensive or prominent the use of it has been in schools rather than on whether the teacher has been able to utilize it for ‘new’, ‘better’, or more ‘relevant’ learning outcomes (Moyle, 2010)” (Prestridge, 2014). Suggests a need to “reconceptualise both the intentions and approaches to professional development” if there’s going to be an ROI on this government investment and if we’re to help teachers deal with this.

PD is “an instrument to support change in teacher practice”. Long held view that PD should move from “up-skilling in the latest software” to a deeper approach that focuses on pedagogy and context rather than technology; building teachers’ confidence in change; development of teachers’ metacognitive skills; and, as a philosophical/revisioning of ICT in learning (references attached to each of these). References work by Fisher et al (2006) as requesting “a cultural change in the teaching profession”, the principles of which need to be “activiated within ICT professional development if we are going to move from retooling teachers to enabling them to transform their practices”.

Note: I wonder how well this academic call for a cultural change matches the perceptions of teachers and the organisations that employ them? I have a feeling that some/many teachers are likely to be more pragmatic. As for the organisations and the folk that run them….

And now onto the importance/relevance of reflection to this. Schon gets a mention. As does the action research spiral, teacher-as-researcher, inquiry based professional development, reflective action, Dewey. Leading to research suggesting “that reflection brings automatic calls in the improvement of teaching” and other work suggesting there’s a lack of substantive evidence.

This paper aims to investigate “the role of written reflection as a central process in a ‘hearts and mind’ approach to ICT professional development.

Note: The mix of plural and singular in “hearts and mind” is interesting/a typo in this era of standardised outcomes/curriculum and increasing corporatisation.

Methods to framing the research

Background on a broader ARC funded project that aims to develop “a transformative model of teacher ICT professional development”. With “one or two teachers” volunteering from each school it would appear to suffer the problem of pioneers versus the rest. Teachers engaged in classroom inquiries, in particular the “implementation of an ICT application in regard their pedagogical practices and student learning outcomes”. Supported through a local school leader, outside expert, online discussion forum and personal blogging.

Has a table that lists the inquiry questions of the 8 teachers. Questions range from “How can students be supported when creating an electronic picture book using the theme ‘Program Achieve”?” to “What strategies need to be employed to promote effective/productive ICT practices that encourage intellectual demand and recognise difference and diversity?”

Teachers were encouraged to blog after teaching. Provided with a framework for reflecting after teaching (5R framework). Weekly blog mandatory. School leaders asked to encourage blogging.

This work focuses on

  1. teachers’ perceived value of the reflective activity

    Data from teachers’ final interviews and reports analysed using constant comparative method

  2. the role of written reflection in enabling change in pedagogy

    Blog posts analysed with Hatton and Smith’s (1995) three types of writing: descriptive reflection; dialogic reflection; and, critical reflection.

Results

  • 1 teacher had consistent reflection of the year implementation
  • 4 teachers had spasmodic entries, mostly at the beginning
  • the other teachers writing could be seen as simply record keeping

Finds similar results in other use of reflective blogs and suggests that “teachers’ lack of understanding on how to reflect limits their reflective writing abilities”.

Note: Not a great result perhaps, but not entirely unexpected. Might get some idea of this from my students posts in 2013 later today.

Perceived value of reflective writing

Not surprisingly, the “consistent reflection” person liked blogging. Others didn’t.

A major theme on the value was “a lack of understanding on how to reflect”.

Note: I have a feeling this may be one factor for my students. Though I wonder how much pragmatism and especially how reflective blogging falls outside the realm of standard practice for many plays a role.


“What to write in the reflective blog and then what to do with these reflections were issues raised by the teachers”

Note: Raising the issue of BIM being better at providing “prompts” to students.

Ahh, a quote from a participant brings back the “realm of standard practice” issue

I think because this inquiry thing was such a different way of doing things I’ve ever done before, it took me a while to get fair dinkum about it. I still couldn’t get the blog…..that’s one positive that’s come out of it because if I were asked to do something like this again then I would do it much more readily.

Picks up on the idea of “reflection as description” through a number of quotes. An apparent lack of priority given to analysing on what had happened and going beyond description.

This is even though the teachers were given the 5R framework and a range of questions/prompts in project documentation and comments by the outside PD expert.

Note: Given this difficulty in understanding how to write reflectively, what impact does it have on the next part of the paper “examining the role of written reflections to identify how reflection supports teacher change in pedagogy”?

The role of reflective writing

The obvious “solution” is to focus on the 1 teacher who consistently blogged, generating the problem of a sample of 1.

The posts were analysed in chronological order. Emphasis on linking the type of reflection and the role it plays in “improving and or supporting teachers in transforming the beliefs and practices”

The most common type of reflection is descriptive, which really isn’t reflection. But this descriptive reflection “provides a leverage for dialogic reflection” which may or may not be pursued. As it turns out, generally not chosen. Only when a critical friend provides some additional prompting does it appear.

When it did occur, it helped shape the teacher’s pedagogical beliefs and practices. Descriptive reflection made conscious the connection between pedagogical beliefs and actual practice, but more as a justification.

Only spasmodic evidence of critical reflection.

Suggests that data supports the conclusion that there’s a developmental sequence to reflection. Start with descriptive and then the more demanding forms emerge.

The role played by each type of reflection in transforming pedagogical beliefs and practices

  1. Descriptive reflection – a connector, making conscious the links between pedagogical beliefs, current teaching practices, and student learning outcomes.
  2. Dialogical reflection – the shaper, where the connections were examined and explored, enabling transformation.
  3. Critical reflection – a positioner. Placing the role of teacher in the broader context and critical evaluate the role.

Conclusion

If how to reflect in written form is understood, then “reflective action plays a significant part in enabling them to change their pedagogical beliefs and practices”. Each type of reflection plays a different role.

A lack of guidance and support were found to affect reflective action.

References

Prestridge, S. J. (2014). Reflective Blogging as part of ICT Professional Development to Support Pedagogical Change. The Australian Journal of Teacher Education, 39(2).

Blogs, learning analytics, IRAC and BIM

In 2014 I am hoping to make some changes to BIM that will enhance the course I’ll be teaching. The hope is to leverage various learning analytics to enhance student learning. The following is an attempt to use the IRAC framework to think about what might be done. Essentially a bit of brainstorming about possible future development.

Each of the headings below link to the IRAC framework. First off the content and the purpose of this use of learning analytics is described. Then each of the four components of the IRAC framework – Information, Representation, Affordances and Change – are considered.

I’ve just learnt about the proceedings from the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning, will need to read through that content for any additional insights.

Context

The course is a 3rd year course in a Bachelor of Education. It’s taken by folk hoping to become teachers at every level from prep, through Grade 12 and into the VET sector. The focus is the students be able to use Information and Communication Technologies to enhance/transform the learning of their students. During the course the students complete a three week practical in a school setting. The course is offered twice a year. The first offering has average around 300 students spread over three campuses and online. The second offering averages around 100 students all online. The students in the course are not necessarily all that ICT literate.

The students are required to maintain an individual blog that they use as a learning journal. The learning journal is intended to be used for capturing experiences, feelings and reflections. Contributions to the learning journal contribute 15% of the final mark. There is no formal marking of blog posts. Marking is done on the basis of the number of posts per week, the average word count, and the number of links to both external resources and blog posts from other students.

2013 was the first year the learning journal assessment was used. All 2013 student blog posts are archived in two instances of BIM. The plan is to use learning analytics to explore this data and test out various approaches that could be integrated into BIM and the course’s operation in 2014.

Purpose

At a high level, improve student learning while keeping the staff workload appropriate. Briefly, the pedagogy in the course is trying to encourage students’ self-regulation, reflection and building a PLN/making connections. I want the students to take ownership of their learning around ICTs and Pedagogy and for them to create and share a range of artefacts, insights and perhaps knowledge. The purpose that learning analytics can play in this is helping both the students achieve this and help the teaching staff support this.

Some high level aims for harnessing learning analytics.

  1. Provide students with some idea of how they are going and perhaps more importantly how to improve.
  2. Increase the diversity, quantity and quality of the connections between students and their posts and blogs.
  3. Allow teaching staff to identify who is struggling, who is doing well and who is in between and then help support staff in engaging appropriately.

A quick skim of 2013 course evaluation responses reveal some comments (emphasis added) from the semester 1 offering

The blog is a good idea to ensure the students are trying new ICTs during the course however the assessment was pointless. There was no real reason for us to be writing a certain amount of blogs per week. I found it a nuisance to maintain (on-campus student)

Probably the blogging was unnecessary but I still didn’t mind that. (on-campus student)

The blogs were very time consuming – and considering they were marked without being reviewed/marked then I am concerned that we could have done what ever we wanted! (on-campus student)

I also found blogging to be very beneficial in building my PLN (online student – most effective aspects of the course)

being forced to blog was actually great as it brought online students together as we shared resources and got to know each other. (online student – most effective aspects of the course)

the blog we had to keep, it had no purpose to it (online student – least effective aspect of the course)

the amount of blogs expected (online student – least effective aspect of the course)

The blogging, although I can see why we had to do it, I found it was hard to keep to the time frames as an online student (online student – least effective aspect of the course)

I don’t believe that I gained from blogging 3 times per week. I would rather have been assess on the quality of 1 blog per week and professional feedback that I could have provided to another student (rather than just links in the 3 blogs) (online student – least effective aspect of the course)

There were other comments on the blogs, common themes so far seem to be

  • The purpose of the blogs was non-existent to some, especially given that they weren’t marked based on quality by a human.
  • Blogs were potentially seen as more problematic in Semester 1 because of other issues with the course.

Change

Change is actually the last part of the IRAC acronym, but I’ll put it first. Mainly because it is the IRAC components that is least considered in learning analytics related projects (IMHO) and the one that I think is the most important.

In this case, I can see there needing to be three types of change considered: going outside of Moodle, using features inside of Moodle, and insidie BIM.

Outside Moodle

In short, thinking about, designing and implementing the type of changes to BIM and pedagogy outlined below is inherently a learning experience. I’m not smart enough to predict what is going to happen prior to implementation. I always gain insight when engaged in these activities that I want to leverage straight way into new approaches and new technological capabilities. i.e. I want to be able to make changes to BIM during the semester.

Not something I can do with the standard processes used for supporting a Universities institutional LMS. Hence the need to look at how I can do changes to BIM outside of Moodle and the institutional installation. In 2013, I did this via a kludge, essentially some Perl scripts and a version of Moodle/BIM running on my laptop.

Beyond the constraints of the institutional LMS processes, there’s the question of information and resources other than what is typically available to a Moodle module. Some examples include

  • Activity completion.

    Currently a small part of the 15% for the learning journal assessment in this course is based on students completing the activities for set weeks. This is in Moodle, but a module like BIM will typically not be able/expected to access this information.

    QUESTION: Can/how a module access information from other parts of a Moodle course site?

  • Student demographic and academic data.

    e.g. GPA of a student, how many times they’ve taken the course, might be used to help identify those at risk. Typically not information in Moodle.

  • Student dispositions.

    Data about students dispositions and self-regulation may be useful (see below) in providing advice. This would have to be gathered via surveys and would not be normally in Moodle.

  • Computationally heavy analytics.

    It is likely that a range of natural language processing and other potentially computationally heavy algorithms could be used to analyse student posts. Most enterprise IT folk are not going to want to run these algorithms on the same server as the institutional LMS.

All of this combined, means I’ll likely explore the use of LTI mentioned in this post from earlier in the year. i.e. use LTI to enable the version of BIM used in the course to be hosted on another server. A server only used for BIM in this course so that change can happen more rapidly.

In addition, that other server is also likely to run a range of other software for the computationally heavy analytics – rather than try and shoe-horn it into a Moodle module.

Inside Moodle

There’s a line of though – with which I agree – that learning analytics are most useful when supporting a specific learning design. The more specific, the more useful. This is in tension with the tendency of LMS tools to being generic. For example, much of what I’m talking about here moves BIM away from it’s original pedagogy of students answering questions to be marked by markers toward a more connectivist approach. Becoming more specific may limit the people who can use BIM. Not a big worry at the moment, but a consideration.

Moodle 2.0 has evolved somewhat in its ability to support change. For example, the introduction of renderers separates out the representation of BIM from the data and allows different themes to override a rendered. In theory, allowing other people to modify what is shown. However, the connection with a theme is potentially a bit limiting.

Task: Explore the concept of renderers more often.

Inside BIM

There is much that could be done to the structure of BIM to enable and support rapid development. e.g. Moodle is now supporting unit tests, BIM needs to move toward supporting this.

Information

To scaffold this look at the information that could be drawn upon, I’ll use the DAI acronym. i.e. the information to be used in learning analytics can be listed as

  • Data – raw data that is the starting point (e.g. blog posts for BIM).
  • Analysis – what method/algorithm is going to be used to analyse and transform the source information into….
  • Insight or perhaps Information – something that potentially reveals something new (e.g. how good is the reflection in this blog post)

Source

Information we currently have access to

  • All student blog posts from 2013.

    As part of the BIM database tables in the Moodle database.

  • The date and time when posts were made.
  • Student performance on assignments and in the course.

    Currently in a database in another non-Moodle assignment submission system. Pondering if this needs to move to the Moodle assignment submission system and thus the Moodle gradebook. But which raises a question..

    Question: Can/how would a module like BIM get access to Moodle gradebook data in the same course?

  • Some student demographic data.

    Currently as a CSV file manually downloaded from Peoplesoft by someone else. Includes age, postcode, sector, GPA.

  • Course and institution related dates.

    e.g. assignment due dates, semester start and end dates etc.

Information that we don’t have access to, but which might be useful

  • Comments on student blog posts.

    There’s no really standard way between different blogging engines of tracking and archiving the comments made on blog posts. So we don’t record those. Anecdotal observations suggest that many of the “connections” between students occur as comments. EduFeedr did some work around this.

  • Student perceptions of the learning journal assessment.

    Might be some mention in the 2013 course evaluation results.

    TASK: Take a look at the 2013 course evaluation results and see what mention is made.

  • Student dispositions and mindsets – e.g. this work.

Analysis

A very limited list of possible forms of analysis on the information we currently have

  • Link and social network analysis etc.

    Who is linking to who? etc.

  • Natural language processing, computational linguistics etc – which might open up possibilites such as

Combining the above with student demographic information and dispositions could also reveal interesting correlations and relationships.

I need to become more aware of what possible forms of analysis might exist. At the same time, the list of affordances (see below) may also suggest forms of analysis that are required.

Representation

Early suggestions for representation might include

  • Social network diagrams of various types.

    For students and teachers to see the structure and evolution of the social network of posts/blogs. e.g. this EduFeedr scenario

  • “My progress”

    Allow students to see a collection of stats about their blog and to see it in connection with others.

  • Student posting

The work reported in this paper on using badges gives on possibility for representation and also in terms of affordances for students to compare what they’re doing with others.

Affordances

The actual definition of affordances in the IRAC framwork – like the IRAC framework itself – is still in the early days of refinement. Here I’m going to use affordances as functionality that BIM might provide. Obviously influenced by the purpose from above.

  • Help students find interesting and relevant posts from other students.
  • Help students find interesting and relevant external links.
  • Allow students to see how “good” their blog is.
  • Show students how their blog compares to other students.

    There are reservations about this.

  • Allow all participants to get some idea of the important topics being discussed each week and over other time periods.
  • Show staff a progress bar/heat map/visualisation of some sort of student progress against expected milestones/questions.

    The EduFeedr progress visualisation below (click on it to see it bigger) is an inspiration.

  • Help staff to intervene and track interventions with all students.
  • Support staff in creating auto-marking approaches.

EduFeedr Progress

Measuring impact and improvement

If we ever get around to doing something in 2014, how will we know what’s changed? Alternatively, what might be useful to learn about the use of the student blogs in 2013?

Some possibilities

  • When did student post?

    Students were expected to have a number of posts each week, however, it was only assessed over a 3 or 4 week period.

    • How many students posted consistently each week and how many did the mad dash toward the end of the 3 or 4 week period?
    • Was there any correlation between when posts were made and the content of the posts, the students performance in the course, their GPA or anything else?
  • How did (if at all) did student posts change over the semester?
    • Is it possible to tell when holidays, professional experience, other assignments were due etc. from the student posts?
    • Did the emotions in posts change over semester?

      The course is quite heavy going. Especially in the first few weeks. I would expect some great nashing of teeth in the early weeks and perhaps in the leadup to assessment.

    • How did the connections between posts/students change over the semester?
  • Is it possible to develop indicators that might identify certain types of students/posts?
    • Indicators to identify students who are about to drop out?
    • Indicators to identify popular posts?
    • Indicators of students at all levels?

      e.g. what does a “good” student write about that an “ok” student writes about?

  • What were the most mentioned concepts during the semester?

To do

Some tasks left to do include, in no particular order

  • 2013 blog posts
    • Do some analysis of the 2013 blog posts.
    • Test out some of the planned analytics on these posts.
  • BIM
    • Explore the transition to renderers.
    • Explore unit tests.
    • Explore the “Moodle way” for assignments, marking, rubrics, outcomes etc.
    • Develop the “automated” marking feature.
    • Explore how the select “analytics” features will be identified.
  • LTI
    • Identify a good external hosting service.
    • Confirm that an LTI version of BIM will work with the course.
  • Purpose
    • Clarify exactly what pedagogical aims are going to be valuable.
    • Explore the self-regulated learning literature.
    • Look at the course evaluation responses from 2013 and see if there’s anything important to address.
    • Eventually identify a specific set of outcomes I want to work toward.
  • Information
    • Explore the various analysis methods that could be useful.
    • Explore how the analysis is best done with BIM, Moodle and PHP.
  • Representation
    • Explore how/if badges might be a possibility? USQ Moodle version and capabilities.
    • What PHP support is there for visualising social network diagrams?
  • Affordances
    • Get more into the literature around affordances, especially any work people have done on how to design affordances for learning/teaching.

#moodle, blogs, Moodle blogs and #bim

BIM is a Moodle activity module that I developed and use. BIM evolved from an earlier system called Blog Aggregation Management (BAM). BIM’s acronym is BAM Into Moodle. As the name suggests, BIM is essentially a port of all of BAM’s functionality into Moodle. Both BAM and BIM are designed to help with the task of managing students in a course writing and reflecting on their own individual web blogs. In particular, it was designed to do this for courses with hundreds of students.

The aim of this post is to explore and explain a comment that often arises when BIM is first mentioned. i.e. doesn’t Moodle already offer blog integration? The following tweet from @tim_hunt is an example of this.

The aim here is to answer the question, “What does BIM offer that Moodle’s existing blog integration doesn’t already provide?”

In short,

  • Blogs in Moodle are focused at providing a way for authors to create a blog inside of a Moodle instance.
  • BIM is focused on supporting teaching staff in managing a course where all students are expected to write on their own externally hosted blog.

Blogs in Moodle

Each user in Moodle has their own blog. i.e. the user’s (student, teacher or other) blog resides in Moodle. The functionality used to create and edit blog posts is provided by Moodle.

Each user’s blog can have an RSS feed if configured (by default this is turned off). However, standard advice appears to be to have RSS feeds secured (i.e. only people who can login to Moodle can access the feed).

There is support for “course tags” which allow particular posts to be associated with a course. Posts associated with courses in this way are still visible elsewhere.

If the Moodle administrators have enabled it, users can register their external blog with their Moodle blog. For example, if I registered this blog with a Moodle blog, then anything I post to this blog would also appear in my Moodle blog. Posts from an external blog can be deleted from a Moodle blog, but can’t be edited.

Summary

Moodle’s blog functionality is focused on helping users create and maintain a blog that sits within a Moodle instance.

It is user-focused, not course-focused. e.g. it appears to offer no functionality for teaching staff to find out which students have blogged or haven’t, and no functionality to mark blog posts.

The problem here (at least for some) is that

Reflective learning journals are demanding and time-consuming for both students and staff (Thorpe, 2004, p. 339)

Blogs with BIM

BIM doesn’t provide any functionality for students or teachers to create a blog. Instead, BIM relies on the author creating a blog on their choice of blogging platform (e.g. I always recommend WordPress.com). This means that the students’ blogs (it’s almost always student blogs that BIM works with) are hosted external to the LMS. Each student’s blog is their individual blog.

What BIM does is

  • Make a copy of all the posts students make on their blog within the LMS just in case the dog eats it.
  • Provide a couple of aggregated views that shows you who has blogged, how much they’ve blogged and how recently they’ve blogged.
  • Allows different teaching staff to see these aggregated views for the students they are responsible for (while the “in charge” teacher can see all).
  • Shows which students haven’t registered their blogs yet and provides a mail merge facility to remind them to do it.
  • Provides an interface so students can check what BIM knows about their posts.
  • If you really want to, allows you to mark student posts.

    This is done by specifying a set of questions that student posts should respond to, and the provision of a marking and moderation interface. Finally, the marks will integrate into the Moodle gradebook.

Summary

BIM functionality is focused on managing (and marking) of student blog posts. It aims to reduce the time-consuming nature of reflective journals implemented using blogs.

What functionality BIM currently provides for this task remains essentially the same as was designed into BAM in 2007. I’m hoping 2014 will see some long overdue evolution in functionality.

Moodle blogs and BIM?

The Moodle blog functionality is all about helping authors produce blogs. BIM is currently all about helping teachers manage and mark the student use of blogs. It is possible to argue that neither do an overly fantastic job.

This means that it should be possible for the two to work together. i.e. a student could register their Moodle blog with BIM, rather than using WordPress or some other external service. Indeed it is. I’ve just successfully registered a Moodle user blog in BIM.

This is of potential interest in situations where what the students are reflecting on might raise privacy concerns (e.g. nursing students – or just about any other profession – reflecting on their placement experiences). In this situation, the students could create their blog within Moodle and register the RSS feed with BIM.

However, the privacy of this approach depends on the blog visibility settings within Moodle and their impact on the generation of the RSS file. There appear to be three relevant settings for “blog visiblity” in Moodle

  • “The world can read entries set to be world-accessible”
  • “All site users can see all blog entries”
  • “Users can only see their own blog”

The question is what effect this visibility setting will have on the RSS file required by BIM. i.e. If visibility is set at “Users can only see their own blog” will this stop generation of the RSS file? A quick test seems to suggest that the RSS file is still generated.

This begs another question about privacy. The “security” or “privacy” of the RSS file generated by a Moodle blog is an example of security through obscurity. i.e. if you know the URL for the RSS file, you can view. The “security” arises because the URL includes a long list of hexadecimal numbers that make it hard to guess.

References

Thorpe, K. (2004). Reflective learning journals : From concept to practice. Reflective practice: International and Multidisciplinary Perspectives, 5(3), 327–343.

BIM for Moodle 2.5

Earlier this week @sthcrft asked

Talk about good timing. My shiny new Mac laptop arrived the same day and I’d been waiting on its arrival to explore whether or not BIM was Moodle “2.5ish happy”. It turns out that there are a few tweaks required and some improvements made possible. The following is records those tweaks.

Current status

BIM seems to be working on Moodle 2.5.

I have made a minor change so that there is now a branch of BIM specific to Moodle 2.5. Will probably become the master branch in coming days.

Tested the changes with my current course’s use of BIM – about 100 students – but have yet to add this to the Moodle plugin database.

Crashing on tabs

It was looking quite good for BIM on Moodle 2.5. Installed without a problem and appeared to be basically working. Some of the interface tweaks helped the unmodified BIM look a bit nicer.

But then I tried to “Find a student”. At which stage it appears to crash/stall/hang. Sit’s there never completing (or at least not for a very long time).

A bit of exploration of what’s happening suggests that the problem is with print_tab which appears to be deprecated from Moodle 2.5 onwards. A quick translate to the new alternative still left the same problem. The tabs work for all of the pages, but not on the submission of “Find Student”.

And back to this on the next day.

After a lot of wasted time – you idiot – I haven’t setup the http proxy on my server and that’s causing the delay. And again, you idiot.

Other tests

Tests as other users all seemed to work fine.

Layout issues

Some of the more “busy” pages for the coordinator (some overlap with the marker) don’t display very well. Never have really, but the current default theme emphasises those problems. Let’s change to another theme and see.

  • The text editor for comments on MarkPost overlaps a bit

These are minor issues and after a quick look, can’t see any quick way to solve it beyond a broader re-working of the interface.

Nested tabs

The move to tab tree apparently gives scope for nested tabs, that could solve one of the (many) uglies in BIM. i.e. the coordinators ability under “Your students” to view details and mark posts. Implementing these as nested tabs could be useful. An exploration.

That seems to work surprisingly easily. Now to remove the old kludge.

BIM and broken moodle capabilities

The following is a long overdue attempt to identify and solve an issue with BIM.

The problem

BIM provides a three different interfaces depending on the type of user, these are

  1. coordinator;

    The name is a hangover from a past institution, but essentially this is the teacher in charge. Can do anything and everything, including the management of the marking being done by other staff.

  2. marker;

    Another staff role, mostly focused on marking/looking after a specific group of students.

  3. student.

    What each student sees.

The problem is that the code that distinguishes between the different types of users is not working.

For example, a user who should be a coordinator, BIM thinks is potentially all three.

The method

The method I use (and which was used in BIM 1 and has worked fine) is based on capabilities, essentially a few ifs

if (  has_capability( 'mod/bim:marker', $context )) {
   # do marker stuff
} 
if ( has_capability( 'mod/bim:student', $context )) {
   # do student stuff
}
if ( has_capability( 'mod/bim:coordinator', $context)) {
   # do coordinator stuff
}

These are then defined in db/access.php via the publicised means

What’s happening

To get to the bottom of this, I’m going to create/configure three users who fit each of the BIM user types and see how BIM labels them.

  1. coordinator user – BIM thinks can be marker, student or coordinator.
  2. marker user – is a marker
  3. student user – is a student and a coordinator

The above was tested within BIM itself. There’s a capability overview report in Moodle that shows “what permissions that capability has in the definition of every role”.

For coordinator, it’s showing “Allow” for “Student” and not set for everything else. Not even the manager. Suggesting that there is a mismatch between the BIM code and what Moodle knows. Suggesting that an upgrade of the BIM module is called for.

So, let’s update the version number, visit the admin page and do an upgrade. Success. Now check the capability overview report.

The capability overview report is reporting no change. This appears to be where the bug is. What’s in the db/access.php file is not being used to update the database.

Seem to have it working.

Clean test

Need to do a test on a clean Moodle instance.

  1. Coordinator – CHECK
  2. Teacher – CHECK
  3. Student – CHECK

Glad that’s out of the way. More work on BIM in the coming weeks.

Neither strategy nor “space” to innovate is enough

Read this piece – Why CIO’s, IT and Faculty need to find common ground on technology – by David Wiley yesterday as it did the rounds. The article argues that the focus of the CIO/IT on highly reliable systems is a mismatch for the needs of innovation in learning and teaching. It brings up the tension between standard systems and rogue systems (aka shadow systems). The solution to this is a space – a policy space – that enables innovation.

In general, I am sympathetic to this argument, but I also don’t think it truly captures what is required for innovation in contemporary learning and teaching.

I then had a real laugh when I read the comments. The comments suggested that there was a misconception underpinning Wiley’s argument. In particular, that it’s not the responsibility of the CIO to maintain uptime. That’s the CTO’s job. The CIO should be innovative, they should be focused on strategy and on how to enable innovative ways to use technology.

I’m not convinced that either side has found the solution.

The idea that strategic thinking by a CIO will result in innovation in learning and teaching is, based on the literature and research I’ve seen, essentially wrong. That’s not where innovation originates. There are a variety of reasons, but one of the main ones is that being strategic inevitably leads to large scale projects and projects are based on incorrect understanding of the world. This essay – The seemingly peculiar property of projects – explains these problems and offers the solution – Tinkering.

And tinkering is where the “space” for innovation faces a challenge. I haven’t gotten to the stage of abstract principles, so instead I’ll talk about specifics. I’m currently using BIM in my teaching. To tinker with BIM while I’m using it, BIM needs to both have access to the institutional data/systems and yet be in a space where I can play. Without the institutional data I can’t use it effectively within the rest of the organisation. But without the space to play, I can’t tinker. There’s more, but taxi duty awaits.

BIM, Moodle, Simplepie, curl and HTTP proxy issue

Good news this week. BIM got into the institution’s testing site for Moodle. One step close to going live. The bad news is that there were a couple of issues to resolve. This post is a record of the attempt to address the big one (successfully as well).

The problem

When you attempt to register the blog for a student, BIM/Moodle generates this error

Unable to access the URL you provided

http://davidtjones.wordpress.com

The error created was
cURL error 28: connect() timed out!

It appears that it doesn’t play nicely with the institutional HTTP proxy. I had noticed this same problem with the development install of Moodle on my laptop, but had thought that was simply my bad practice.

Seems the problem may be a little more than that.

The plan

The rough plan is

  1. Find out if this a known problem?
  2. Does this problem effect other Moodle tools that rely on SimplePie?
  3. Is there an identifiable difference between what BIM and those other tools?

A known problem

A search for “moodle simplepie proxy” and similar doesn’t reveal a lot. (Simplepie is the 3rd party library that used to search for, parse and generally work with feeds.

You get this from GitHub which shows the Moodle modified version of GitHub. It includes evidence of modifications to SimplePie to

make sensible configuration choices, such as using the Moodle cache directory and curl functions/proxy config for making http requests in line with moodle configuration

There is also this closed issue on the Moodle tracker where there was a problem with a proxy that requires authentication. It’s been fixed and the fix should be in the versions of Moodle we’re using here. Also, I don’t believe the institutional proxy fits this problem. In fact, the error is very different.

Does it effect other Moodle tools?

There’s a “register an external blog” facility in Moodle. It connects the external blog to the users Moodle blog (I believe).

I do find it interesting that this asks the user to enter the Feed URL and not the blog URL. SimplePie does a good job of finding feeds from a blog URL (in my experience). Have just checked the code and it does use SimplePie.

Using this to register a URL without having configured the HTTP proxy results in a long period of waiting and then the error “This feed is invalid”. Seems to suggest some limitations of the code. Wish I had the time to look at this more.

Configure my box with the proxy details and try again. Oops, that didn’t work. Ahh, “Some settings were not changed due to an error” an error message when saving the HTTP proxy that didn’t exactly leap out at me. Not immediately obvious what the error was.

Checking the database (mdl_config) reveals that the proxyhost wasn’t set, apparently you don’t need the “http://” and the error doesn’t identify that. Fixed.

Okay, that works. External blog registered. And posts from the blog showing up in my Moodle “blog”.

Let’s try BIM now. Nope. The timeout problem again. Implying there’s something different going on here.

Is there a difference?

Yes, eventually tracked down one of the calls to SimplePie is using the normal SimplePie class and not the moodle simple pie class. Hence not using the proxy setup.

Tested it with the student registering process and that works as well.