How to capture the “full benefits of the creative, original and imaginative efforts of” teaching staff

What’s good for research, must surely be good for teaching?

An article on the Australian’s higher education page quotes the following advice from this policy note from the Group of 8 (an obviously non-self-serving document, of course)

If Australia is to capture the full benefits of the creative, original and imaginative efforts of its researchers, it will always need a means to support the ideas and challenges coming from individuals and small groups, even when these ideas fall outside formal priority setting mechanisms

Having engaged a bit in the formal priority setting mechanisms around institutional e-learning over the last month or so, I was struck by how this perspective could be moved across from research to institutional e-learning.

I don’t think anyone could claim that the institutional governance processes around e-learning – especially the LMS – could ever be described as “a means to support the ideas an challenges coming from individuals and small groups”.

This is not to suggest there isn’t some level of need for these processes to ensure the availability of institutional systems. It is to suggest that if you want “creative, original and imaginative” efforts then the processes need (I would argue) to be able to to support the ideas an challenges coming from individuals and small groups”.

For example, as mentioned previously as part of the case for getting BIM installed on the institutional version of Moodle I had to explain why others might use it. It seemed that the governance processes/bodies etc didn’t know that there were 30 odd courses this year that were using learning journals of one type or another that might have benefited from BIM. There appears to be a lack of knowledge of the ideas and challenges of teaching staff and students with institutional e-learning systems within the priority setting mechanisms that “govern” them.

The trouble with this type of argument is that it’s strange. Perhaps because of the lack of knowledge about the issues and challenges, it’s impossible for those responsible to see a problem with the priority setting mechanisms. Or perhaps it’s an example of the following.

From “Status Quo”

Or, of course, it’s not that big of a deal.

How Knowledge Workers like to learn and implications for BIM and LMS design

I found out last week that the abstract I submitted to Moodlemoot AU 2013 had been accepted. The talk will attempt to outline what I’m hoping will be my primary line of research over the next couple of years, which is probably going to be something like

  • How can the design of institutional e-learning tools be improved to support teachers and students to learn?
  • If this is done effectively, what happens?

The focus on institutional e-learning tools is mainly one of self-interest. I have to work with these tools in my current position and I want better tools. If my research can help my teaching, then it’s two birds and one stone.

In an earlier post I gave an initial idea of the “knowledge” problem that is one area with potential for improvement. i.e. most of the existing e-learning tools do less than a stellar job of helping teachers and students develop/access the sort of knowledge needed to get the most out of e-learning.

This morning, first Stephen Downes and then Steve Wheeler took me over to Jane Hart’s 5 characteristics of how Knowledge Workers like to learn at work. A post that describes findings from a 2013 “Learning in the Workplace” survey with 600 respondents from 46 countries. The image below summarises the 5 characteristics.

These characteristics may offer suggestions about how e-learning tools can be better designed to help teachers and students.

In the flow of work and other characteristics

The first characteristic is “In the flow of work” which Hart describes as

Workers don’t want to leave the workflow unless it is absolutely necessary for them to do. This means EITHER physically to go to a classroom OR virtually to work on an online course for an extended period of time (i.e more than about 10-15 mins) and/or which is more than a couple of mouseclicks away. (Taking a course at your desk, doesn’t mean it’s in the workflow!) Workers prefer to learn as an integral (NOT an extra) part of their daily job and not separately from it, either.

This resonates with me. For better or for worse, when I’m currently teaching “in the flow of work” means within my Moodle course site. The Moodle course site is the learning environment I work most in. If I want to learn – be it something about the students in my course or some new pedagogical strategy or technological technique – I would prefer it to be in the flow of work. i.e. in the Moodle environment. When I’m using BIM (or any other tool) I want it to be able to help me learn about my students, their learning and how I can improve it. I don’t want to book a session with an instructional designer or attend a scheduled training session. Raising the questions of what shape might this take and how might you do it?

The remaining characteristics offer similar suggestions. In particular, Immediately

Workers want to be able to find answers to their learning and performance problems as soon as they encounter them

which has some overlap with the above.

Socially suggests knowledge workers want to learn from others, as they work both internally and with external networks and communities. Are there any e-learning tools (within an LMS) that allow teaching staff to connect with a network of other people using the tool? To compare and contrast how others are using the tool and learn new ideas about how the tool might be used.

Making e-learning tools that are more supportive – BIM, TPACK and truncated feeds

The following is a mini-argument for and example of how the e-learning tools should be made more supportive. i.e. actually help the staff and students using them actively address common problems in a pro-active way. It continues some more thinking about an earlier question I asked, Does institutional e-learning have a TPACK problem?” and hopefully will inform the on-going research and development around BIM (now officially released to the Moodle community) and also inform the Moodlemoot’AU 2013 presentation I proposed.

The problem

The last post mentioned the recent research around the increasing workload faced by academics dealing with the current practice of e-learning in Australian Universities. One of the premises of my thinking is that a contributing factor to this workload pressure is that the tools aren’t provided sufficient help.

Just recently @masmithers re-tweeted an “oldie but goodie” blog post of his from 2011 – eLearning at Universities: A Quality Assurance Free Zone? – in which he reports on a range of poorly designed online courses he had seen. He talks about the “secret communion with students in the classroom” as one of the contributing factors for this. The comments on the post point to a range of other factors: “no requirement to have any teaching experience or qualifications” and limited (if any) funding for the move from on-campus to online (or dual mode). There are also a couple of comments along the lines of “we are at the mercy of crappy tools….If we’re going to build decent sites, we need decent tools” and connecting back to the workload question “spend extra hours and time away from my family trying to remediate what is essentially a bad system”.

The questions I’m keen to explore are along the lines of

  • Would the provision of better tools help reduce workload and increase the quality of the learning and teaching experience?
  • What does it mean for these tools to be better? What types of problems need to be removed? What positives built in?
  • How can this be type of improvement be carried out within the current institutional processes?

A solution

In the previous post and my current line of thinking is that the TPACK (Technological Pedagogical And Content Knowledge) framework can provide a useful lens for thinking about this problem. The basic idea is that

  • TPACK proposes that it “identifies the knowledge teachers need to teach effectively with technology” (Koehler, n.d.).
  • This knowledge does not need to reside in each individual academic, the tools they use can help provide this knowledge (either directly or by providing in context connections to others).
  • Can we use TPACK to identify the type of knowledge that we need to design into these tools.

An example

BIM is the tool that I plan to use to explore these ideas. It’s a tool I wrote and hope to increasingly use in my own teaching. The following illustrates one idea for how BIM could be re-designed to better contribute to the overal TPACK required to “teach effectively with technology” (Koehler, n.d.).

BIM is all about aggregating the posts students make to their own individual blogs. These blogs are hosted on whatever blogging platform they decide. BIM provides ways to mark (both manually and eventually automatically) the student posts.

Today, one of my students has reported a problem with the marking of her blog. After a bit of email tag between both of us, I have identified that the student has configured her blog so that the RSS feed generated has summaries of her posts, rather than the full text. This means BIM cannot see the full post, it’s marking a portion. No surprise it got it wrong.

This problem has caused confusion and disappointment on the part of the student. It has required her to expend more effort on chasing this up and required me to do more work to diagnose and remedy the situation.

Wouldn’t it have been so much better if BIM was capable of identifying this problem as soon as it happened and informed both the student and myself about the problem? Technically, it would be fairly easy to implement this.

Doing this requires that the tool have embedded into it a lot more technical knowledge (e.g. that feeds might be summarised and how that looks) and the ability to make use of that knowledge.

Being aware of this need requires that the people capable of designing and changing BIM, are close enough to its operations that this type of problem becomes recognised. I’m not sure that in all situations this is the case. How can a tool like BIM be designed to make this possible?

And they don’t even know enough to expect better

The title for this post is (probably a slight re-phrasing) of something @palbion mentioned last week during a conversation about the low quality of information systems within higher education (or at least our experience thereof). The comment was in relation to the professional and academic staff who are struggling with the various information systems universities are increasingly using to support tasks such as managing research higher degree students form application through to graduation, managing the process of sending students out into industry for practicums, and of course the more general LMS and student records system.

All of the staff involved are having to bumble along with systems with inherent limitations and attempt to develop what workarounds they can. For example, the system that sends out an email to students saying that their application is incomplete and to try again. The problem is that it doesn’t tell the students what is missing (even though it must know to have identified the incomplete state) and doesn’t tell them how to try again. Or, for example, the online assignment submission and management system that requires the staff involved with marking to repeat the same manual process for each assignment they are marking. Repeating processes is what computers are good at, not human beings. This inevitably leads to mistakes which need to be fixed. Leading to less than stellar efficiencies (which last weekend took on a much higher profile within Australian higher education)

The point of the title and Peter’s point was that many of the staff struggling with these systems think this is how information systems work. In their experience there has always been problems like this, it’s something you live with. They don’t know it can be better.

This particular discussion also arose out of some of my earlier discussions about the limits of our students’ technical knowledge. A reason, perhaps a significant reason, that these limitations became obvious was the poor design of the technical (perhaps socio-technical systems) within which the students had to operate.

Experience with the online assignment submission system suggests that it’s not just the students that are struggling with these problems

Application or targeted learning analytics – another scope?

Was skimming through Mark Drechsler’s slide deck from THETA 2013 when I came across the following slide.

Analytics Scope

It’s part 3 of a model of learning analytics (Target, Consumer, Scope, Automation) Mark used in his talk and got me thinking and hence the following. Still early days on this.

The slide above describes two of the extremes of the data being used in learning analytics. Just the stuff from the LMS at one end through to the entire learning ecosystem. As the slide points out, the former is increasingly becoming limited and the latter is just a bit pie in the sky.

I’m wondering if there is another part of the scope that might be a bit more fruitful, or at least of more interest to me. i.e. learning analytics at the application – or to use Moodle speak, the module/plugin level.

Tweeting from the Learning Analytics and Knowledge 2013 Conference @shaned07 suggests

This resonates a bit with the point @beerc made in Beer et al (2012)

The inherent unpredictability of agents within a CAS (complex adaptive system) suggest that the most appropriate place to situate learning analytics tools and resources designed to inform and improve online learning and teaching, would be within the micro-level context.

One interpretation of this “micro-level context” could be the individual application. i.e. the scope is a specific application being used for learning. Perhaps the best example of this is the discussion forum and the SNAPP tool (as it happens an output from @shaned07 and friends).

It’s within the application as it is applied within a particular learning context where the deepest knowledge about what is happening may reside. Analytics that help the student or the teacher understand what is (or isn’t) happening would seem to have the best potential of improving outcomes.

It may be helpful to have information from other sources, but I wonder if more specific analytics about the specific learning process as embodied in a particular tool can be more useful.

Of course, this raises the question about how such tools are used. For example, I’m pretty sure that many discussion forums aren’t used in a “pedagogical” way.

I’m wondering whether the origins/connections of learning analytics with data warehouses and business intelligence units (not to mention the nature of big data toward collating information into “big data”) drives this interest in the broader scope at the expense of the specific? Wondering just how useful “application analytics” might be as it would likely lose the “big” in big data?

Much more to think about here, but I have to get back to other work.

References

Colin Beer, David Jones, Damien Clark (2012) Analytics and complexity: Learning and leading for the future. In M. Brown, M. Hartnett, & T. Stewart (Eds.), Future Challenges, Sustainable Futures. Proceedings of ascilite Wellington 2012 (pp. 78–87). Wellington, NZ.

More evidence of the limits of student technical knowledge

The following is just a “diary entry” recording a bit more evidence for the story that our students are neither digital natives nor digitally literate. It may or may not become useful in future research/writing. It’s not meant to be insightful, just a record of an experience.

The context is marking of assignment 1 for EDC3100. 300 odd students have created online artefacts via their choice of online tool. Youtube videos, Wix/Weebly/Wordpress websites, Sliderocket and Prezi are the most common I’ve seen so far. There have been some really good ones and some not so good ones. But there’s also been some evidence to suggest limits on the student’s technical knowledge.

Most of the problems appear to revolve around the idea of providing a URL to a post on the student’s blog that includes a URL to the online artefact. The double link caused some problems, but also has the idea of providing a URL. Some examples from tonight

  1. Rather than provide a URL for the post, students are providing the URL for their blog.
  2. A small number of students is providing a URL to their blog, which doesn’t have any posts with links to their online artefact.
  3. Prezi URLs.

    Been a small trend with Prezi URLs not working. It appears that the students are providing a “long URL” generated from something they see. I’m assuming copying from the browser. This URL doesn’t work for anyone but them. If we cut away some extraneous material, we get to a URL that works.

  4. Spectacularly wrong URLs.

    For example, we’ve seen URLs like this

    davidjones@edublog.org.com

    for blogs that are actually located at

    http://davidjones.edublogs.org

As mentioned previously

  • These are 3rd year students the majority of whom have some significant online learning experience beyond their own typical use of social media.
  • This perhaps says more about the technology and its design and use than the students themselves.
  • It raises questions about some of the assumptions underpinning common institutional e-learning practice within universities.
  • It raises questions about whether encouraging exploration, creativity and student choice can be viable in a course with 300+ students and limited time and support resources.

    i.e. the time I’ve spent diagnosing and fixing these mistakes has taken time away from engaging with student queries about the course content and assessment.

An ad hoc exploration ethnographic research

The following is an initial attempt to restart some earlier explorations of research methods that may prove useful in examining the “Story of BIM” for potential useful insights. The starting place is ethnography and auto ethnography and an exploration of some writings.

Rescuing Autoethnography

Atkinson, P. (2006). Rescuing Autoethnography. Journal of Contemporary Ethnography, 35(4), 400–404. doi:10.1177/0891241606286980

Apparently a response to or a continuation of an on-going debate about the value and problems of analytic autoethnography.

(Atkinson, 2006, p. 401)

These are not just matters affecting the choice of fieldwork site but a biographically grounded, experientially rich engagement with the social processes that are observable in the field, and that render those processes comprehensible in particular ways.

Comment arising from a range of examples where the particular skills and background of researchers enabled engagement/insight that would have previously been not possible.

This close connection need not be justified “exclusively on postmodernist rationales” but is indicative of a longer history of close relationships between the researcher and the informant. More broadly the idea of understanding a social life ethnographically depends on the “homology between the social actors who are being studied and the social actor who is making sense of their actions” (Atkinson, 2006, p. 402) This is linked to the fundamental principle of reflexivity – a much abused term – which is defined as

the ineluctable fact that the ethnographer is thoroughly implicated in the phenomena that he or she documents, that there can be no disengaged observation of a social scene that exists in a “state of nature” independent of the observer’s presence, that interview accounts are coconstructed with informants, that ethnographic texts have their own conventions of representation. In other words, “the ethnography” is a product of the interaction between the ethnographer and a social world, and the ethnographer’s interpretation of phenomena is always something that is crafted through an ethnographic imagination. (p. 402)

If it’s so embedded in ethnography, what is the problem with auto-ethnography? When the ethnographer becomes more memorable than the ethnography.

The solution to this is to insist on the analytic aspect of ethnography. The “experiential value, its evocative qualities and its personal commitments” should not be promoted at the expense of the “scholarly purpose, it’s theoretical bases and its disciplinary contributions”.

Obviously a time to look at the rest of the discussion.

Analytic autoethnography

Anderson, L. (2006). Analytic Autoethnography. Journal of Contemporary Ethnography, 35(4), 373–395. doi:10.1177/0891241605280449

Appears to be the origins of “analytic autoethnography”. An explicit attempt to distinguish it from “evocative autoethnography”. Analytic autoethnography is defined as, (Anderson, 2006, p. 373)

research in which the researcher is

  1. a full member in the research group or setting;
  2. visible as such a member in published texts, and
  3. committed to developing theoretical understandings of broader social phenomena.

Ellis, Bochner and Denzin said to be influential in rise of auto-ethnography. But Anderson does also outline a broader history of “an autoethnographic element i qualitative sociological research” (Anderson,2006, p. 375). Though much of this work “continued the earlier tendency to downplay or obscure the researcher as a social actor in the settings or groups under study” (Anderson, 2006, p. 376). They were “neither particularly self-observational in their method nor self-visible in their texts”. There were other strands of research but it is 1979 and an essay on autoethnography by David Hayano in 1979.

There is then the rise of “the descriptive literary approach of evocative autoethnography” (Anderson, 2006, p. 377). An approach that moves away from the analytical and toward an empistemology of emotion. Anderson (2006, p. 377)

Evocative autoethnographers have argued that narrative fidelity to and compelling description of subjective emotional experiences create an emotional resonance with the reader that is the key goal of their scholarship.

Within the paper five key features of analytic autoethnography are then proposed, these are

  1. Complete member researcher (CMR) status.

    quotes Merton (1988, p. 18) describing the research as “the ultimate participant in the dual participant-observer role”.

    Patricia and Adler (1987, p. 67-84) identifies 2 types

    1. opportunistic” – the more common, born into a group, thrown into it by chance or acquired familiarity through occupation, lifestyle etc. Group membership may precede the research decision.
    2. covert – begin with a purely data-oriented research interest but are converted into immersion and membership during the course of research.

    Membership gives close connections, but does not “imply a pnaoptical or nonproblematic positionality” (Anderson, 2006, p. 380). Auto-ethnographers are apart in that the spend time documenting and analysing action as well as engaging in action.

    Analysis raises the “Schutzian distinction” (Schutz, 1962) between practically oriented, first-order interpretations and the more “abstract, transcontextual , second-order constructs of social science analysis”. Then there is the problem of the variety of first-order interpretations within the social social groups. Different members see things differently and the researcher’s role in the group makes some of these more accessible than others. This leads to the question of how or even if it is possible for the auto-ethnographic research to achieve “becoming the phenomenon” (Mehan and Wood, 1975, p. 227).

    What the auto-ethnographer “knows”/learns emerge from engaged dialogue, rather than detached discovery

  2. Analytic reflexivity.

    reflexivity involves an awareness of reciprocal influence between ethnographers and their settings and informants. It entails self-conscious introspection guided by a desire to better understand both self and others through examining one’s actions and perceptions in reference to and dialogue with those others. (Anderson, 2006, p. 382)

    For auto-ethnographers it goes deeper than this. Their data arises from their own experience and sense marking. They are part of the representational process but are also partially formed by those processes through co-creation in conversation, action and text.

    While this is an important component, it’s not enough to engage in reflexive social analysis etc. There’s a need to be…

  3. Narrative visibility of the researcher’s self.

    In convention ethnography there is apparently a problem with the enthnographer being often invisible in the text, but omniscient. Even though this has not always been the case.

    Autoethnography “demands enhanced textual visibility of the researcher’s self. Demonstrate the researcher’s personal engagement in the social world. Illustrate analytic insights through recounting experiences and thoughts as well as those of others. Should also “openly discuss changes in their beliefs and relationships over the course of field work”. To show the grappling that occurs with issues in “fluid rather than static social worlds”.

    The goal of reflexive ethnography (and autoethnography) according to Davies (1999, 5) is to “seek to develop forms of research that fully acknowledge and utilize subjective experience as an intrinsic part of research” (Anderson, 2006, p. 385)

    The descent to self-absorption is where autoethnography loses its value. The visibility has to be more than “decorative flourish”. For analytic ethnography the aim “is to develop and refine generalised theoretical understandings of social processes”.

  4. Dialogue with informants beyond self.

    Quote from Rosaldo (1993, 7) “if classic ethnography’s vice was the slippage from the ideal of detachment to actual indifference, that of present-day reflexivity is the tendency for the self-absorbed Self to lose sight altogether of the cuturally different Other.”. There is a need to engage with others in the field. “No ethnographic work – not even autoethnography – is a warrant to generalise from an “N of one”. There is a need for dialogue with “data” or “others”

  5. Commitment to theoretical analysis.

    There must be some aim to use “empirical data to gain insight into some broader set of social phenomena than those provided by the data themselves” (Anderson, 2006, p. 387). Using evidence to formulate and refine theoretical understandings of social processes. This narrower definition of “analytic” is in line with Lofland (1970, 1975) and Snow et al (2003) points to “a broad set of data-transcending practices that are directed toward theoretical development, refinement and extension”.

    The aim is not to produce “undebatable conclusions”, but instead to contribute “to a spiraling refinement, elaboration, extension and revision of theoretical understanding”.

Virtues and limitations of analytic autoethnography

Analytic autoethnography is positioned as a sub-genre of analytic ethnography.

Virtues fall into

  • Methodological.

    Being a CMR makes data more available. There are multiple incentives to participate. But the multitaking also creates potential pitfalls. Research focus fading. Participation outweighing writing of field notes.

    There is also access to “insider meanings”.

    Personal involvement also provides access to data not normally available.

  • Analytic.

    Provides “grounded opportunities to pursue the connections between biography and social structure”.

Limitations. Most don’t find research interests that are deeply intwined with personal lives – as required by autoethnography. Analytic ethnography assumes a “professional stranger” role.

There is little more conversation of the limitations, beyond that all methods have limits. Perhaps this is taken up more by the rest of the articles in the issue.

Conclusions

Specific research method flourish in the absence of other well-articulated methods. This is one explanation given for the rise of evocative autoethnography and the paucity of analytic autoethnography.

An example closer to home

Time to explore autoethnography a bit closer to the context or type of application I’m interested in.

Clark, C., & Gruba, P. (2010). The use of social networking sites for foreign language learning : An autoethnographic study of Livemocha. In C. Steel, M. Keppell, P. Gerbic, & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010 (pp. 164–173).

Data collection – self-aware participation, learner diaries and peer debriefing. To investigate use of social networking sites in foreign language learning. A grounded, thematic analysis used.

In describing the method, starts by mentioning history of “large-scale diary studies” in a range of fields. “particularly useful to examine events in their natural context to obtain reliable, person-level information”. Autoethnography can be considered “unstructured, uncontrolled ….and necessarily subjective and anecdotal”. And a quote or two to justify.

One author recorded all language learning experiences with the chosen site in 3 phases: register as himself, a four week study of Korean starting from scratch. Detailed notes were taken. Using Bolger et al (2004) principles for event-based autoethnographic design – details/impressions/experiences were recorded during and after. Later collation of all into a learner journal.

Phase three – thematic analysis involving two analysts. Primary issue was return to the site and continued study – was the site “addictive and effective”.

In terms of continued use: three themes emerged – motivation, frustration and demotivation. These are explained in detailed, summarised in a table and linked to suggestions for pedagogical improvement.

And another

Duarte, F. (2007). Using Autoethnography in the Scholarship of Teaching and Learning: Reflective practice from “the Other Side of the Mirror”. International Journal for the Scholarship of Teaching and Learning, 1(2), 1–11.

Initially strikes me as potentially more evocative autoethnography.

this essay is based on my reflections and recollections of important events and insights that occurred during the Redevelopment Project, and on the notes of the reflective journal I kept to document my shifts of consciousness as I gained new pedagogical knowledge and skills.

The author has used autoethnography in prior research and argues that it has good links with SoTL – particularly due to the focus on reflection in both. Gives quotes from Bass around SoTL and reflection.

The story told reveals a great deal about the experience of the “early adopter” of blended learning in a system with technology and processes that isn’t set up well to handle it. In particular focuses on the “time pressures created by the shift to blended learning”. Includes a reference to a study by Lefoe and Hedberg (2006, p. 334)

Some other interesting quotes there. Not sure the paper offers a strong example of a method that might be accepted, but some good insights into blended learning and it’s implementation.

Adding bim 2.0 to “CONTRIB”

Next step in the development of bim v2.0 is to start the process of submitting it to CONTRIB. i.e. essentially getting out officially into the Moodle community.

The following is my attempt to figure out and record the process for doing this. This was actually started a couple of months ago but then semester started, a bit of breathing space now and I need to catch up on this.

Evidence of progress

Overall, the process hasn’t been that difficult. A little disjointed in places and between that and my rushing it in places, there may be a few things to fix up. But it’s done.

Will be interesting to observe what happens from here. Beyond responding to that, it’s time to start thinking about further changes to BIM.

Finding the right process

Of course, the background on CONTRIB I linked to above is obsolete. CONTRIB is using git, not CVS anymore (I believe). Time to find where the right process is documented. It’s a bit of an issue when the obsolete documentation on this are what Google is returning as the top hits.

The “Guidelines for contributed code seem more up to date. The process listed – or at least my interpretation – is summarised in the following sections.

Create a git repository

Create a git repository, preferably using the format moodle-{plugintype}_{pluginname}.

This implies I’ll need to rename the existing BIM git repository. I wonder what support git has for this? I discover and implement one approach below.

Forking the existing bim repository might be an option. But not sure I would want to maintain the connection. Can it be renamed? Well apparently there is a feature to do this. That appears to have worked, the question will be what ramifications are discovered into the future. At the very least, I imagine my local clones of bim will need to be updated.

Here’s what I did

git clone https://github.com/djplaner/moodle-mod_bim.git
cd moodle-mod_bim
git checkout bim2
mv moodle-mod_bim bim
# make some changes
git push

All seems to be working. Of course, all the hypertext links on the blog are now broken because of the new repository name. Will have to update a few of those.

Test the code

Make sure the code is tested. Here’s what I’ve done to date

  • A range of testing while under development.
  • Including some testing of BIM under versions 2.4.1, 2.3.4, 2.2.7 and 1.9.19 of Moodle.
  • Thanks to a bug report from a Russian user of BIM solved a problem with 2.3.2 version of Moodle.
  • Been using BIM for my current teaching. Not in a full on way but it’s getting some testing.

Some more work to do

Backup and restore test

Do a backup and restore of the EDC3100 blogs between BIM installs on different versions of Moodle. Doing this from 2.4 to 2.4 as well as 2.3 will also provide me with a decent test space for the issues below and in an on-going way. 300+ students with multiple posts is a much more reasonable foundation for testing. That worked surprisingly easily.

Revisit open issues

Some recent playing with BIM revealed a potential issue, so need to explore that a bit and also look at any of the other immediately open issues that should be addressed before adding the code to CONTRIB.

  • Problem with adding questions and hanging process_unallocated
    With a BIM activity already created with students registered and posts mirrored, it appears that when you add a question, then the function process_unallocated (attempts to decide if any student posts match the question) hangs.

    Recreate the problem. Add a question to a copy of the EDC3100 BIM activity and do the processing thing. And I can’t re-create it. All working as expected. Thinking this may have been due to proxy problems giving the appearance of a problem.

Documentation

Documenting the plugin is important and suggested to be done on the English Moodle docs.

Where would the BIM docs reside? It should be in “the most recent version of Moodle for which the plugin works”. One of the bits of advice for the process is to go into BIM and find where the link for “Moodle docs for this page” points. Which in the case of BIM is here.

Of course, it’s not that simple. It appears this is the proper place to get started http://docs.moodle.org/24/en/BIM_module

What is required of the docs, The stamp collection module is given as an example and the provided list of contents includes:

  • Template code.
  • Features overview.
  • Installation instructions.

Done sufficiently for now, I hope.

Request to be tested

Request that the code be tested/reviewed. Done, at least I think this is what was required.

Add it to the plugins directory

Share the code in the Moodle plugins directory.

Need to create a zip file containing the module that can be installed and tested. Apparently this will do it.
git archive -o ~/Desktop/BIM2.zip –prefix=bim/ bim2

Will need to test that this works ok.

  • Delete BIM from Moodle install.
  • Unzip the zip file created by the above.
  • Go to notifications.
  • Install it.
  • Create a BIM activity.

That seems to work. I do wonder what I’ve missed.

Well, $module->release in version.php appears to be one of those things.

While I am here, might be a good time to update the BIM icon. Moodle 2.x appears to support much larger icons and the image @rolley provided for BIM 1.x doesn’t scale too well. That appears to be a step too far for me.

I’ll leave it at that. It’s been uploaded. Not sure I’ve gotten everything, but it will do for the day.

Meaningless freedom and auto-marking the learning journals

The course I’m teaching requires each student to create and user an individual blog. The blog should be created on an external blogging platform of their choice and used to reflect on their learning in whatever way they see fit. There are a couple of constraints around regularity (at least 2/3 posts a week), length (average of 100 words), links to resources (60% with links to online resources), and links to other student blogs (2 of all posts over a 3 week period). All this is meant to be automatically marked.

The following is the story of putting in place the code to check, track and mark the student blogs. Much of it has been written over the last few weeks. I’m adding the last extra touch today as the assignment has been submitted and the automated assessment needs to be completed.

As it happens, I’ve also just read Lisa M Lane’s “The illusion of the LMS/cloud-based/self-hosted solution” and am finding that it resonates strongly. If I didn’t have the technical background I have, none of the following would have been possible. I’d be constrained by the tools available in the LMS and any manual workarounds I could come up with. As it is, I could have done without the additional work required by the following.

At the moment, the message I’m taking from both Lisa’s and my own experience is that the use of technology in learning and teaching is messy. Especially when you’re trying to do something different. Being an explorer is always going to be difficult. The institutional systems and support processes are not set up for exploration, they are set up for exploitation. This is why they are constraining. If you want to be an explorer, it’s going to be hard, but it can also bring benefit. Alen Levine’s comment on Lisa’s post perhaps contains the main solution to this problem

the way to do this on the open/free/public end is to leverage the connections of others. I rely on this all the time. The “solving” is in our human networking.

No BIM

I haven’t completed BIM2 in time for the organisational processes to consider installing it into the institutional version of Moodle (this post details one step in the process). So the plan is

  1. Students register their blog via a Moodle database activity.
  2. That is exported, checked and stuck into a local version of Moodle (with BIM) on my laptop.
  3. Marking of the blogs will be done via some additional code, either in BIM or in Perl.

    At this stage Perl has been used because I have a large collection of infrastructure and experience with a Perl code base that was developed as part of my PhD work. i.e. I’m a native Perl speaker, PHP and the Moodle code remains a second language to me. Eventually this work will need to be brought into BIM in some ways.

Registration, reassurance and the perils of meaningless freedom

Way back in the late 1990s experience with the design and use of online assignment submission systems led to this observation (Jones, 1999)

An important lesson from the on-going development of online assignment submission is to reduce the amount of “meaningless freedom” available to students. Early systems relied on students submitting assignments via email attachments. The freedom to choose file formats, mail programs and types of attachments significantly increased the amount of work required to mark assignments. Moving to a Web-based system where student freedom is reduced to choosing which file to upload was a significant improvement.

Having the students register their blogs with a Moodle database activity meant that the students had to correctly

  • Enter their student number.
    USQ has two types of student number that it users interchangeably.
  • Copy the URL of their blog.

Here’s a list of what I’ve found in the registered data tonight, out of 275 registered blogs

  • 105 students used one form of number, 170 the other sort (roughly).
  • 10 student numbers were incorrect.
    Some were just minor typos, but others were more major.
  • 37 URLs were incorrect
    Missing the http://, typos (edublog.org not edublogs.org etc.), copying the dashboard URL not the home page.

This is not to suggest that the students are stupid. It’s to show how badly designed systems (i.e. the stuff I’ve cobbled together) allow mistakes to happen. If BIM had have been available none of these errors would have been possible and I would have saved quite a few hours of work.

Not only would BIM have provided immediate feedback on registration, it would have allowed the students to be reassured what what was known about their blog. With BIM the just visit the activity and its there. In this semester, I’ve had to send bulk emails out letting students know what the system knows about their blog.

Statistics

Time now to finish off the script that will generate statistics about the students’ blogs and generate their mark. As shown in this prior post I’m also using this facility to generate some visualisations of the interconnections, but that’s another post.

The statistics being used for marking include

  1. Number of blog posts per week.

    Currently being calculated by dividing the number of existing posts on the student blog by the number of weeks.

  2. Average length of blog posts.
  3. % of posts that contain links to external resources.
  4. number of posts that link to other student blogs.
  5. % of the learning path activities completed.

    This isn’t a blog statistic. It’s from the activity completion report on Moodle. Each week has a collection of activities/resources (the learning path) and students are expected to complete them.

Each of these is currently being generated. But I need to

  1. Double check the links to other student blogs, not sure it’s counting blog posts.
  2. Exclude links to their own blog.

This is all done. So some statistics. With 330 students mostly still enrolled in the course

  • Average word count per post – 184.9
  • Average posts per student – 11.5
  • Average posts with links – 7.7
  • Average posts with links to another student blog – 1.7
  • Average completion of Moodle activities – 89.8%

The last one is a bit disappointing. Need to explore it more.

Missing students

I have a script that automatically “marks” the students blogs and also their completion of activities on the Moodle study desk. Trouble is it appears that at least one student is missing from that list. Why?

Some possibilities

  • The student has dropped the course? – NO, still there
  • The student didn’t register their blog? – YES, that’s the problem

If there’s one, I wonder how many others there are? Even after we did a dry run a couple of weeks ago to identify folks in this situation there appear to be a few. In theory, there are 327 students still enrolled in the course. Of those, 20 students haven’t successfully registered their blog.

Question is whether this is a problem with my kludges, or the students haven’t registered their blog. I’ll let them figure that out.

A visualisation

The following is the latest Gephi visualisation of the links between student blogs. A bit more complex than the last one, but obviously connections aren’t a priority.

Blog connections - EDC3100 1 April