Back into the thesis

After a week away traveling with my sons in Western Queensland, I have to get back into the thesis. This post is essentially an attempt to capture where I’m up to and plan out what I need to do for the next week or so. As with the last update, I’ll start scratching these out as I complete.

The outstanding work can essentially be broken into two parts:

  1. Make modifications to existing chapters; and
    The last PhD update has a good summary of much of this work.
  2. Complete the last chapter.
    Chapter 5 is the last chapter not yet in first draft stage, and it’s not far from it with only two sections not really there yet. So, the focus has to be on finalising those: the ISDT and Evaluation.

The ISDT

This is the major product of the thesis. Much of it is written, what’s remaining to doincludes:

  • Clean up the intro section to use the summary table.
  • Develop a section outlining that the ISDT is more a guide than a hard blue print – what other literature is there on this?
  • Define the purpose of constructs and double check the list in the table
  • Principles of form and function
    • Get a few broader definitions of architecture, especially leaning towards the inclusive idea.
    • Explain the scaffolding idea a bit more, perhaps borrow from this post.
    • Better explanation of human support as an aspect of scaffolding.
  • Principles of implemenation
    • Work in a connection with learning.
  • Artifact mutability
    • Work in the ideas proposed by Hovorka and Germonprez (2009) around artifact mutability.
    • Connect it with learning for academics, students and supporters.
  • Justificatory knowledge
    • Expand support for scaffolding conglomerations.
    • Problems and references for best of breed.
    • MOre work on principles of implementation – including learning
    • distill some of the references and double check outstanding references.
    • Cioborra and related work around the move away from the teleological and analytical.
  • Put more on BAM/BIM into the expository instantiation section.
  • Link the summary table of Webfuse testing with the results of the evaluation.

Evaluation

  • Finalise table 5.6.
  • Decide on which time frame to limit evaluation to and argue why.
    Use all of Webfuse post 2000. Rationale: While Webfuse did not at all times follow the ISDT, it did use aspects of it and even with just aspects showed some benefits. If all had been followed it may have been better again.
  • Table 5.14
    • Calculate number of students in courses with webfuse sites
      • Fix up 1998 entries in stats::courses to put in right term and also add 3rd term.1997, 1999, 2000, 2001 counts.
    • Re-examine course site hits, page updates, unique authors from 2002 onwards?
  • Table 5.15
    • Add standard deviation for updates per staff – based on updates from Table 5.14
    • Update accesses per enrolled student – based on data from Table 5.14
  • Complete the table “usage of webfuse courses”
  • Do power laws get a mention? — NO
  • % of BAM requests of overall staff requests for 2009.
  • Rethink the adoption section and draw on summaries of previous sections and also comparisons with similar section from chapter 4. Also incorporate work from the TAM papers.
  • Modify Figure 5.15 so that it uses a year where data is not missing.

Misc to dos

In working on the above I’ll almost certainly identify additional tasks that need to be performed, they are captured below.

  • Add a summary table of the ISDT in chapter 4.
  • Come up with a title for the chapter 4 ISDT.

References

Hovorka, D., & Germonprez, M. (2009). Tinkering, tailoring and bricolage: Implications for theories of design. Paper presented at the AMCIS 2009.

More problems with BIM and special characters

The following is a record of some work to investigate some more apparent problem with BIM mirroring blog posts that contain “special” characters due to a bit of copying and pasting from Word into WordPress.

An aside on supporting a tool like BIM

All of my previous software support has been around software that I (or the team I work for) have developed and for which we are also responsible for hosting and supporting the people using it. i.e. we knew exactly what was going on where.

With BIM the development, support and hosting are all separated. I write BIM, but I don’t host of support it. Which means I don’t know what the folk who are doing the hosting/supporting have done (or haven’t done). Which adds all sorts of complexity to the process.

This sort of separation is increasingly common and is often aimed at saving resources/money. But I do wonder whether or not that if viewed from an overall perspective it is adding more cost (in the broadest possible sense i.e. including the hassle and inefficiency caused by the difficulty but which doesn’t typically show up in anyone’s budget bottom line) into the whole task of supporting systems.

The problem

There are at least two, almost invariably related, problems:

  • Special characters that aren’t being translated safely.
  • Some situations where Moodle/BIM on Postgres is not able to insert student posts.

The postgres problem

There appears to a problem with Postgres/Moodle/PHP/BIM falling over when attempting to insert some posts. Maybe because of special characters. The only way I have of testing this at the moment is black box, i.e. re-creating it on a Postgres-based Moodle install. First step is to identify where this is happening and see if something can be done to make it not so catastrophic a failure.

Fail all posts, if one post fails

The error I’m seeing is

bim_process_feed: inserting bim_marking **url here**

Ok, that seems to match this bit of code

if ( ! insert_record( "bim_marking", $safe ) ) {
    mtrace( get_string( 'bim_process_feed_error', 'bim', $entry->link ) );
    return false;
}

Ok, the first thing here is that the “return false” should go. This breaks out of the whole insert process. What should happen is simply move onto the next entry and try to insert that.

Ok, that’s updated and into github.

Why are posts/inserts failing

The above was causing a problem because Postgres would report a failure when blog posts with certain characters were being inserted. The same posts are not causing a problem with MySQL.

Is this a known problem with Postgres? Are there solutions from the Postgres community that might help out here?

Doesn’t seem to be too much via Google, at least with what I was searching for. Guess I turn to exploring more the special characters in the posts that are causing problems.

More special character problems

The process here is to repeat what I did last time, modify my version of BIM to be somewhat explicit about the characters in posts it is inserting, find out which aren’t being handled, and then add some code to handle them.

The list of potentially problematic characters are:

  • a bullet point
    Turns out that this is a middot (search for 183. So, I’ve added a translation for this. This works, however, it does highlight a problem with “bad handling” described in the next section.

Bad handling

At least on MySQL, some of the changes seem to be introducing some rather weird translations. For example, a middot and some white space is translated into similar looking characters but with each surrounded by a single quote. This needs to be fixed.

Ahh, some code I lifted is using ereg_replace, which according to this is not multi-byte safe. Replace it with mb_ereg_replace, and all is good.

This seems to be the major problem.

Will see how it goes when testing with other problem data.

Light-weight analytics tools as part of scaffolding, context-sensitive conglomerations

A couple of days ago I floated the idea of scaffolding, context-sensitive conglomerations as one idea/model/suggestion for how e-learning systems (currently mostly LMS, but hopefully other models will arise).

George Siemens has posted about light-weight analytics tools such as SNAPP. Both the comments on that post are, to my current somewhat focused/biased perspective, suggestions for the need for scaffolding conglomerations. Both comments are from practitioners who talk about how the supplement their use of discussion forums with other forms of representation. It would appear obvious that these combinations of tools are useful. I’m pretty sure you could find quite a few talented and motivated academics across the world that are using this combination. I’m also pretty sure that few of them would be located within the same institution.

Are there any discussion forum tools in e-learning systems that already provide this sort of scaffolding for users? Are there any IT departments in universities that have recognised this need and are helping academics make this connection?

I’m not aware of any, and this suggests to me that there are some fundamental problems with the way these systems are being supported and structured. i.e. current approaches mean it is unlikely for these sorts of networks of tools/conglomerations to arise.

SNAPP has used a good approach that makes it simpler to create these conglomerations, through the use of browser plugins. But the advantages of that approach come with a negative. i.e. I don’t believe you can currently generate a SNAPP visualisation for a group of courses (e.g. to see how the students/staff in a program are interacting), or compare visualisations between different courses. You also can’t easily combine SNAPP with other context specific data sources such as student records system etc.

Consequently, SNAPP is a great example of a tool that enables a “scaffolding conglomeration” when combined with an LMS discussion forum. But it still remains difficult to add the “context-sensitive” component.

I think there’s value in exploring how SNAPP and similar tools can be used as both scaffolding and context-sensitive conglomerations, and more importantly, what impact it can have on the practice of teachers, and subsequently the quality of learning.

Adding advice

While I remember, there’s a next step that I’d like to see a “scaffolding, context-sensitive conglomeration” take. Advice, examples and connections.

For example, assume I’m teaching a course, I use a discussion forum and I’ve designed its use for a specific pedagogic purpose. I’ve installed SNAPP and to my horror discover a problem. What do I do? What strategies can I employ to address this problem? What strategies have other academics in similar (or even different situations) used? What happened? Can I get their contact details so I can have a chat?

I would imagine that this addition could also be implemented for students who discovered a “bad” pattern in their own practice. They could receive advice, examples and connections from other students.

The learning analytics group seems to include a leaning towards “intelligent”/”adaptive” software to provide this sort of service. I’m more interested in how we can use these tools to connect people and provide the scaffolding that enables and encourages them to take some action.

Misc. reflections on reading about situated cognition

For various reasons, mostly PhD related (and somewhat related to procrastination), I’m taking the time to read a bit more about situated cognition. Not sure how far it will go. The following are some ad hoc reflections and essentially a diary of what I’m reading. Not aiming for this post to fulfil any purpose beyond being a place to dump observations.

The wikipedia page

So far, the Wikipedia page on situated cognition seems fairly extensive and a reasonable place to start.

Misapplication of community of practice?

The wikipedia page has the following definition of community of practice

The concept of a **community of practice** (often abbreviated as CoP) refers to the process of social learning that occurs and shared sociocultural practices that emerge and evolve when people who have common goals interact as they strive towards those goals.

I find this somewhat interesting in that my experience with CoPs around university learning and teaching has been with special groups set up for specific purposes above and beyond normal teaching. i.e. rather than have a CoP around teaching at university X, where the common purpose is to teach. A CoP is set up around attrition, graduate attributes etc and focuses on that as the goal, rather than the teaching.

I do wonder whether this on-going ad hoc creation of CoPs around special topics that are important, but haven’t been embedded into common institutional practice is a symptom of CoP misuse. i.e. if normal practice of teaching within an institution was more like a CoP, would you really need a separate CoP on retention etc? Does the need for separate CoPs indicate that the normal practice of teaching within an institution isn’t like a CoP and hence, perhaps there isn’t a sense of a common purpose amongst those involved in the process of normal teaching? Instead of a common purpose, do the actors within an institution’s normal practice of teaching and learning have their own different purposes?

The glossary from which this definition comes highlights the amount of though and subsequently special language that has arisen around situated cognition. Also has some interesting resonance with the need for design theories to have a constructs section.

Affordances

Interesting to see some of the origins of the concept of affordances beyond Norman and HCI/usability. The idea that affordances are the individual’s interpretations about what action is possible within the given environment through their perception of the environment connects strongly with some of the problems I have around the stereotypical university environment around teaching and the nature of e-learning systems. i.e. I think the affordances seen by many teaching staff aren’t good in terms of improving learning and teaching.

The relationship between affordances and schemata seems an area of some disquiet and more reading – Glenberg and Robertson (1999)

Perception

This section was a little disappointing, it only mentions visual perception. While this is apparently an importent influence on situated cognition, even from my limited reading and knowledge there appears to have been a lot more work done on perception.

Memory

Again I feel there could be more here. But it does pose the interesting question of how situated cognition is downplaying the importance of stored, symbolic representations in memory. Instead having a belief that perception and action are co-determined by effectivities and affordances… Raises the question about what is
learning and knowing (which are covered next). Also links back to the disquiet about the link between affordances and schemata.

Knowing and learning

So knowing is not a think, a memory, but a verb, it is action/participation of an agent in an environment. This is where the idea that knowledge cannot be separate from context. It gives rise to the importance of context.

This is interesting, challenging and somewhat comforting. It is comforting because to some aspect it represents an idea embedded in how Webfuse worked. For Webfuse, context was important. Webfuse wasn’t a general purpose tool that could be used elsewhere, Webfuse could not be separate from its context. Mmm, situated cognition as a kernel theory for the ISDT and Webfuse?

It does seem that some of these sections have been made specifically very narrow and focussing on language/literacy learning.

Pedagogical implications

So, situated cognition is the theory of “mind/knowing/learning etc”, while cognitive apprenticeship etc are instructional design theories drawing on that theory, it appears.

Critiques

A small section on critiques close off the page. On the face of it, the critiques do a reasonable job of removing many of the assumptions on which situated cognition is based. Need to have a look further into Anderson et al (2000).

Of course the criticisms arise from cognitivists – more information processing types – who hold a perspective that has also been challenged. Interesting that it appears that Herbert Simon is one of the critiques (last author on Anderson et al).

References

Anderson, J. R., Greeno, J. G., Reder, L. M., & Simon, H. A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 29, 11-13.

Scaffolding, context-sensitive conglomerations in e-learning systems

For the last week or so I’ve been attempting to bring together the principles that underpin the design theory for e-learning that will be the main contribution of my thesis. This post summarises one of the principles of form and function that is emerging from the last week or so. I using this blog post to escape the confines of PhD-ese and see if writing about it can generate some further refinement.

Am aiming for brevity and clarity in the following.

A design theory contains principles for form and function; and for implementation. The following is more form and function, while the previous post is more implementation. There is more to the ISDT than just these two posts.

The problem

The functionality provided by most e-learning systems are in the form of low-level primitive tasks (e.g. upload a document, add a discussion forum, create a quiz). Most e-learning systems provide a fairly large collection of these low-level features. Many of these features, because they are designed for the general case, come with large numbers of options and configuration settings so that the task can be tweaked to the broadest collection of uses.

Consequently, achieving a high-level task (e.g. supporting a discussion forum that encourages student-to-student collaboration, or creating an educationally well-designed course website) requires a significant amount of knowledge, skills and time on the part of teaching academics. Some (perhaps much) of the time spent by teachers in leveraging these low-level, very general features is expended on bridging the gap between the generality of the feature set and the specifics of the context. Much of the time is spent trying to translate and wrangle what is known about good practice into the specifics required to implement it with the low-level feature set of the e-learning system.

I suggest that these difficulties limit the quality and quantity of how these feature sets are used. These difficulties appear to be one of the factors that contribute to the on-going fairly poor quality of most e-learning within formal education settings, especially universities.

Scaffolding, context-sensitive conglomerations as a solution?

The idea is that an e-learning system should provide the ability to create “scaffolding, context-sensitive, conglomerations” of its low-level feature set.

Examples

An example of this is the default course site “conglomeration” used in the Webfuse system. This is the system I designed and forms the basis for my PhD. So you can see where the idea has come from. The default course sites conglomeration was aimed at making it very simple for academics to create a default course site. So simple, in fact, that if they wanted to, they didn’t have to do anything.

Another possible example of a “conglomeration” from my own work could be the Moodle module BIM. BIM is a conglomeration of external blog engines (e.g. WordPress.com and various Moodle features, especially the gradebook. The BIM “conglomeration” is designed to make it simpler for teachers to manage students using individual blogs for assessment and other purposes. It provides a bridge between the external blogs and Moodle’s making/management features.

Moving further afield, the work being done by Desire2Learn with its instructional design wizard provide another approach to the provision of a conglomeration. The “Wizard” approach is used.

Conglomerations

In this context, a conglomeration is essentially a way to group together the existing low-level functionality provided by the e-learning system into something at a much higher level of abstraction. The term conglomeration has been chosen specifically to avoid terms like component, framework, package etc that, at least with some technical systems, refer to fairly specific ways of combining technology features. I’m trying to avoid these terms for a range of reasons, including:

  • Non-specificity;
    The aim of the ISDT (information systems design theory) is to provide broad guidance that is not specific to a particular approach or technology.
  • Inclusivity;
    Most of the technical approaches to grouping functional (e.g. component-based architectures) are not inclusive. i.e. if you’re using architecture X, you can only group features that are implemented using architecture X. While recognising there’s always going to be a practical need for something like this, with my ISDT I am trying to argue that for e-learning systems, the conglomeration mechanism needs to be as inclusive as possible (e.g. BIM’s reliance on RSS/Atom feeds – loose coupling – rather than a specific API for a particular blog service).
  • User focus.
    As part of all this the conglomerations only have to group these features from the perspective of the user. It doesn’t actually mean that they are technically grouped through the use of the same architecture. What’s important is that the user experience with the conglomeration leads them to believe the different bits of low-level functionality are integrated and working together.

Context-sensitive

Context-sensitive implies that the conglomerations know something about and offer support for factors or knowledge that is specific to the context of the people using the conglomeration. i.e. it’s not such a general purpose tool as to require the user to bridge the gap between their context and the tool.

For example, the default course site approach within Webfuse was designed to know about and integrate with as much of the local context as possible. The course synopsis, details about the assessment, staff details etc were automatically drawn from institutional databases and available within the conglomeration. In a more able and educationally enlightened context, the default course site conglomeration might have known about and offered specific support for the use of institutional graduate attributes or course learning outcomes.

Scaffolding

This aspect is still a work in progress and builds somewhat on the context-sensitive attributed. The term scaffolding was chosen because of the connection between this idea and situated/distributed cognition, cognitive apprenticeship etc. The idea is that the e-learning system should be designed in a way to help the teachers (and perhaps students) learn about using the system effectively.

The assumption is that for most teachers using the e-learning system to improve and/or change their teaching practice is a learning process. They haven’t done this before. Rather than treat learning about how to make this change a separate professional development activity, the e-learning embodies/embeds the learning into its conglomerations. In some way, the system adopts an approach that includes situated modelling, coaching and fading.

Collins et al (1991) define it this way

When scaffolding is provided by a teacher, it involves the teacher in executing parts of the task that the student cannot yet manage. A requisite to such scaffolding is accurate diagnosis of the student’s current skill level or difficulty and the availability of an intermediate step at the appropriate level of difficulty in carrying out the target activity. Fading involves the gradual removal of supports until students are on their own.

Am still considering the implications of this addition and how far to take it/interpret it. Potentially this could suggest that the system have a model of the teachers’ abilities and be “intelligent”. I don’t see this as being a requirement. An equally plausible, and probably more likely, approach would be for this diagnosis to be performed by people.

Need to think this through some more and see how much more of the insights offered by the learning theories this idea is based on could/should be used.

References

Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator(Winter), 6-11, 38-46.