Understanding and using the idea of “network learning”

The following seeks to engage with some thoughts shared by Brigitte, bring together some earlier ramblings of my own, and connect this with R&D related work I should be doing over coming months (though it’s historically rare for those plans to come to fruition).

The title of Brigitte’s post is the question “What is networked learning?” This is an important question in the context of the NGL course we’re participating in because the overall focus is developing your own answer to that question, identifying the principles of your conception of NGL, and then using those principles to design a change to how some task you are involved with “as teacher”.  Hence if your answer to “What is networked and global learning?” isn’t all that great, the rest of what you do will suffer because of it.

Features of less than great answers

It’s not hard to see less than great answers to this question. The following lists some of the features of those that I’m familiar with.

It’s the technology, isn’t it?

The most common is that the use of networked digital technology (even an LMS) is the key feature of network learning. Or if you’re really cool, it’s use of blogs, Diigo, Twitter, Facebook, Instagram, Slack or insert latest sexy networked digital technology. While I’m keen on digital technology and it can be a great enabler for efficiency, or a great catalyst for rethinking and transformation of practice. It’s just a (increasingly useful) means to an end.

This post from last year – titled “There’s more to it than the Internet and social software” – picks up a similar refrain and links it to various thoughts from 2015 NGL participants and beyond, including the idea that everything is a network.

It’s groups of people, isn’t it?

Another common less than great answer revolves around groups of people. i.e. multiple people all working toward a common goal. An answer that often suggests that the absence of commonality of purpose (or some other form) means it can’t be what passes for networked learning. And/or, it’s an answer that often assumes that a single person – someone not talking directly to someone else – can not be engaged in what passes for networked learning.

In this comment on one of my earlier blog posts comparing connected and networked learning, Nick Kelly expands the comparison to include communities of practice. The most common “groups of people” model that comes to most people’s minds.  The particular view of network learning Nick uses in that comment is described as

NL emphasises the possibility for technology and design to enable better connections between learners and between learners and resources

Nothing there about common purpose.  It’s a definition that includes the idea of connections between learners and resources.

It’s about students (and teachers), isn’t it?

Another common less than great answer tends to limit network learning to the learners. Or, as I suggest in this post it might also include the teachers

Typically networked learning – at least within an institutional setting – is focused on how the students and the teachers are engaging in networked learning. More specifically, how they are using the LMS and associated institutional systems (because you can get in trouble for using something different).

But what about everyone else? If we live in a rapidly changing world where ubiquitous digital technology is transforming the very assumptions upon which we operate, aren’t we all learners who might benefit from network learning? Harking back to Nick’s description above

NL emphasises the possibility for technology and design to enable better connections between learners and between learners and resources

Which is the point I try to make in the earlier post, that network learning shouldn’t just be thought of as what the students and teachers engage in, but as

how the network of people (teaching staff, support staff, management, students), communities, technologies, policies, and processes within an institution learn about how to implement networked learning.

The argument made in this paper is that the use of digital technology to enhance learning and teaching in most formal educational institutions is so terrible because “everything is a network” is only thought to apply (if then) to learning and teaching, not the support and management roles.

Learning and knowledge are people things, aren’t they?

In this paper some colleagues and I draw on what Putnam and Borko (2000) have to say about new views of knowledge. Views of knowledge that certainly do not agree that knowledge is something that is solely in people’s heads. It’s a view that’s connectes

Better answers

Brigitte draws on the Wikipedia definition of networked learning

Networked learning is a process of developing and maintaining connections with people and information, and communicating in such a way so as to support one another’s learning.

That’s a better answer (IMHO). No explicit mention of technology or common purpose. But of course there are alternatives and this remains a short description that doesn’t offer much detail. What are good and bad ways of developing and maintaining connections? What is a connection? What is its form? How might it be formed?

It’s in answering these types of questions where the variety between different interpretations of NGL enter the picture. Exploring these different interpretations and find one that works for them is one of the challenges for participants in the NGL course.

Putting it into practice

Formulating and justifying principles for action

A use the following definition of educational theory quite often because it resonates with my pragmatic view of theory. Hirst (2012) describes educational theory as

A domain of practical theory, concerned with formulating and justifying principles of action for a range of practical activities. (p. 3)

And that’s the aim of the NGL course, to encourage participants to draw upon their view of network learning to formulate and justify principles for action. Action that involves them planning some intervention into an act of teaching.

This post seeks to compare two different perspectives on network learning. One titled connected learning (getting a lot of traction and doing interesting stuff in the USA) and more European view of network learning. What’s interesting is that both appear to formulate principles for action.

It’s the formulation of principles for action that are based on an appropriate perspective of networked learning, and then using those principles to design a contextually appropriate intervention is the main focus of the last task in the course.

Is it worth it?

Adam isn’t alone when he expresses the following, related uncertainty

While I myself am a big enthusiast of implementation of ICT in education, I still haven’t convinced myself that online and distance curriculums actually offer learning advantages aside from flexibility and convenience

Indeed, this may be the big question for many people, but whenever people ask the “does it work” question with learning and teaching (with or without digital technologies), I am immediately put in mind of the following quote

That is why ‘what works’ is not the right question in education. Everything works somewhere, and nothing works everywhere. – Dylan Wiliam

A previous offering of NGL included a UK-based university educator teaching one of the sciences. Her definition of “what worked” was, not surprisingly, a very objective one. Either, NGL worked, or it didn’t work. And you could only know if it worked if there were double-blind, randomised, controlled trial. The gold standard for knowing if something works, or doesn’t work.

Along with Wiliam, I think education is much more difficult than that. It’s much more contextual. What works today, may not work tomorrow with the same learners.

Why is e-learning like teenage sex?

I’ve given a presentation that argues that almost all e-learning is like teenage sex. Not because I think that digital technologies cannot have any positive effect. But because I think the way that formal education institutions and the people within them understand and harness digital technologies remains extremely limited.

From this perspective, in this type of context, NGL is rarely going to provide advantages beyond flexibility and convenience.  Especially when the mindsets that underpin how formal education institutions do anything is stuck in a very non-NGL view. Which is what we argued with the BAD/SET framework, and where the D in BAD stood for Distribution and was defined as

the world is complex, dynamic, and consists of interdependent assemblages of diverse actors (human and not) connected via complex networks.

For me, network learning involved effectively recognising and leveraging that view of the world.

References

Hirst, P. H. (2012). Educational theory. In P. H. Hirst (Ed.), Educational Theory and Its Foundation Disciplines (pp. 3–29). Milton Park, UK: Routledge.

Putnam, R. T., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 4-15.

Exploring Moodle book usage – part 3 – who and how much

Continuing on from yesterday’s post this post seeks to explore a bit further how the Moodle Book module (the Book) is being used at my current institution.

The plan is that this post will explore:

  • What percentage of all courses are using the Book?
  • How big are the courses using the Book?

Follow up posts should look to explore

  • Compare the number of online students between courses using the Book and those that don’t.
  • Which staff are using/creating resources with the Book?
  • How many students actually use the Book – leading into questions of how they use it?
  • Exploring the content of the Books might also be useful.

Percentage of courses using the Book

Yesterday’s post showed the number of courses using the Book per year has grown from 17 in 2012 to 152 in 2015. To put that into perspective what percentage does that represent of all courses offered.

There’s a difficulty in calculating the percentages. I’m not entirely certain what makes an “official” course. The following is based on the criteria of being a course with at least one student enrolled, that also has a Moodle course site.

Year % courses using the Book
2012 2.02%
2013 3.01%
2014 6.36%
2015 10.35%

 How big are the courses using the Book?

Flowing from that, I’m wondering whether the courses using the Book are big courses or small courses?

The next graph shows the number of students per course, for each course using the Book. The following graph shows the number of student per course, for all courses. The aim is to see if there is any difference in the size of the courses using the Book.

The indication is that the courses using the Book tend to be larger than the typical course. Overall the median course has between 11 and 13 students. Compared to a median between 85 and 142 for the courses using the Book.

Possible implications might include:

  • The larger courses see the need to make use of the Book, perhaps as a source of information distribution.
  • The larger courses are more likely to have existing study guide material.
  • The larger courses are more likely to have significant numbers of online/distance education students.
  • The larger courses may be more likely to have assistance in using the Book.

Students per course using the Book     Students per all courses

 

Exploring Moodle Book usage – part 2 – overall use

This is the second in a series of posts exploring the usage of the Moodle Book module at my current institution. The first post gave some background and outlined an initial series of questions about Book usage that I’m aiming to explore. This post reports on initial findings related to the following

  • Correctly identify the number of course offerings using the Book each year.
  • How many books are being produced by each course?
  • How are they edited, in particular how many times they are edited.

The hope is that the next post will explore the following

  • Identify the number of different teaching staff are responsible for those courses.
  • Identify the type of courses using the Book.
  • How do the books fit into the structure of the course?

Note: Click on any of the following images to see larger versions.

Number of course offerings using the Book module

The following graph shows the number of different course offerings (my course – EDC3100 – is offered twice a year, so it’s counted twice) using the Book module each year.

It shows the growth in use of the Book that initially sparked my interest in this analysis. One of the reasons I was interested in this is due to the authoring process around the Book module being quite difficult to do a reasonable job. I put in place a range of additional support to make it meaningful, and I’ve observed other people give up on using the Book because it’s too hard. Raising the questions,

  • Did I just miss out on some simple way to author Book content?
  • Are the teaching staff in these courses out-sourcing the authoring of the Book content?
  • Is there some other driver that is encouraging them to overcome the hassle?

Annual Book usage

Number of Book resources per course

The Moodle Book module is used to produce individual resources (aka books or book resources). How many books did each course produce with the Book module? The following graph gives an answer.

Note: The next two graphs include data from 2016, but this only includes data up until February 2016. Hence the data really only shows results for a few early course offerings getting set up for first semester.

In the graph below the median number of books per course doesn’t exceed 3. The maximum – excluding outliers – gets no bigger than 22 books per course.

Books per course

Number of revisions per Book

Resources produced by the Moodle Book module can be edited and updated. The Moodle database keeps a simple counter revision that indicates the number of times such a resource has been updated.

My guess is that the growth in usage of the Book module at my institution has been driven by people moving the course study guides into the Book module. Previously these were available as stand-alone PDF documents, but the technology used to produce these was phased out. Since these PDF documents were generally not updated during semester, I was predicting that there would be a relatively low frequency of revisions to Book resources.

The graph below shows that the median number of revisions per book resource is no more than 2 (i.e. 50% of the books aren’t edited more than twice). The maximum (minus outliers) peaks at 45 and 32 in 2015 respectively.

It also shows a large number of books with 0 revisions. A quick check of the data reveals that the percentage of books that were never revised each year was:

  • 2012 – 72.9% from a low number of courses (less than 20)
  • 2013 – 33.2%
  • 2014 – 48.5%
  • 2015 – 46.3%

Revisions per course

Exploring Moodle Book Module usage – part 1 – background and planning

I’m due to have the slides for a Moodlemoot Australia presentation in a few weeks. Time to get organised. The following is (perhaps) the first of a sequence of posts reporting on progress toward that presentation and the related research.

Background

My interest in research is primarily driven by the observation that most educational usage of digital technology to enhance learning and teaching is fairly bad. Typically the blame for this gets laid at the feet of the teaching staff who are digitally illiterate, not qualified to teach, or are laggards. My belief/argument is that the problem really arises because the environment within formal education institutions just doesn’t understand what is required to make a difference. Much of what they do (e.g. institutional standards for course sites, checklists, training, support documentation, design and support of technlogies…) does little to help and tends to make the problem worse.

You want digitally fluent faculty?

A contributing factor to that is that institutional attempts to improve digital learning actually fails to be based on any insights on how people (in this case teaching staff and all those involved with digital learning) learn. How institutions implement digital learning actually gets in the way of people learning how to do it better.

Schema and the grammar of school

The ideas of schema and the grammar of school offer one example of this failure. This earlier post includes the following quote from Cavallo (2004) establishes the link

David Tyack and Larry Cuban postulated that there exists a grammar of school, which makes deviation from our embedded popular conception of school feel as nonsensical as an ungrammatical utterance [1]. They describe how reform efforts, whether good or bad, progressive or conservative, eventually are rejected or denatured and assimilated. Reform efforts are not attempted in the abstract, they are situated in a variety of social, cultural and historical contexts. They do not succeed or fail solely on the basis of the merit of the ideas about learning, but rather, they are viewed as successful based upon their effect on the system and culture as a whole. Thus, they also have sociological and institutional components — failure to attend to matters of systemic learning will facilitate the failure of the adoption of the reforms. (p. 96)

The grammar of school problem is linked to the idea of schema which links to the following quote that I first saw in Arthur (2009) and which is taken from Vaughan (1986, p. 71)

[In the situations we deal with as humans, we use] a frame of reference constructed from integrated sets of assumptions, expectations and experiences. Everything is perceived on the basis of this framework. The framework becomes self-confirming because, whenever we can, we tend to impost it on experiences and events, creating incidents and relationships that conform to it. And we tend to ignore, misperceive, or deny events that do not fit it. As a consequence, it generally leads us to what we are looking for. This frame of references is not easily altered or dismantled, because the way we tend to see the world is intimately linked to how we see and define ourselves in relation to the world. Thus, we have a vested interest in maintaining consistency because our own identity is at risk.

Evidence of schema in how digital technologies are used

Horsey, Horseless Carriage

The schema idea means that people will perceive and thus use digital technologies in ways that fit with their “integrated sets of assumptions, expectations and experiences”. This is an explanation for the horsey, horseless carriage way people respond to digital technologies. It’s why courses where the majority of students are online students and will never come onto a campus are still designed around the idea of face-to-face lectures and tutorials.

It also explains why when I finally returned to teaching a course I adopted the idea of a ramble for the structure of the course.  It explains why the implementation of the ramble evolved into using the Moodle Book module the way it does today. The images below (click on them to see larger versions) illustrate the connection between my practice 20 years apart, more detail follows.

1996 2016
The 85321 "online" book - 1996 Online book 2016

The 1996 image is a page from  the study guide (wonder how many people can play the au file containing the Wayne’s World II quote) for the Systems Administration course I taught in 1996. The 2016 image is a page from the “study guide” I developed for an Arts & Technologies C&P course.

I believe/suggest that the influence of schema also plays a significant contributor in the practice of other teaching staff as they transition into digital learning. It’s a factor in why most course sites remain dumping grounds for lecture slides and the subsequent widespread growth in the use of lecture capture systems.

And it’s not just the teaching staff. Students have developed schema about what it means to be taught, and what it means to be taught at university. A schema developed either through direct experience, or via the experience of others and various media. The typical schema for university education involved large lecture halls and tutorials.

 

So what?

The above suggests that whenever students and teachers engage with a digital technology (or any change around) and its use for learning and teaching, there are three main possibilities:

  1. It seen as nonsensical and rejected.
    e.g. whatever was said doesn’t make sense from existing grammar rules and seen as just being wrong.
  2. It sounds like something familiar and is modified to fit within the confines of that familiar practice.
    e.g. whatever was said sounds an awful lot like an existing use of grammar (even though it is different), and thus is interpreted as matching that existing use.
  3. The significant difference is seen as valued and existing practice is modified to make use of that difference.
    e.g. the different use of grammar is both understood as different and the difference is valued, and is subsequently existing practice is modified to incorporate the new grammar.

If this is the case, then examining the use (or not) of a digital technology in learning and teaching should reveal evidence of these possibilities.  This seems very likely, given widespread common complaints about the use of digital technology to enhance learning and teaching. Complains that see most practice stuck at possibility #2 (at best).

If this is the case, then perhaps this way of thinking might also identify how to address this.

But first, I’m interested in seeing if use of a particular digital technology matches this prediction.

Use of the Moodle Book module

Due to a 2015 grant from the USQ OpenTextbook Initiative I’m going explore the use the Moodle Book module. The plan is to analyse the use of the Moodle Book module (the Book) at USQ to see how both learners and teachers are engaging with it, see if the above expectations are met, and figure out what might be done in terms of the support and development of the Moodle Book module to help improve this.

What follows is an initial map of what I’ll be exploring.

A major aim here is to explore whether a student or teacher using the Book have made the transition from possibility #2 (treating the Book as a print-based book) to possibility #3 (recognising that this is an online book, and using that difference). I’ve highlighted some of the following questions/analysis, which I think be useful indicators of this transition. The darker the yellow highlight, the more strongly I think it might indicate someone making the leap to an online book.

Question for you: What other practices might indicate use that has moved from #2 to #3?

Which courses use the Book

First step is to explore whether the Book is being used. How many courses are using it? How many books are being produced with the module.

As the abstract for the talk suggests, early analysis revealed a growth in use, but I’m wondering how sound that analysis was. Hence there is a need to

  1. Correctly identify the number of course offerings using the Book each year.
  2. Identify the number of different teaching staff are responsible for those courses.
    Longer term, it would be useful to ask these staff about their background and reasons for using the Book.
  3. Identify the type of courses using the Book.
  4. How many books are being produced by each course?
  5. How do the books fit into the structure of the course?
    1. Is the structure the same from offering to offering?
    2. How much does the number and content of the books change from offering to offering?

Characteristics of the book content

  1. Statistics around the level of readability of the text (e.g. Flesch-Kincaid etc).
  2. The structure of the book – are sub-chapters used.
  3. Are images, video, Moodle activities included?
  4. What about links?
    • Are there any links at all?
    • What is linked to?
    • Are links purely to external resources? 
    • How many links connect back to other parts of the course’s Books?

Patterns in how the books are authored

  1. How are the books authored?
    • From scratch?
      1. Using the web interface?
      2. Via an import process?
    • Copied from previous offerings?
    • ?? other??
  2. How are they edited? 
    My expectation that a teacher who sees the Book as a replacement for a print book will not be editing the books during semester.

Patterns in how the books are read/used

  1. Are students reading the books online or printing them out?
  2. Does printing always happen at the start of semester? Does it continue through semester? Does it drop off?
  3. When are students reading the books?
  4. What is the nature of the paths they take through the books?
    1. Do they read the books and the chapters in order?
    2. How long do the spend on each chapter?
    3. Do they revisit particular books?
  5. How many times do discussion forum posts in a course include links to chapters/sub-chapters within the books
    • Posts written by teaching staff
    • Post written by students

References

Arthur, W. B. (2009). The Nature of Technology: what it is and how it evolves. New York, USA: Free Press.

Cavallo, D. (2004). Models of growth – Towards fundamental change in learning environments. BT Technology Journal, 22(4), 96–112.

How many digital devices do you have?

In a couple of the courses I teach I ask students (for slightly different purposes) the question from the title of this post, “How many digital devices do you have?”.  In one of the courses that question takes the form of a quiz and looks something like the following.

Question text

How many different digital technologies do you own?
Select one:
a. 0
b. 1 to 5
c. 6 to 10
d. 11 to 20
e. More than 20

 What answer would you give?

Count them up folks. What answer would you give. I’ll give you some space to think about that before talking about what some other folk have said.

 

 

What others have said

Some of the students in another course (where the question is framed somewhat differently) have offered the type of answers I expected, based on the framing of the question.

Jay identifies 3 devices. Neema lists 2.

Thinking a bit further afield than that I can probably count quite a few more than that in my house. I’ll ignore devices personal to other members of my family. This gets me the following list: laptop, 2 smart phones, digital camera, printer, various external drives, Apple TV device, T-Box, X-Box One.  That’s 9.

 

 

 But that doesn’t really start to count them

Fleming (2011) writes that it is

estimated that today’s well-equipped automobile uses more than 50 microcontroller units (p. 4)

Wikipedia defines a microcontroller as “a small computer) on a single integrated circuit containing a processor core, memory, and programmable input/output peripherals.

So your car alone potentially has you well into double figures. Remember that Fleming was writing in 2011. If you have recently purchased the latest Mercedes E-Class, chances are the number of microcontroller units in your car goes well beyond 50.

And of course, with your thinking re-calibrated by this example, you can probably quite easily identify additional devices in your house that are likely to control microcontrollers.

Implications

Digital devices are increasingly ubiquitous. Digital isn’t limited to a separate device like a computer, tablet, or smart phone. It’s wearable and in every thing.

I expect most people not to be aware of just how reliant they are on digital technologies in everything they do. Hence it’s uncertain that they understand or are prepared for what this might mean for what they do. For example, I don’t think many people in higher education or education more broadly quite understand the implications this has for how those organisations operate, perform, or exist. I’m not convinced that the patterns they use to make sense of the world are ready yet to deal with these changes effectively.

But then I’m not convinced the technologists are either.

Interesting times ahead.

References

Fleming, B. (2011). Microcontroller units in automobiles. IEEE Vehicular Technology Magazine, 6(3), 4–8. doi:10.1109/MVT.2011.941888

 

Teacher presence in network learning

A new semester and the Networked and Global Learning course is running again. Apologies to those in the other courses I teach, but this course is consistently the most engaging and interesting. It’s a course in which I typically learn as much as the other participants. However, due to the reasons/excuses outlined in the last post, I haven’t been able to engage as much as I would have liked with the course.

This has me thinking about something Adam wrote, in particular the argument/observation from Rovai (2002) which Adam describes as

This is bringing to light the sense of disconnection students are often experiencing due to physical and psychological separation from teachers, peers and institutions

What follows is some random reactions to this particular quote and an attempt to connect it with my teaching.

Badly designed learning generates bad outcomes

As someone who has been working, learning and teaching online for a long time I am biased and this idea troubles me. In fact, it puts me in mind of the following point made in this recent post around the question of banning laptops in the classroom, because handwriting is better for learning

Those studies about the wonders of handwriting all suffer from the same set of flaws, namely, a) that they don’t actually work with students who have been taught to use their laptops or devices for taking notes. That is, they all hand students devices and tell them to take notes in the same way they would in written form. In some cases those devices don’t have keyboards; in some cases they don’t provide software tools to use (there are some great ones, but doing it in say, Word, isn’t going to maximize the options digital spaces allow), in some cases the devices are not ones the students use themselves and with which they are comfortable. And b) the studies are almost always focused on learning in large lecture classes or classes in which the assessment of success is performance on a standardized (typically multiple-choice) test, not in the ways that many, many classes operate, and not a measure that many of us use in our own classes. And c) they don’t actually attempt to integrate the devices into the classes in question,

In terms of student disconnection,is it arising from there truly being something essential that a physical face-to-face learning experience provides that can’t be provided in an online space?

Or, is it because the types of online learning experiences being examined by Rovai have not been designed appropriate to draw on the affordances offered by an online learning environment?  Do these online learning experiences examined by Rovai suffer the same problem that most of the attempts to engage in open education illustrate? i.e. an inability to break out of the “persistent patterns of relations” (Bigum, 2012) that are formed by someone brought up teaching face-to-fact?

Given that the abstract for Roavi (2002) includes

Data were collected from 375 students enrolled in 28 different courses, offered for graduate credit via the Blackboard e-learning system by a private university

Indicating that the “persistent patterns of relations” under examination in this paper is from a North American university in 2000/2001 where online learning was limited to the Blackboard LMS. A time and system which is unlikely to be described by anyone as offering only the pinnacle of an online learning experience.

Might the sense of disconnection arise from the poor quality of the learning experience (online or otherwise) rather than the lack of physical presence.

Or is it simply that both teachers and learners have yet to figure out how to leverage the affordances of online learning?

What type of presence should a teacher have?

The following two images represent connections formed between participants in two discussion forums in a large course I teach (these are from first semester 2015). Each dot represents a participant. A red do is a teacher, blue dot a student. A connection between two people is formed when one of them replies to a post from the other.

This first image is from the general Question and Answers forum on the course site.

Forum network map The second image is from the Introduction and welcome forum, where students introduce themselves and say hi to someone the same and different. Screen Shot 2016-08-07 at 3.01.34 pm

In the first image, there is on red dot (me) that is strongly the center of all that’s going on. I’m answering questions. In the second image, the red dot that is me, is only lightly connected.

Which is better? Of course it depends. Which is scalable in an effective way?

The Equivalency Theorem suggests that as long as one of student-teacher, student-student, or student-content interaction is high, deep formal learning can occur. High levels of more than one and the educational experience will be more satisfying.

So far the NGL course has been suffering from low student-teacher interaction.  I wonder about the other two? Time will tell.

Teacher as meddler in the middle

A couple of years ago I wrote this post as an example of a “as teacher” post – a requirement for the NGL course. Not a lot has changed, and all this talk of interaction and connection has me thinking again of the first question I was interested in two years ago

How I can be a more effective “meddler in the middle”?

In particular, how can I be more aware of where the types of interactions students are having in the courses I teach, and subsequently what actions can I take to help strength as necessary? If I do this, what impact will it have on student learning and their experience?

I wonder if the paucity of methods for me to understand exactly how and interactions that are occurring that has me refining teaching materials. Materials that students may not be engaging with.  I’m hoping that this project will help reveal how and if students are engaging with the content in at least one course. Anecdotally, it appears that for many interaction with the content is little more than a box to tick. If borne out, this raises the question of how to get students to interact/engage effectively with the content.

There are similar questions to be explored around use of blogs and the connections between students….

References

Bigum, C. (2012). Edges , Exponentials and Education : Disenthralling the Digital. In L. Rowan & C. Bigum (Eds.), Transformative Approaches to New Technologies and student diversity in futures oriented classrooms: Future Proofing Education (pp. 29–43). Springer. doi:10.1007/978-94-007-2642-0

Rovai, A. (2002). Development of an instrument to measure classroom community. The Internet And Higher Education, 5(3), 197-211. http://dx.doi.org/10.1016/s1096-7516(02)00102-1

Open Educational Practice: the boring way and more interesting ways

Due to a combination of personal and institutional flaws (the information systems and enterprise processes within Universities continue to be “interesting”) that last month or so has been pretty horrendous from a work perspective. Only now starting to get some space to reflect on what’s been happening and engage more students and research. My apologies to the students who have been impacted.

One of the major tasks during this time has been designing learning experiences for the course EDM8006, Curriculum and Pedagogical Studies 2: Arts & Technologies. In particular, two weeks of learning experiences intended to introduce students to Australian Curriculum: Technologies learning area and its two subjects: Design and technologies, and Digital technologies. This is part of work that was connected with attempts earlier in the year to explore open educational practices (OEP) within initial teacher education (ITE).

That grant application was unsuccessful (largely due to me), however, another member of the group has been successful in getting a small grant that will restart that work over the coming months. The following is an attempt to get my head back into this space, reflect on the experience with the EDM8006 materials including making them “open” (the boring way), and suggest that there are more interesting ways to engage in OEP.

In part, the following also draws on the nascent model of the properties and nature of digital technologies initially proposed in this presentation. A version of that model (see image below) was actually used within the content I wrote for the EDM8006 course. It also wonders much further from what I’d originally intended to write.

A model of digital technologies

The boring way

Horsey, Horseless Carriage

The failed grant application from earlier this year included the following

But OEP has a horsey horseless carriage problem (Bigum, 2012). Most use of OEP is designed not to “disrupt the smooth running routines” (Bigum, 2012, p. 35) of existing educational practices and institutions. Open textbooks are still textbooks. Open courses are still courses.

And for me this remains the boring (and perhaps even wrong) way to think of open education resources/practice. i.e. spend the time and effort to polish course materials into a book and make that book openly available to all.

Cover of 3rd edition

Of course, it’s not really a “wrong” way, if it becomes a stepping stone toward doing something more interesting. The Zone of Proximal development and a range of other perspectives explain why it is difficult for any individual to make a huge leap from something they already know (and are expert in) to practices that are completely different. I know, I’ve been through that. The image to the right (with the penguin) is the cover from the open text book some colleagues and I wrote in the late 1990s. Creating an open text book might actually be a stepping stone to a less boring application of open, but I wonder if it can be for most people in most educational institutions.

For example, last year I had an interesting session where the institution brought in the “reference” panel for the Open Textbook grants scheme. That panel included a research professor who had just recently generated lots of publicity by producing an open textbook and a MOOC based on his teaching. Consequently, being “open” meant producing an open textbook using the same assumptions and practices he had used. Assumptions and practices that mirrored the work I’d done almost 20 years ago. Many others have identified and bemoaned the on-going prevalence of open textbooks in higher education. For example, Clint LaLonde’s post from late last year reflecting on OpenEd15 and includes a range of related observations, including (emphasis added)

Additionally, there is a divide as to whether open textbooks mark an entry point into open education for new people (and there was a massive number of people at OpenEd for the first time), or whether open textbooks are the beginning, middle and end of the open journey for some…

Problematically, textbooks are so deeply ingrained in our education systems that trying to find others ways of doing education for many is very difficult, especially in an education world where we continually remove capacity for those faculty who DO want to change and experiment and try different things. Rarely will you ever find a faculty member who says they have enough time to do their job, let alone undertake a radical overhaul of their pedagogy. Often faculty are p/t, or only brought in at the last minute to teach a course and grab at that teacher-proofed course-in-a-box

The last touches on Bigum’s (2012) argument that the “persistent patterns of relations that are performed in schooling” (p. 30) are limiting what can and can’t be done with digital technologies. Part of the argument being developed from this presentation is that educational institutions (and the individuals within them) not only aren’t aware of how these “persistent patterns of relations” are limiting what they do with digital technologies. They don’t even understand the nature of the very digital technologies that they are trying to harness.

Looking for evidence of these limitations is one of the reasons behind this work that is seeking to explore how and why people are using the Moodle Book module. My suspicion is that learners and teachers are seeing the Book module as a way to produce a print-based book. That they aren’t seeing it as a way to create a collection of web pages, nor are they seeking to leverage the affordances that medium offers. Arguably, the same problem applies at the institutional level.

A less boring way

It is in leveraging the affordances of web pages  that a less boring approach to open educational practices may lay. Early on in my use of the Book module I was trying to move beyond seeing it as a book. Even though my suspicion is that most students continued to print them out (something I’m hoping to test real soon now), and I suspect that most within the institution still see the Moodle Book as a way to produce books to be printed. For example, back in July 2012 I identified this as the 3rd problem from student feedback.

Both sets of students tended to mention the difficulty of finding that great idea or resource that was mentioned in one of the rambles that they’d seen previously.

Even though a solution to this problem had been developed in 2009, the institution didn’t have it installed. It wasn’t until July 2016 that it was installed.

In terms of making this content open, the slightly less boring way I’ve been exploring is the Moodle Open Book project. Modifying the Moodle Book module so that the content of books can be shared via GitHub. For example, this GitHub repository contains the “book” content I wrote for EDM8006. An approach that builds on the data homogenisation property (which is a link to one of the “books” hosted on GitHub) of digital technologies.

I’m not sure that this approach is going to have a great impact, for the following reasons.

  1. The technology involved (Moodle Books and my mod to connect to GitHub) aren’t that fantastic. Authoring for the Moodle Book module is a right pain and requires significant technical abilities. Using GitHub is also challenging, especially as integrated with the Moodle Book.
  2. A less than successful technical change is not likely to have any significant impact on the “persistent patterns of relations” around course content and courses themselves. Are other educators going to re-organise their patterns of operations to use the material I share on GitHub? I don’t think so, at the very least due to the first reason.
  3. The way these resources are written – embeded within the course site for EDM8006 – means that they have too much contextual information to be easily reused. This is Wiley’s reusability paradox and something that’s been touched on before. On the plus side, the use of GitHub at least promises the capability to “allow and enable for contextual modification”, but the integration with Moodle Book doesn’t yet do that well.
  4. The foundation of this approach is still the “expert” (me) crafting a path through the learning for other people to follow. I try to design it to encourage going off path, but I still design the path. Something touched on in work on customisable pathways design.

In particular, it’s an approach that separates learning from doing. Perhaps the fundamental “persistent pattern of relations” in education.  For a teacher (pre or in service) teacher to learn about using the Digital technologies subject, it is expected that they will come to the course/book, read about and perhaps do some stuff, and then take it back to the classroom where they will actually do something with their new learning.

Perhaps not surprising then that there is an apparent practice/theory divide in initial teacher education.

A more interesting way

The last activity in the “book” on data homogenisation from EDM8006 includes a link to this page titled “Digital technologies resources”. It contains a collection of resources produced by students in another similar course EDP4130 Technology Curriculum and Pedagogy. As part of an assessment task for the course, students are asked to engage in a project that has them

designing, developing, reviewing, and sharing resources to support implementation of the Australian Curriculum: Technologies in primary school classrooms

This task is an example of what Wiley calls renewable assessments, defined as

A “renewable assessment” differs in that the student’s work won’t be discarded at the end of the process, but will instead add value to the world in some way.

I can attest to the value that the resources produced by these pre-service teachers have added value to the world. The material I wrote for EDM8006 is so much better (I believe) because of the availability of these resources. I’ve been able to integrate use of those resources in EDM8006 and hopefully broaden the learning of the EDM8006 students. Time will tell.

Part of the problem I have is that the way in which I’ve integrated those resources into EDM8006 is fairly limited. First, because this is the first time I’ve developed material for this course and I did it quickly. Second, because of the digital tools I have access to reuse these resources are very limited.

Time will give me tie to learn and solve the first problem. I don’t believe time is going to solve the second problem.

An even more interesting way

Mike Caufield identifies one of the big problems with the technologies currently widely available for renewable assessment (and open educational practice in general) in the post – Why renewable assignments must be recyclable as well. With a particular focus on collaborative renewable assessments he identifies the following dichotomy

  • Small class sites (such as wikis) have a hard time bootstrapping to something useful, and even when they do get there they start to rot right after finals.
  • Large collaborative sites like Wikipedia make student work durable and provide a scaffold to build on, but require that the needs of the class bend to the needs of the site.

The context within which I’m working reveals related problems, including:

  1. The EDP4130 resources cannot be shared on an institutional system, because the institution doesn’t have any technology that could support this type of practice. The institutional folk are so focused on supporting the institutional systems (because it’s efficient and safe) that anything that doesn’t fit, can’t work.
  2. The EDP4130 resources are shared on a website (it provides links, it doesn’t host the resources) setup by the teacher involved. A teacher who is will be retiring soon. While the resources will likely continue to be available….
  3. The resources themselves are produced by students and placed on freely available websites like Weebly and Wix. I was slightly amazed that I didn’t find any that had already expired.
  4. All of this information is provided through web pages. It’s not available in a form that can leverage the capability of digital technologies to provide additional, context specific affordances that would actually help people use these resources in teaching. Some examples might include:
    • show a representation of all the content descriptions in the curriculum and how many resources have been produced for each;
    • create a path of resources that could form the basis for a unit plan or a year of teaching;
    • support version control on resources to allow people to update and modify the resources;
    • allow people to comment and evaluate resources.

If someone within the institution became aware of these problems (gaining awareness is the first stumbling block) the most likely solution would be similar to what Caufield identifies

The first impulse of people who haven’t lived through the past decade and a half of OER initiatives is “Wait, why don’t we just build a central site of student work!”. You don’t need federation at all, right? “You could make — a STUDENT WIKIPEDIA! Or, or, or — a central OER repository!”

Locally, I’m guessing the most likely suggestions wouldn’t be as foward thinking as a Student Wikipedia. It would be one of either of the following:

  1. get them all to use the institutional e-portfolio; or,
  2. worse yet get them to use the institutional learning object repository (Equella).

The first problem with these suggestions is that both of these systems are horrendous to use and aren’t designed to support the type of activity being envisaged here.

The second problem is that neither approach implements the type of federation that Caufield argues for in his post and elsewhere.

The third problem – and the one I think would be really interesting to solve – is that this type of approach does nothing to help embed the use, modification, and re-mixing of these resources into the what it is that teachers do from day to day.

Someone involved with teachers and teacher education might think that a good solution to this problem might be to integrate this type of assessment into Scootle or related services. There is some value to this idea, already Scootle is reasonably well integrated into the Australian Curriculum. But there remain (at least) two limitations with using Scootle

  1. It is not federated and thus suffers the problems identified by Caufield.
  2. It is still not connected (and perhaps not connectable) enough into what teachers (pre and in service) do.
    A simple example (there are more) would be a lesson/unit plan template that automatically links to sample resources that are relevant to the learning objectives I’ve just selected.

The last problem is related to the following quote from Norman.

I think it would be really interesting to design an environment that leverages the nature of digital technologies to make it easy for teachers (of all types) to engage in activities, help them do their job effectively, and enable them to learn and break out of the persistent pattern of relations that currently exist. And by the by, have them engage in open educational practices.
Norman on affordances

 

References

Bigum, C. (2012). Edges , Exponentials and Education : Disenthralling the Digital. In L. Rowan & C. Bigum (Eds.), Transformative Approaches to New Technologies and student diversity in futures oriented classrooms: Future Proofing Education (pp. 29–43). Springer. doi:10.1007/978-94-007-2642-0

Norman, D. A. (1993). Things that make us smart: defending human attributes in the age of the machine. Cambridge, Mass: Perseus. Reading, MA: Addison Wesley.

How and why do people use the Moodle Book module?

Below are collection of resources related to a presentation given at Moodle Moot’AU 2016, titled “How and why do people use the Moodle Book Module?”

The Graphs section below provides links to individual web pages that contain many of the charts included in the presentation slides below. The graphs are interactive. Roll the mouse over the graph to see some numbers, zoom in, pan around, etc.

Slides

Abstract

The Moodle Book module “makes it easy to create multi-page resources with a book-like format”. During the first half of 2012 the Moodle Book module was used in only 2 courses at the University of Southern Queensland (USQ). Those 2 courses contained 32 resources produced with the Book module. By the second half of 2015, 68 courses used the Book module to produce 396 resources. This presentation will report on some of the findings of an exploratory case study that is interested in answering the question: How and why do learners and teachers use the Moodle Book module?

The presentation will aim to

  • examine the characteristics (media used, readability, structure etc.) of the of content of the Book resources;
  • reveal the patterns and paths behind how authors and readers interacted with the Book resources;
  • suggest possible relationships between the content characteristics and usage by authors/readers;
  • identify factors that contributed to why teaching staff made the decision to adopt (or not); and, continue using (or not) the Book module;
  • outline some initial implications that these findings might have for the support and development of the Moodle Book; and,
  • invite alternate explanations, implications and suggestions for what was found and what might be done next.

 Pointers to graphs

The following provide links to web pages that contain charts etc that are used in the presentation.

How big are the courses?

What percentage of students are online?

When are the books used?

What percentage of the students are reading all of the Book

Misc other

 

Valuing the “residue of experience” a bit more

For a while now I have been drawing on the following quote from Riel and Polin (2004)

Over time, the residue of these experiences remains available to newcomers in the tools, tales, talk, and traditions of the group. In this way, the newcomers find a rich environment for learning. (p. 18)

to explain why I encourage/require the use of various types of social media (blogs, social bookmarking, feed readers) in my courses. This 2014 post identifies the problem (what happens in a course site, stays and dies in a course site) and how the social used in these courses helps address that problem.  If you do a Google search for edc3100 blog, you will get another illustration of how at least some of the residue of experience remains available to newcomers in at least one of the courses.

The problem is that this year has revealed that the design of the course doesn’t yet value that residue of experience, at least not in terms of the main value measure for many students – assessment. Students gain marks for writing blog posts that link to posts from other students, but the code that does this marking only recognises currently enrolled students. Linking to the broader residue of experience doesn’t count.

Interestingly, this has only become an issues this year. Only this year have students been asking why they missed out on marks for links to other (“old”) student posts. Leaving aside why it’s only started this year, this post documents the move to valuing the residue of experience.

After implementing the code below, it appears that at least 28 (about 25%) students this semester have linked to blog posts from students in previous offerings of the course. Would be interesting to explore this further. See how prevalent the practice has been in previous courses. Update these visualiations to show the connections between offerings.

What I need to do

The process will be

  • Refamiliarising myself with how the “valuing” is currently done.
  • Planning and implementing how to value the residue of experience.
  • Figuring out if/how to check how often the residue of experience has been used.

How it is currently valued

Some perl code does the work.  Details follow.

BlogStatistics class gathers all information about the blogs for students in the current course offering.  A method generateAllStatistics does some of the grunt work.

But this class also creates a data member MARKING for each student. Based on the Marking class and its GenerateStats method. This class gets the content from the bim_marking table (i.e. all the posts by the student).

GenerateStats accepts a reference to a hash that contains links to all the other blogs in the course (for the specific offering).  It calls DoTheLinks (gotta love the naming) passes it the hash ref to do the count.

One question is how much old data do I currently have?  Seems like there’s only the 2015 and 2016 data easily accessible.

Planning and implementation

One approach would be

  • BlogStatistics generates a list of old student blog URLs
    • add BlogStatistics::getOldStudentBlogs that creates $%BlogStatistics::OLD_BLOGS DONE
  • BlogStatistics passes this into each call to Marking::GenerateStats  DONE
  • Marking::GenerateStats would pass this onto Marking::DoTheLinks DONE
    • also increment POSTS_WITH_STUDENT_LINKS if a link is to an old student blog DONE
    • increment POSTS_WITH_OLD_STUDNET_LINKS if a link is to an old student blog DONE
  • Modify the report generator to show OLD links DONE

 

 

Planning changes to EDC3100 assignment 1

In the first half of the year there was a new assignment in EDC3100 designed to both enhance student learning, but also experiment with making the data produced by students and markers as part of the assessment process more accessible for manipulation by software. i.e. the students and markers entered data into a spreadsheet.

It’s a new semester, time to reflect on that initial use and see what changes should and can be made.

Student results

Let’s start with student results. (Note: this is all a bit rough and ready)

Overall the average mark for the assignment was 13.8 (72%) out of 19 with a standard deviation of around 3.  But that’s for both parts of the assignment.

Given current practice of using Word documents as assignment cover sheets, extracting out the specific marks for the checklist/spreadsheet assignment is difficult. But I have an Excel spreadsheet and I can run a script to get that data.

The average mark is about 9.5 (68%) out of 14, with a standard deviation around 2.

Let’s dig a bit deeper into the three criteria that made up that mark. The three criteria were

  1. Mark – students use a checklist to evaluate a lesson plan and its use of ICT and pedagogy.
  2. Acceptable use – focused on students ability to identify a lesson plan they can use wrt copyright.
  3. RAT – students use the RAT model to evaluate the use of ICT and pedagogy in the course

The following table compares cohort performance on the criteria and overall.

Criteria Average % stdev %
Overall 68 15.8
Mark 75.2 17.2
Acceptable Use 63.2 16.7
RAT 59.3 17.8

The RAT question was where the students were least successful.  It’s also (arguably) the more difficult question. The checklist was the highest mark.  Acceptable use is also quite low and needs some work.

Those last two is where the focus will go for now.

Other thoughts and experiences

Student feedback

Student feedback included the following comments related to the assignment

Some of the items we were required to assess in Assignment One could have been better explained

more guidance was required for Assignment 1. I didn’t like the use of the Excel document

 The last point was connected to the issue of not being able to justify the interpretation, which links back to points raised elsewhere. The first point is one to ponder. The results above suggest that’s not where the real need lays.

Marker feedback

Feedback from markers included

  • Identifying use of an IWB, when in fact it’s just being used as a data projector.
  • Little understanding of what constitutes: an authentic problem, and connections beyond the classroom
  • Some surprise that even with 50 assignments to mark, there were few double ups of lesson plans.
  • Another liked the format in that it gave students a better handle on what to look for in an ICT-rich lesson and the RAT model was useful for framing an evaluation.
  • The wording and nature of the statements for the acceptable use and the RAT question need to be clarified – to confusing (for marker and student)

One aspect of the assignment that troubled one of the markers was that the lesson chosen by the student only had to include some form of ICT.  It didn’t need to be rich nor effective ICT. This was actually one of the aims of the assignment, to allow students develop some appreciation for the breadth of what is possible and just how narrow use often is.

Questions asked during semester

  • Struggles to find a CC-licensed lesson plan.
  • Clarity about what makes an acceptable lesson plan
    • e.g. Can an American lesson be used?
    • Linked to concerns about Q10 and distinguishing between an appropriate lesson plan and whether or not you can use it due to copyright
  • Questions re: term of use and uploading
  • What if I can’t find any information about copyright?
  • How can/should the lesson plan be put online?
  • The distinction between what a student is using an ICT, and when the teacher is using it
  • Explanation of how the checklist questions are marked – e.g. those that don’t apply
  • Reporting bugs in the formatting of the cells

Personal thoughts

Early reflections on the semester included

The spreadsheet worked reasonably well. The checklist within the spreadsheet requires some refinement. As does some aspects of the rubric. The duplication of a Word-based coversheet needs to be removed.

 Other thoughts during the semester included:

  • Students had a tendency to treat the free text questions as requiring an essay.
  • The “pretend” context for the task wasn’t clear enough.
  • In particular, a problem about the exact legal status of ACME’s server, links and making copies of files.
  • Issues with specific questions and the checklist
    • The “web applications” option under “What is used” causing confusion about overlap with “web browser” question
    • Q16 includes mention of print material around ICT
    • Q26 mentions embedded hardware, a question of it and the connection with IWB
    • Appears to be strong connections between Q22 and A46
    • The purpose of Q10 is not clear enough, confusion with matching curriculum etc.
    • A feeling that there are too many questions and perhaps overlap
    • Criteria for RAT question isn’t clear enough about the quality of the response
      • e.g. not mentioning all uses of ICT and Pedagogy
      • Missing out on themes
      • Incorrect identifying something as belonging to a theme
    • Suggestion for a drop down box around linkage of ICT to objectives: not related, somewhat related, essential, extends, transforms
  • More explicit scaffolding/basic activities around the evaluation questions
    • e.g. Is ICT being used to Question, Scaffold, Lecture, in an authentic task

Random suggestions

Due to institutional constraints (not to mention time) none of the changes to be made can be radical.  Keeping with that, some initial suggested changes to explore include:

  1. Pre-submission checks
    1. What pre-submission checks should I run?
    2. Can they be run? How does that integrate with the Moodle assignment activity workflow?
  2. Remove the cover sheet entirely, just use the spreadsheet
    1. Need to include the learning journal mark into the spreadsheet
    2. Would be nice to do this automagically
  3. Tweaking the marking
    1. The criteria for Acceptable use and RAT questions need to be improved
    2. Look closely at each of the points about the questions
  4. Student preparation
    1. Make clear the need not to write essays for the free text questions
    2. Finding CC licensed lesson plans
      1. Great difficulty in finding those that are CC licensed
      2. Provide a list of prior sites people have used
      3. Generate some sort of activity to test understanding of CC with a specific example
    3. RAT Model
      1. More activities in learning paths
      2. Better labeling on  the spreadsheet
    4. More questions/activities around specific terms and concepts within the checklist