The following is the next part of the evaluation section of chapter 5 of my thesis. It seeks to evaluate how well the Webfuse system fulfilled the guideline of “an integrated online learning environment” for the period 2000-2004. This is only the first section which talks about the provision of course sites. Subsequent sections will talk about feature adoption, staff and student usage of course sites, and staff and student usage of Wf applications.
An integrated OLE
As described in Chapter 4 (see Table 4.3 and Section 4.3.2) a primary aim for Webfuse was to provide a single, integrated interface for all activities and resources required for learning and teaching. Work from 2000 through 2004, especially from 2001 with the expanded Infocom web team, was aimed at expanding on earlier work and addressing known limitations. Of the lessons learned from the 1997 through 1999 (Section 4.7) work on Webfuse, two are related to this design guideline:
- the need for better support for the concept of a course; and,
This need was address through the design and implementation of the default course site approach in 2001 (Section 5.3.5). This included the development of a number of new page types that offered specific course related services and the ability for the Infocom web team to automatically create course sites for all courses offered by Infocom.
- the need for better integration with the requirements & practice of staff and students.
The main mechanism for addressing this lesson was the adoption of an adopter-focused (Section 5.3.1) and emergent (Section 5.3.2) development process. A process aimed at rapidly addressing the needs of both staff and students within the one integrated system (Webfuse). This included both the idea of the default course sites but also the additional interactive web applications developed during this period.
The overall success of this work can be illustrated by a quote which Danaher, Luck and McConachie (2005) cite as coming from the Infocom annual report for 2003
[t]he best thing about teaching and learning in this faculty in 2003 would be the development of technologically progressive academic information systems that provide better service to our students and staff and make our teaching more effective. Webfuse and MyInfocom development has greatly assisted staff to cope with the complexities of delivering courses across a large multi-site operation.
The following sub-sections provide additional support for the success of work from 2000 through 2004 in implementing an integrated, online learning environment. The first three sub-sections match those used in Section 4.5.2 and enable a comparison between what happened in 1997 through 1999 and 2000 through 2004. These three sections examine: the number of course sites created; the spread of feature adoption within those course sites; and, how much the course sites were used by staff and students. The final sub-section covers usage of the interactive web applications that were first implemented in 2000.
The aim of this sub-section is to briefly comment on the number of course sites that were created. From the start of Webfuse, all courses offered by the faculty had course websites created. This practice continued with the default course sites, the major difference is that the actual creation of the course sites was automated. While the creation of default course sites was entirely automated, the gathering of all the necessary data from various institutional sources in order to create the course sites was not entirely automated. Consequently, the resources required were not minimised as much as possible. This problem of limited integration is picked up again in section 5.4.3.
Before the default course site approach was introduced there were always a small number of course sites created without using Webfuse, this practice also continued afterwards in terms of the real course sites. Table 5.8 shows the details of the number of course sites created in Webfuse from 1997 through 2009. It also shows the number and percentage of courses that were created outside of Webfuse.
|Year||All courses|| Real course
|% Real course sites|
During 2000 through 2003 all of the non-Infocom faculties at CQU used the institutionally supported LMS, WebCT. At the end of 2003, when CQU was moving to Blackboard as its institutionally support LMS, there were 141 WebCT courses to be migrated to Blackboard. WebCT had been the official institutional LMS at CQU since 1999 and was primarily used for courses other than those offered by Infocom.
Table 5.9 offers a comparison between Infocom and the non-Infocom parts of CQU in terms of course sites, courses and student/courses during 2003. The “# Course Sites” and “# Courses” columns in Table 5.9 shows the number of unique courses or course websites. A number of CQU courses are offered more than once a year, each offering is not shown in Table 5.9. For example, for the 940 non-Infocom courses offered in 2003, there were a total of 1322 course offerings. Lastly, the “# Courses” column in Table 5.9 only shows those courses for which students enrolled.
|Faculty||# Course sites||# Courses||# Student/Courses|
Table 5.9 shows that only about 15% of the non-Infocom courses had course websites, while all Infocom courses had course websites on WebCT. In the same year, Table 5.9 shows that there were more Webfuse course sites than there were Infocom courses. The additional courses come from a number of sources including courses: with no enrolments in 2003; classed as non-Infocom, but taught primarily to Infocom students; and a single test course used during the evaluation process that selected Blackboard as the institutional LMS. So, by 2003 all Infocom courses would have a default course site, while little more than 15% of non-Infocom courses had a course website.
Due to the earlier Webfuse practice of manually creating course websites, the introduction of the default course site approach did not increase the percentage of Infocom courses with course websites. However, it did expand the default level of information and services available through those course sites. In the last term before implementation of the default course site approach, a standard Webfuse course site provided a course synopsis, a link to the course profile, details about course coordinator, and if used, a web-based archive of a mailing list. The default course site design described in Section 5.3.5 added a range of additional information (e.g. assessment item details) and services (e.g. each course had a course barometer). A manually created default site had a minimum of three pages, a default course site had a minimum of 10. In both types of course sites, academics could then choose to add further to the initial course website.
Table 5.10 shows for all Webfuse course sites from 1997 through 2009, the number of Webfuse courses, the number of Webfuse pages for those sites, the number of Webfuse page types used, and an average number of pages per course. From the introduction of the default course sites in the second half of 2001 there is a steady increase in the average number of pages per course from 22.7 to 30.8. However, there are contextual factors to consider before reading too much into the figures from Table 5.10. These factors are discussed below and include:
- the impact of two Webfuse courses created by the Webfuse designer;
- changes in the make up of the default course sites in 2004; and
- related organisational changes in 2004.
|Year||Webfuse courses||Number of Webfuse pages||Pages/Course|
As described in Section 4.5.5 two courses designed by the Webfuse creator were significantly different than the standard Webfuse course. For example, in 1999 these two courses accounted for 47.2% of hits on course site pages and 75.4% of page updates. In 1997, just one of these courses accounted for 63.8% of the total pages for Webfuse courses. From 2000 through 2004, the creation of web-based archives of course mailing lists was done using Webfuse page types. In the last term of 2003, 44% of Webfuse pages were web-based archives. In 2004, CQU’s central IT division introduced a centralised service for Web-based archives of mailing lists. Consequently, the default course site was modifed to use this central service. Similarly, a small number of courses used the Lecture page type to create online lectures. The Lecture page type would create one LectureSlide page per slide in a lecture. In the first term of 2001, the LectureSlide page type accounted for 44% of page types. Table 5.12 is a “normalised” version of Table 5.10. It shows the same data as Table 5.10 modified to reduce the impact of these unusual factors.
|Year||Webfuse courses||Number of Webfuse pages||Pages/Course|
The larger page per course ratio for 1997 shown in Table 5.12 was caused by a larger, manually created default site being used for the first year of Webfuse’s operation. To reduce workload, the default course site that was manually created from 1998 through the first term in 2001, was somewhat reduced in the number of pages. As shown in Table 5.12 the introduction of the default course site approach in 2001 saw almost a doubling of the pages per course ratio. In 2002, the first full year’s operation of the default course site there is another 25% increase in the pages per course ratio. This level is essentially maintained for the following seven years with some minor fluctuations. Even though support and development of the default course site idea was significantly reduced after 2004. The fluctuations in the pages per course ratio after 2002 generally arise from increased or decreased modifications on top of the default course site by academics. This behaviour is examined in more detail in a subsequent section.
In summary, the default course site approach provided an expanded range of services and functionality over the previously manually created default course site approach without the need for significant amounts of manual editing of web pages and the subsequent increased chance of human error. As a result, by 2003 all of Infocom’s courses had course websites compared to only about 15% of courses offered by the rest of CQU. The next sub-sections examine the features available within those websites and how much they were used by students and staff.
Danaher, P. A., J. Luck, et al. (2005). “The stories that documents tell: Changing technology options from Blackboard, Webfuse and the Content Management System at Central Queensland University.” Studies in Learning, Evaluation, Innovation and Development 2(1): 34-43.