The following is a summary and probably some thoughts on
Endsley, M. (2001). Designing for Situation Awareness in Complex System. In Proceedings of the Second International Workshop on the symbiosis of humans, artifacts and environment. Kyoto, Japan.
@beerc is excited by some of the potential of this and related work for university e-learning. It seems to fit with our thoughts that most universities and the individuals therein aren’t even scratching the surface in terms of what technology could offer.
My initial thoughts
I like some of the initial outlining of the problem, however, I think the solution smacks too much of the complicated systems approach, rather than the complex adaptive approach. The solution is essentially a specialised waterfall model (requirements/design/evaluate) with a focus on situation awareness. There’s some interesting stuff there, but I’m not sure how applicable it is to university e-learning. I remain leery of this idea that experts can come in an analyse the problem and fix it. It needs to be more evolutionary.
There are some nice quotes for higher ed and its systems.
The challenge of the information age
“The problem is no longer lack of information, but finding what is needed when it is needed.” Actually, I’d have to argue that when it comes to information about the learners in the courses I teach, I’m still suffering the former problem when it should be the latter.
Describes “The information gap” where “more data != more information”. Draws on some of the common explanations.
From data to information
Draws on a Bennis (1977) quote “This post-technological age has been defined as one in which only those who have the right information, the strategic knowledge, and the handy facts can make it”….makes the point “making the right decisions will depend on having a good grasp of the true picture of the situation”.
The overflow of data needs translating into information. But it will “need to be processed and interpreted slightly differently by different individuals, each of whom has varied dynamically changing but inter-related information needs”.
This translation “depends on understanding how peopel process and utilize information in the decision making activities”
Understanding “human error”
A few examples of this before “the biggest challenge within most industries and the most likely cause of an accident receives teh label of human error. This is a most misleading term, however, that has done much to sweep the real problems under the rug.”
Instead it’s argued that the human was “striving against significant challenges”….coping “with hugely demanding complex systems”. Overload in terms of data and technology. This is addressed through long lists of procedures and checklists which are apt to eventually fail. Instead
The human being is not the cause of these errors, but the final dumping ground for the inherent problems and difficulties in the technologiges we have created. The operator is usually the one who must bring it all together and overcome whatever failures and inefficiencies exist in the system
This resonates quite strongly with my experience at different universities when trying to teach a large course with the provided information systems.
Situation awareness: The key to providing information
“Developing and maintaining a high level of situation awareness is the most difficult part of many jobs”. SA is defined as “an internalised mental model of the current state of the operator’s environment…..This integrated picture forms the central organising feature from which all decision making and action takes place”.
Developing and keeping SA up to date makes up a vast portion of the person’s job.
“The key to coping in the ‘information age’ is developing systems that support this process. This is where our current technologies have left human operators the most vulnerable to error.”…..cites research that shows SA problems were “the leading causal factor”.
I wonder if such research could be done in a contemporary university setting?
Success will come is you can combine and present the vast amounts of data in a way that provides SA. “The key here is in understanding that true situation awareness only exists in the mind of the human operator”.
This is an interesting point given the rush to automated analytics.
The successful improvement of SA through design or training problems requires the guidance of a clear understanding of SA requirements in the domain, the individual, system and environmental factors that affect SA, and a design process that specifically addresses SA in a systematic fashion
SA defined
Citing Endsley (1988) SA is defined as
the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future”
and there a levels of SA
- Perception of the elements in the environment
“perceiving the status, attributes and dynamics of relevant elements in the environment”
- Comprehension of the current situation
More than awareness of the elements, includes “an understanding of the significance of those elements in light of one’s goals”.
A novice operator may achieve the same Level 1 SA as an experienced operator, but will likely fall short at Level 2.
- Projection of future status
What are the elements in the environment going to do?
Theoretical underpinnings
Links to broader literature that has developed a theoretical framework model. Apparently heavily based on the cognitivist/psychology research. Working memory, long term memory etc. e.g. Fracker’s (1987) hypothesis that working memory is the main bottleneck for situational awareness and other perspectives. Mental models/schema get a mention as a solution.
“Of prime importance is that this process can be almost instantaneous due to the superior abilities of human pattern matching mechanisms”. Hence the importance of expertise and experience.
Designing for situation awareness enhancement
The type of systems integration required for SA
usually requires very unique combinations of information and portrayals of information that go far beyond the black box “technology oriented” approaches of the past
Designing these systems is complex, but progress made. Too complex to cover here, but talks about three major steps
- SA requirements analysis
Frequently done with a form of cognitive task analysis/goal-directed task analysis. The point is that goals/objectives form the focus, NOT tasks.
Done using a combination of cognitive engineering procedures with a number of operators.
Done (with references) in many domains.
- SA-Oriented design
Presents 6 design principles for SA, that is also applicable more broadly.
- Measurement of SA in design evaluation
Mentions the Situation Awareness Global Assessment Technique (SAGAT) measuring operator SA.
When initially reading those three steps my first reaction was “Arggh, it’s the SDLC/waterfall model all over again. That’s extremely disappointing”. I then started wondering if this was because they are thinking of complicated systems, not complex adaptive systems?
