Augmenting Monitoring Performance during Multi-UAV Supervisory Control with Adaptive Displays

Unmanned aerial systems have become a main sector to explore and further push
the boundaries of automation. In contrast to conventional, manned aircraft, unmanned
aerial vehicles (UAV) are controlled by human operators from dislocated
control stations. Possible scenarios for using UAV instead of manned aircraft are
of both, civil and military nature, including surveillance, reconnaissance, agricultural
imaging, border patrol, construction inspection, and firefighting. The
main advantage is the reduction of risk for human personnel, which results from
operating in hazardous terrain or conflict areas. Today, a single aircraft-size
UAV typically requires control through several human operators via dedicated
human-machine interfaces. However, research on future architectures targets on
inverting the human-UAV ratio, allowing one human operator to control many
UAV simultaneously.
Because human cognitive and physical resources are limited by nature, the
key technology allowing one human operator to safely control many UAV is automation.
The human operator mainly plays the role of a supervisor. As a
supervisor, the human operator is the “ultima ratio” in abnormal situations that
cannot be handled by the automation. The underlying human-machine interaction
paradigm is commonly known as human supervisory control. Within this
paradigm building and maintaining sufficient situation awareness is very important.
However, human factors research revealed that insufficient situation awareness
is the main cause of human error in context of human supervisory control.
The key enabler of good situation awareness is adequate monitoring behavior.
Therefore, it is a safety-critical requirement that displays are designed in a way
that supports human operators to selectively perceive the information they need.
However, human factors research again showed that monitoring behavior is very
often inadequate, e. g., because of distraction, perseveration or fatigue.
In this work, we present an assistant system to improve the monitoring behavior
of a human operator in charge of supervisory control of multiple highlyautomated
unmanned aerial vehicles. The underlying concept is based on the
real-time assessment and augmentation of monitoring performance. The realtime
assessment of monitoring performance is based on an eye tracking-based
tool that allows to assess the demand for attention of information conveyed by
a user interface for supervisory control of multiple highly-automated UAV. The
real-time augmentation of monitoring performance is based on two strategies to
adapt the characteristics of the user interface used by a human operator. The
first strategy invokes visual cues on a situation awareness display to support visual
search for information demanding attention. The second strategy invokes
visual cues on a peripheral display to support the human operator’s awareness
for monitoring behavior adequacy. Our results indicate that assisting the monitoring
task using visual cue invocation improves monitoring performance and
situation awareness.