HEAD-MOUNTED EYE-TRACKING TECHNOLOGY AND AUGMENTED REALITY: MULTI-MODAL ANALYTIC SYSTEMS AND APPLICATIONS
Wan, Qianwen.
2019
-
Today's autonomous
recognition systems are challenged by the many facts of the multi-modal nature of the
data. They can involve images, video streams, audio and a variety of sensor devices
including thermal, 3d imaging as well as eye-trackers, and virtual and augmented reality
devices. Recognition systems are predominantly known for their significant contributions
to advance security and ... read morebiomedical applications, however many research fields still tend
to use humans to annotate, evaluate and analyze massive datasets. In this thesis, the
research focuses on leveraging image characteristics based feature extraction
algorithms, statistical and machine learning classification methods and human visual
system inspired methods to achieve robust and efficient recognition and classification
systems across different modalities. This includes human visual system inspired image
enhancement and de-noising processing for data captured under varying illumination
conditions and using diverse sensing devices. While adapting the advancements of current
state-of-the-art deep-learning technologies, the knowledge gained from single image
physical characteristics and classification methodologies are applied for developing
automated solutions for massive eye-tracking data analysis, which eliminates the need
for cost-prohibitive and time-consuming manual annotation in cognitive research.
Accordingly, image quality measurement, enhancement, colorization and segmentation
algorithms are presented as contributions to refine the applicability of the proposed
automated solutions. To further aid in effectively analyzing multi-modal data and
promptly providing feedback for human behavior research exploring complex cognitive
processes, novel software solutions with stand-alone graphic user interfaces are
proposed utilizing transfer-learning object classification, voice activity detection,
speech prosody analysis, natural language processing, and multi-stream data fusion. The
proposed engineering solutions have been successfully adapted by cognitive psychologists
and shown its exceptional performance in analyzing real-life mobile eye-tracking case
studies. Experimental results illustrate the game-changing potential of the proposed
systems for increasing usefulness and ecological validity of using eye-tracking
technology in conducting human behavior research and validating hypothesis. Meanwhile,
the applied autonomous systems are successfully deployed on mobile and head-mounted
augmented reality platforms for applications such as vision-based indoor navigation,
interior decoration, tourism, entertainment, facial emotion analysis, aerial border
surveillance for search and rescue missions, universal accessibility, clinical and
medical support.
Thesis (Ph.D.)--Tufts University, 2019.
Submitted to the Dept. of Electrical Engineering.
Advisor: Karen Panetta.
Committee: Sos Agaian, Ronald Lasser, Holly Taylor, and Karen Panetta.
Keyword: Electrical engineering.read less - ID:
- c821gx785
- To Cite:
- TARC Citation Guide EndNote
