Principal Investigator: Lisa Anthony

Sponsor: National Science Foundation

Start Date: January 1, 2016

End Date: December 31, 2020

Amount: $501,582

Abstract

Natural user interfaces allow users to interact with technology through modalities like touch, gesture, and motion. They are a key element in realizing the vision of ubiquitous computing, yet they present challenges with respect to supporting children. The PI’s research on touchscreen interaction for children has found that existing surface gesture recognition algorithms designed, trained, and tested on adult input, and interaction design guidelines developed based on adult interaction patterns, do not apply equally well to children. For example, if a typical gesture interface is expecting a gesture to be entered as a single stroke, the system will not be able to process the multiple strokes generated by a child, leading to an unsuccessful interaction for that child. Recognition for whole-body interaction gestures experiences similar challenges; a child is more likely to perform an action or gesture (e.g., “”jump”” or “”wave””) with greater intensity or different motion paths than an adult performing the same gesture. In this research the PI’s goal is to fundamentally advance our understanding of how to design and develop natural user interactions for children. The research will be carried out in three phases: (1) Data Collection and Analysis: collection of input behaviors from elementary-school aged children in each modality, and analysis for patterns and characteristics of children’s input; (2) Recognition and Classification: development of new recognition algorithms attuned to the expected input behavior patterns of children and use of machine learning to evaluate their performance; and (3) Multimodal Interaction: investigation of multimodal input patterns exhibited by children, and validation of new approaches to multimodal synthesis that perform well on children’s natural input. A testbed application will be developed to showcase the findings in an educational domain. Open-source natural user interaction recognition and synthesis algorithms and clear, practicable design recommendations will be developed and released in both peer-reviewed papers and on the project website for use by researchers and practitioners.