1st Edition

Interaction Design for 3D User Interfaces The World of Modern Input Devices for Research, Applications, and Game Development

    804 Pages 24 Color & 261 B/W Illustrations
    by A K Peters/CRC Press

    804 Pages 24 Color & 261 B/W Illustrations
    by A K Peters/CRC Press

    In this new era of computing, where the iPhone, iPad, Xbox Kinect, and similar devices have changed the way to interact with computers, many questions have risen about how modern input devices can be used for a more intuitive user interaction. Interaction Design for 3D User Interfaces: The World of Modern Input Devices for Research, Applications, and Game Development addresses this paradigm shift by looking at user interfaces from an input perspective.

    The book is divided into four parts:

    • Theory of input devices and user interfaces, with an emphasis on multi-touch interaction
    • Advanced topics on reducing noise on input devices using Kalman Filters
    • A collection of hands-on approaches that allows the reader to gain experience with some devices
    • A case study examining speech as input

    Most of the chapters contain exercises that provide practical experience to enhance knowledge of the material in the related chapter. With its hands-on approach and the affordability of the required hardware, this book is an excellent flexible resource for both the novice and the expert in 3D user input device development. Researchers and practitioners will gain a much deeper understanding about user input devices and user interfaces. Game developers and software designers will find new techniques to improve their products by adding intuitive user interaction mechanisms to their games and applications. In addition to the resources provided in the book, its companion website, http://3DInputBook.com, provides additional resources, which include: additional exercises and project ideas, additional chapters, source code, and class instructors’ resources. The additional resources are provided to keep helping you with new research and new technology as it becomes available to help you stay up to date.

    THEORY

    Introduction
    The Vision
    Human–Computer Interaction
    Definitions
    Further Reading

    Input Interfaces
    Introduction
    Input Technologies
    User Input Interfaces
    Input Devices
    Input Recognition
    Virtual Devices
    Input Taxonomies
    Further Reading

    Output Interfaces and Displays
    3D Output Interfaces
    Displays
    Further Reading

    Computer Graphics
    Computer Graphics
    Further Reading

    3D Interaction
    Introduction
    3D Manipulation
    Further Reading

    3D Navigation
    3D Travel
    Wayfinding
    3D Navigation: User Studies
    Further Reading

    Descriptive and Predictive Models
    Introduction
    Predictive Models
    Descriptive Models
    Further Reading

    Multi-Touch
    Introduction
    Hardware
    Multi-Touch and Its Applications
    Figures of Large TabletTop Displays
    Further Reading

    Multi-Touch for Stereoscopic Displays
    Understanding 3D Touch
    Touching Parallaxes
    Multi-Touch Above the Tabletop
    Interaction with Virtual Shadows
    Perceptual Illusions for 3D Touch Interaction

    Pen and Multi-Touch Modeling and Recognition
    Introduction
    The Dollar Family
    Proton++ and More
    FETOUCH

    Using Multi-Touch with PetriNets
    Background
    PeNTa: Petri Nets
    Further Reading

    Eye Gaze Tracking as Input in Human–Computer Interaction
    Principle of Operation
    Post-Processing of POG Data: Fixation Identification
    Emerging Uses of EGT in HCI: Affective Sensing
    Further Reading

    Brain–Computer Interfaces: Considerations for the Next Frontier in Interactive Graphics and Games
    Frances Lucretia Van Scoy
    Introduction
    Neuroscience Research
    Implications of EEG and fMRI-Based Research for the Brain–Computer Interface
    Neuroheadsets
    A Simple Approach to Recognizing Specific Brain Activities Using Low-End Neuroheadsets and Simple Clustering Techniques
    Evidence of Feasibility of Using EEG Data to Recognize Active Brain Regions
    Conclusion
    For Further Reading

    ADVANCED TOPICS

    Math for 3D Input
    Steven P. Landers and David Rieksts
    Introduction
    Axis Conventions
    Vectors
    Matrices
    Axis Angle Rotations
    Two Vector Orientation
    Calibration of Three Axis Sensors
    Smoothing
    Further Reading

    Introduction to Digital Signal Processing
    Introduction
    What Is a Signal?
    Classification of Signals
    Applications of Digital Signal Processing
    Noise
    Signal Energy and Power
    Mathematical Representation of Elementary Signals
    Sampling Theorem
    Nyquist–Shannon Theorem
    Aliasing
    Quantization
    Fourier Analysis
    Fast Fourier Transform
    z-Transform
    Convolution
    Further Reading

    Three Dimensional Rotations
    Introduction
    Three Dimensional Rotation
    Coordinate Systems
    Euler Angles
    Quaternions
    Further Reading

    MEMS Inertial Sensors and Magnetic Sensors
    Introduction
    Inertial Sensors
    MEMS Inertial Sensor Errors
    Magnetometers
    MEMS Magnetometer Errors
    Further Reading

    Kalman Filters
    Introduction
    Least Squares Estimator
    Kalman Filter
    Discrete Kalman Filter
    Extended Kalman Filter
    Further Reading

    Quaternions and Sensor Fusion
    Introduction
    Quaternion-Based Kalman Filter
    Quaternion-Based Extended Kalman Filter
    Conversion between Euler and Quaternion
    Further Reading

    HANDS-ON

    Hands-On: Inertial Sensors for 3D Input
    Paul W. Yost
    Introduction
    Motion Sensing and Motion Capture
    Types of Motion Sensing Technology
    Inertial Sensor Configurations for Input
    Hands-On: YEI 3-Space Sensors
    Hands-On: YEI Prio for Whole-Body Input
    Further Reading

    Simple Hands-On Project with Unity 3D and Oculus Rift
    Nonnarit O-larnnithipong
    Installation and System Requirements
    Getting Started
    Creating Game Scene
    Lighting, Camera and Skybox
    GameObject and Basic Action Script
    Graphic User Interface (GUI)
    Oculus Rift Integration for Unity
    Further Reading

    Hands-On Approach with Leap Motion
    Frank E. Hernandez
    What Is Leap Motion
    Installation
    Hands-On Mini-Project
    Further Reading

    Hands-On Approach with Kinect Sensor v2
    Frank E. Hernandez
    What Is the Kinect Sensor
    Installation
    Hands-On Mini-Project
    Further Reading

    Creating Home-Brew Devices with Arduino Microcontrollers
    Sudarat Tangnimitchok
    Microcontroller
    Analog Sensor
    Serial Communication
    Hands-On Project: Ultrasonic Proximity Sensor

    Autonomous Bicycle with Gyroscope Sensor
    Panuwat Janwattanapong and Mercedes Cabrerizo
    Introduction
    AU Self-Balancing Bicycle (AUSB)
    Data Processing
    System Implementation and Results
    Conclusion
    Further Reading
    Exercise

    Input Implementation Details
    Input Devices
    Multi-Touch Implementation
    Working with a 3D Graphics Engine: OGRE
    ECHoSS: Experiment Module
    Further Reading

    CASE STUDY: SPEECH AS INPUT

    Multimodal Human-Like Conversational Interfaces
    Ugan Yasavur and Christine Lisetti
    Dialogue Management Overview
    Dialogue Management in Health Dialogue Systems
    Task-Based Spoken Dialog Systems
    Embodied Conversational Agents
    Brief Interventions for Alcohol Problems
    Conclusion

    Adaptive Dialogue Systems for Health
    Ugan Yasavur and Christine Lisetti
    Approach
    Reinforcement Learning Background
    Markov Decision Processes
    Modeling World with Interconnected MDPs
    Agent and Dialogue Strategy Learning
    Reward Function Design
    Speech Recognition and Language Model
    Dialog Corpus
    Conclusion

    CONTENTS OF ONLINE SUPPLEMENTARY MATERIAL

    The World of Gesture Recognition

    Input Frameworks and Development Considerations for Input Technologies

    Hands-On: Multi-Touch and Pen Using Microsoft Windows 8 and Windows 10 with Microsoft Surface Pro

    Hands-On: Intel RealSense SDK.

    APPENDICES

    Displays
    Jorge Chernicharo
    Fixed Displays
    Portable Displays
    Hybrid Systems

    Creating Your Own Virtual Reality Headset
    Karrel Muller
    Introduction
    Google Cardboard

    Biography

    Francisco R. Ortega, PhD, is a postdoctorate research fellow at Florida International University, Miami, where he received his PhD in computer science. He is the current director of the Open Human-Interface Device Laboratory at Florida International University (http://openhid.com). He was a member of the Digital Signal Processing Laboratory at FIU, and has over 17 years of experience in software development and systems integration. His interests are in 3D user interfaces, input devices, human–computer interaction, 3D navigation, and input modeling, among other interests. He has multiple publications in journals, lecture notes, and conference proceedings.

    Fatemeh Abyarjoo, PhD, received her PhD in Electrical engineering from Florida International University, Miami, where she was also a research assistant in the Digital Signal Processing Laboratory, focusing on sensor fusion for human motion tracking. She is currently a Fraud Risk Data Scientist, focusing on financial data analyzing. Her research interests are data mining, data analysis, statistical modeling, sensor fusion and wearable devices. She is a former Open Science Data Cloud PIRE National Science Foundation Fellow.

    Armando Barreto, PhD, is a faculty member of the Electrical and Computer Engineering Department at Florida International University, Miami, as well as the director of FIU’s Digital Signal Processing Laboratory. He earned his PhD in electrical engineering from the University of Florida, Gainesville. His work has focused on applying DSP techniques to the facilitation of human–computer interactions, particularly for the benefit of individuals with disabilities. He has developed human–computer interfaces based on the processing of signals and has developed a system that adds spatialized sounds to the icons in a computer interface to facilitate access by individuals with "low vision." He is a senior member of the Institute of Electrical and Electronics Engineers and the Association for Computing Machinery.

    Naphtali Rishe, PhD, is Eminent Chair Professor of Computer Science at Florida International University, Miami. He has authored three books on database design and geography and has edited five books on database management and high performance computing. He holds four US patents on database querying, semantic database performance, Internet data extraction, and computer medicine. He has also authored 300 papers in journals and proceedings on databases, software engineering, Geographic Information Systems, Internet, and life sciences. His TerraFly project—a 50-terabyte database of aerial imagery and Web-based GIS—has been extensively covered by worldwide press.

    Malek Adjouadi, PhD, is a professor with the Department of Electrical and Computer Engineering at Florida International University, Miami. He received his PhD from the Electrical Engineering Department at The University of Florida, Gainesville. He is the founding director of the Center for Advanced Technology and Education funded by the National Science Foundation. His earlier work on computer vision to help persons with blindness led to his testimony to the US Senate on the committee of Veterans Affairs on the subject of technology to help persons with disabilities. His research interests are in image and signal processing with applications in neuroscience and assistive technology research.