Touchscreen devices are widely used and seem to be the trend for future generations of smartphones. Although touch interfaces are inherently visually demanding, previous research has leveraged touchscreens to provide new input methods for blind users. Particularly, multitouch Braille-based text-entry techniques enable non-visual input using chording actions. However, there is not a non-visual multipoint output method that enables a dialog between the device and the user. Both auditory feedback and single vibrations can inform that a touch occurred; yet, feedback is non-local and usually undirected, failing to provide chording information. Thus, an output channel that actuates directly on users’ fingers, supporting localized multipoint feedback is needed.

This project proposes a system capable of localized vibrotactile feedback that can be combined with the input capabilities of mobile devices.

Project Details

Title: Multi-point Vibrotactile Feedback

Date: Jan 1, 2017

Authors: Hugo Nicolau, Tiago Guerreiro, Kyle Montague, João Guerreiro, Vicki L. Hanson

Keywords: blind, output, input, multitouch, feedback, vibrotactile, mobile, wearable

Related Publications

• 
  
  • HoliBraille: Multipoint Vibrotactile Feedback on Mobile Devices
  • Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues, and Vicki L. Hanson. 2015. HoliBraille: Multipoint Vibrotactile Feedback on Mobile Devices. Proceedings of the 12th Web for All Conference, ACM, 30:1–30:4. http://doi.org/10.1145/2745555.2746643
  • [ABSTRACT] [PDF] [LIBRARY]

  We propose HoliBraille, a system that enables Braille input and output on current mobile devices. We use vibrotactile motors combined with dampening materials in order to actuate directly on users’ fingers. The prototype can be attached to current capacitive touchscreen devices enabling multipoint and localized feedback. HoliBraille can be leveraged in several applications including educational tools for learning Braille, as a communication device for deaf-blind people, and as a tactile feedback system for multitouch Braille input. We conducted a user study with 12 blind participants on Braille character discrimination. Results show that HoliBraille is effective in providing localized feedback; however, character discrimination performance is strongly related with number of simultaneous stimuli. We finish by discussing the obtained results and propose future research avenues to improve multipoint vibrotactile perception.
• 
  
  • UbiBraille: Designing and Evaluating a Vibrotactile Braille-reading Device
  • Hugo Nicolau, João Guerreiro, Tiago Guerreiro, and Carriço Luı́s. 2013. UbiBraille: Designing and Evaluating a Vibrotactile Braille-reading Device. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ACM, 23:1–23:8. http://doi.org/10.1145/2513383.2513437
  • [ABSTRACT] [PDF] [LIBRARY]

  Blind people typically resort to audio feedback to access information on electronic devices. However, this modality is not always an appropriate form of output. Novel approaches that allow for private and inconspicuous interaction are paramount. In this paper, we present a vibrotactile reading device that leverages the users’ Braille knowledge to read textual information. UbiBraille consists of six vibrotactile actuators that are used to code a Braille cell and communicate single characters. The device is able to simultaneously actuate the users’ index, middle, and ring fingers of both hands, providing fast and mnemonic output. We conducted two user studies on UbiBraille to assess both character and word reading performance. Character recognition rates ranged from 54% to 100% and were highly character- and user-dependent. Indeed, participants with greater expertise in Braille reading/writing were able to take advantage of this knowledge and achieve higher accuracy rates. Regarding word reading performance, we investigated four different vibrotactile timing conditions. Participants were able to read entire words and obtained recognition rates up to 93% with the most proficient ones being able achieve a rate of 1 character per second.
• 
  
  • Augmenting Braille Input Through Multitouch Feedback
  • Hugo Nicolau, Kyle Montague, João Guerreiro, et al. 2013. Augmenting Braille Input Through Multitouch Feedback. Proceedings of the Adjunct Publication of the 26th Annual ACM Symposium on User Interface Software and Technology, ACM, 81–82. http://doi.org/10.1145/2508468.2514720
  • [ABSTRACT] [PDF] [LIBRARY]

  Current touch interfaces lack the rich tactile feedback that allows blind users to detect and correct errors. This is especially relevant for multitouch interactions, such as Braille input. We propose HoliBraille, a system that combines touch input and multi-point vibrotactile output on mobile devices. We believe this technology can offer several benefits to blind users; namely, convey feedback for complex multitouch gestures, improve input performance, and support inconspicuous interactions. In this paper, we present the design of our unique prototype, which allows users to receive multitouch localized vibrotactile feedback. Preliminary results on perceptual discrimination show an average of 100% and 82% accuracy for single-point and chord discrimination, respectively. Finally, we discuss a text-entry application with rich tactile feedback.
• 
  
  • Guiding Blind People with Haptic Feedback
  • Slim Kammoun, Cristophe Jouffrais, Tiago Guerreiro, Hugo Nicolau, and Joaquim Jorge. 2012. Guiding Blind People with Haptic Feedback. Frontiers in Accessibility for Pervasive Computing Workshop at Pervasive.
  • [ABSTRACT] [PDF]

  Maintaining orientation while traveling in complex or unknown environments is a challenging task for visually impaired (VI) pedestrians. In this paper, we propose a novel approach to assist blind people during navigation between waypoints (walk straight) with tactors on their wrists. Our main goal is to decrease the cognitive load needed by blind people to follow instructions in overloaded environments. Two issues are discussed, 1) the number of vibration motors used; 2) the type of vibration dimensions issued. Preliminary results from of an informal evaluation performed with two blind users showed that vibrations could help the users maintaining their straight path, however patterns were sometimes confusing. This reinforced that walking an unknown path is a demanding and stressful task and the cognitive load should be reduced to a minimum.