research-article

PalmBoard: Leveraging Implicit Touch Pressure in Statistical Decoding for Indirect Text Entry

Authors:

Yuanchun ShiAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376441

Published: 23 April 2020 Publication History

Abstract

We investigated how to incorporate implicit touch pressure, finger pressure applied to a touch surface during typing, to improve text entry performance via statistical decoding. We focused on one-handed touch-typing on indirect interface as an example scenario. We first collected typing data on a pressure-sensitive touchpad, and analyzed users' typing behavior such as touch point distribution, key-to-finger mappings, and pressure images. Our investigation revealed distinct pressure patterns for different keys. Based on the findings, we performed a series of simulations to iteratively optimize the statistical decoding algorithm. Our investigation led to a Markov-Bayesian decoder incorporating pressure image data into decoding. It improved the top-1 accuracy from 53% to 74% over a naive Bayesian decoder. We then implemented PalmBoard, a text entry method that implemented the Markov-Bayesian decoder and effectively supported one-handed touch-typing on indirect interfaces. A user study showed participants achieved an average speed of 32.8 WPM with 0.6% error rate. Expert typists could achieve 40.2 WPM with 30 minutes of practice. Overall, our investigation showed that incorporating implicit touch pressure is effective in improving text entry decoding.

Supplementary Material

MP4 File (a314-yi-presentation.mp4)

Download
33.60 MB

References

[1]

2016. A note on calculating text entry speed. (2016). http://www.yorku.ca/mack/RN-TextEntrySpeed.html.

[2]

2016. American National Corpus. (2016). http://www.americannationalcorpus.org/OANC/index.html.

[3]

R.U. Ayres and K. Martinás. 2005. On the Reappraisal of Microeconomics: Economic Growth and Change in a Material World. Edward Elgar. https://books.google.com/books?id=ksxK7J95IF8C

[4]

Shiri Azenkot and Shumin Zhai. 2012. Touch Behavior with Different Postures on Soft Smartphone Keyboards. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, NY, NY, USA, 251--260.

Digital Library

[5]

Tyler Baldwin and Joyce Chai. 2012. Towards Online Adaptation and Personalization of Key-target Resizing for Mobile Devices. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (IUI '12). ACM, NY, NY, USA, 11--20.

Digital Library

[6]

Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts Law: Modeling Finger Touch with Fitts' Law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 1363--1372.

Digital Library

[7]

Stephen A. Brewster and Michael Hughes. 2009. Pressure-based Text Entry for Mobile Devices. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09). ACM, NY, NY, USA, Article 9, 4 pages.

Digital Library

[8]

Anna Maria Feit, Daryl Weir, and Antti Oulasvirta. 2016. How We Type: Movement Strategies and Performance in Everyday Typing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 4262--4273.

Digital Library

[9]

Leah Findlater and Jacob Wobbrock. 2012. Personalized Input: Improving Ten-finger Touchscreen Typing Through Automatic Adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 815--824.

Digital Library

[10]

Leah Findlater, Jacob O. Wobbrock, and Daniel Wigdor. 2011. Typing on Flat Glass: Examining Ten-finger Expert Typing Patterns on Touch Surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 2453--2462.

Digital Library

[11]

Mayank Goel, Leah Findlater, and Jacob Wobbrock. 2012. WalkType: Using Accelerometer Data to Accomodate Situational Impairments in Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 2687--2696.

Digital Library

[12]

Mayank Goel, Alex Jansen, Travis Mandel, Shwetak N. Patel, and Jacob O. Wobbrock. 2013. ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 2795--2798.

Digital Library

[13]

Jun Gong, Zheer Xu, Qifan Guo, Teddy Seyed, Xiang 'Anthony' Chen, Xiaojun Bi, and Xing-Dong Yang. 2018. WrisText: One-handed Text Entry on Smartwatch Using Wrist Gestures. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 181, 14 pages.

Digital Library

[14]

Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language Modeling for Soft Keyboards. In Proceedings of the 7th International Conference on Intelligent User Interfaces (IUI '02). ACM, NY, NY, USA, 194--195.

Digital Library

[15]

Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 3817--3821.

Digital Library

[16]

Aakar Gupta, Muhammed Anwar, and Ravin Balakrishnan. 2016. Porous Interfaces for Small Screen Multitasking Using Finger Identification. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, NY, NY, USA, 145--156.

Digital Library

[17]

Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: Miniature Screen Text Entry via Finger Identification. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 59--70.

Digital Library

[18]

Johan Himberg, Jonna Häkkilä, Petri Kangas, and Jani Mäntyjärvi. 2003. On-line Personalization of a Touch Screen Based Keyboard. In Proceedings of the 8th International Conference on Intelligent User Interfaces (IUI '03). ACM, NY, NY, USA, 77--84.

Digital Library

[19]

Min-Chieh Hsiu, Da-Yuan Huang, Chi An Chen, Yu-Chih Lin, Yi-ping Hung, De-Nian Yang, and Mike Chen. 2016. ForceBoard: Using Force As Input Technique on Size-limited Soft Keyboard. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM, NY, NY, USA, 599--604.

Digital Library

[20]

Sunjun Kim, Jeongmin Son, Geehyuk Lee, Hwan Kim, and Woohun Lee. 2013. TapBoard: Making a Touch Screen Keyboard More Touchable. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 553--562.

Digital Library

[21]

Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1Line Keyboard: A QWERTY Layout in a Single Line. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, NY, NY, USA, 461--470.

Digital Library

[22]

Yiqin Lu, Chun Yu, Shuyi Fan, Xiaojun Bi, and Yuanchun Shi. 2019. Typing on Split Keyboards with Peripheral Vision. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 200, 12 pages.

Digital Library

[23]

Yiqin Lu, Chun Yu, Xin Yi, Yuanchun Shi, and Shengdong Zhao. 2017. BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 2, Article 18 (June 2017), 24 pages.

Digital Library

[24]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, NY, NY, USA, 754--755.

Digital Library

[25]

Dinesh Mandalapu and Sriram Subramanian. 2011. Exploring Pressure As an Alternative to Multi-touch Based Interaction. In Proceedings of the 3rd International Conference on Human Computer Interaction (IndiaHCI '11). ACM, NY, NY, USA, 88--92.

Digital Library

[26]

David C. McCallum, Edward Mak, Pourang Irani, and Sriram Subramanian. 2009. PressureText: Pressure Input for Mobile Phone Text Entry. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09). ACM, NY, NY, USA, 4519--4524.

Digital Library

[27]

Paul Nation and Robert Waring. 1997. Vocabulary size, text coverage and word lists. Vocabulary: Description, acquisition and pedagogy 14 (1997), 6--19.

[28]

Weinan Shi, Chun Yu, Xin Yi, Zhen Li, and Yuanchun Shi. 2018. TOAST: Ten-Finger Eyes-Free Typing on Touchable Surfaces. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1, Article 33 (March 2018), 23 pages.

Digital Library

[29]

R. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, NY, NY, USA, 113--120.

Digital Library

[30]

Keith Vertanen, Crystal Fletcher, Dylan Gaines, Jacob Gould, and Per Ola Kristensson. 2018. The Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 626, 12 pages.

Digital Library

[31]

Keith Vertanen, Dylan Gaines, Crystal Fletcher, Alex M. Stanage, Robbie Watling, and Per Ola Kristensson. 2019. VelociWatch: Designing and Evaluating a Virtual Keyboard for the Input of Challenging Text. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 591, 14 pages.

Digital Library

[32]

Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating Fast Mobile Text Entry Using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 659--668.

Digital Library

[33]

Feng Wang, Xiang Cao, Xiangshi Ren, and Pourang Irani. 2009. Detecting and Leveraging Finger Orientation for Interaction with Direct-touch Surfaces. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, NY, NY, USA, 23--32.

Digital Library

[34]

Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain Text Entry on Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2307--2316.

Digital Library

[35]

Graham Wilson, Stephen Brewster, and Martin Halvey. 2013. Towards Utilising One-handed Multi-digit Pressure Input. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, NY, NY, USA, 1317--1322.

Digital Library

[36]

Pui Chung Wong, Kening Zhu, and Hongbo Fu. 2018. FingerT9: Leveraging Thumb-to-finger Interaction for Same-side-hand Text Entry on Smartwatches. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 178, 10 pages.

Digital Library

[37]

Hui-Shyong Yeo, Xiao-Shen Phang, Taejin Ha, Woontack Woo, and Aaron Quigley. 2017. TiTAN: Exploring Midair Text Entry Using Freehand Input. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, NY, NY, USA, 3041--3049.

Digital Library

[38]

Xin Yi, Chun Yu, Weinan Shi, Xiaojun Bi, and Yuanchun Shi. 2017b. Word Clarity As a Metric in Sampling Keyboard Test Sets. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 4216--4228.

Digital Library

[39]

Xin Yi, Chun Yu, Weinan Shi, and Yuanchun Shi. 2017a. Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards. International Journal of Human-Computer Studies 106 (2017), 44--62.

[40]

Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, NY, NY, USA, 539--548.

Digital Library

[41]

Chun Yu, Yizheng Gu, Zhican Yang, Xin Yi, Hengliang Luo, and Yuanchun Shi. 2017. Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 4479--4488.

Digital Library

[42]

Mingyuan Zhong, Chun Yu, Qian Wang, Xuhai Xu, and Yuanchun Shi. 2018. ForceBoard: Subtle Text Entry Leveraging Pressure. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 528, 10 pages.

Digital Library

[43]

Suwen Zhu, Tianyao Luo, Xiaojun Bi, and Shumin Zhai. 2018. Typing on an Invisible Keyboard. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 439, 13 pages.

Digital Library

[44]

Suwen Zhu, Jingjie Zheng, Shumin Zhai, and Xiaojun Bi. 2019. I'sFree: Eyes-Free Gesture Typing via a Touch-Enabled Remote Control. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 448, 12 pages.

Digital Library

Cited By

Moon HLiao YLi CLee BOulasvirta A(2024)Real-time 3D Target Inference via Biomechanical SimulationProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642131(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642131
Yang YGuo BLiang YZhao KYu Z(2024)Cross-device free-text keystroke dynamics authentication using federated learningPersonal and Ubiquitous Computing10.1007/s00779-024-01832-6Online publication date: 10-Sep-2024
https://doi.org/10.1007/s00779-024-01832-6
Yi XLiang CChen HSong JYu CLi HShi Y(2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580829
Show More Cited By

Index Terms

PalmBoard: Leveraging Implicit Touch Pressure in Statistical Decoding for Indirect Text Entry
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Text input

Recommendations

Sandwich keyboard: fast ten-finger typing on a mobile device with adaptive touch sensing on the back side
MobileHCI '13: Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services

This Note introduces a keyboard design that affords ten-finger touch typing by utilizing a touch sensor on the back side of a device. Previous work has used physical buttons. Using a touch sensor has the benefit that it retains the form factor and does ...
Text Entry for Ultra-Small Touchscreens Using a Fixed Cursor and Movable Keyboard
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Emerging ultra-small wearables like smartwatches pose a design challenge for touch-based text entry, due to the "fat finger" problem, wherein users struggle to select elements much smaller than their fingers. In order to address the challenge, we ...
Establishing a baseline for text entry for a multi-touch virtual keyboard

Multi-touch, which has been heralded as a revolution in human-computer interaction, provides features such as gestural interaction, tangible interfaces, pen-based computing, and interface customization-features embraced by an increasingly tech-savvy ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Beijing Key Lab of Networked Multimedia
National Key Research and Development Plan
Natural Science Foundation of China

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
562
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)5

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Moon HLiao YLi CLee BOulasvirta A(2024)Real-time 3D Target Inference via Biomechanical SimulationProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642131(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642131
Yang YGuo BLiang YZhao KYu Z(2024)Cross-device free-text keystroke dynamics authentication using federated learningPersonal and Ubiquitous Computing10.1007/s00779-024-01832-6Online publication date: 10-Sep-2024
https://doi.org/10.1007/s00779-024-01832-6
Yi XLiang CChen HSong JYu CLi HShi Y(2023)From 2D to 3DProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808297:1(1-25)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580829
Yi XZhang SPan ZShi LHan FKong YLi HShi Y(2023)Squeez’In: Private Authentication on Smartphones based on Squeezing GesturesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581419(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581419
Faleel SLiu YCody RRey BDu LYu JHuang DIrani PLi W(2023)T-Force: Exploring the Use of Typing Force for Three State Virtual KeyboardsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580915(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580915
Komninos A(2022)The Value of Open Data in HCI: A Case Report from Mobile Text Entry ResearchMultimodal Technologies and Interaction10.3390/mti60900716:9(71)Online publication date: 23-Aug-2022
https://doi.org/10.3390/mti6090071
Zhang MZhai SWobbrock J(2022)TypeAnywhere: A QWERTY-Based Text Entry Solution for Ubiquitous ComputingProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517686(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517686
He ZLutteroth CPerlin K(2022)TapGazer: Text Entry with Finger Tapping and Gaze-directed Word SelectionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501838(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501838
Zhang QWang DZhao RYu YJing J(2021)Write, Attend and SpellProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781005:3(1-25)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478100
Lee LYeung NBraud TLi TSu XHui PTruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Force9: Force-assisted Miniature Keyboard on Smart WearablesProceedings of the 2020 International Conference on Multimodal Interaction10.1145/3382507.3418827(232-241)Online publication date: 21-Oct-2020
https://dl.acm.org/doi/10.1145/3382507.3418827

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents