skip to main content
10.1145/3025453.3025561acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

TriTap: Identifying Finger Touches on Smartwatches

Published: 02 May 2017 Publication History

Abstract

The small screens of smartwatches provide limited space for input tasks. Finger identification is a promising technique to address this problem by associating different functions with different fingers. However, current technologies for finger identification are unavailable or unsuitable for smartwatches. To address this problem, this paper observes that normal smartwatch use takes places with a relatively static pose between the two hands. In this situation, we argue that the touch and angle profiles generated by different fingers on a standard smartwatch touch screen will differ sufficiently to support reliable identification. The viability of this idea is explored in two studies that capture touches in natural and exaggerated poses during tapping and swiping tasks. Machine learning models report accuracies of up to 93% and 98% respectively, figures that are sufficient for many common interaction tasks. Furthermore, the exaggerated poses show modest costs (in terms of time/errors) compared to the natural touches. We conclude by presenting examples and discussing how interaction designs using finger identification can be adapted to the smartwatch form factor.

Supplementary Material

suppl.mov (pn1521p.mp4)
Supplemental video

References

[1]
Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, and Saul Greenberg. 2012. The fat thumb: using the thumb's contact size for single-handed mobile interaction. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12), 39--48. http://dx.doi.org/10.1145/2371574.2371582
[2]
Philipp Ewerling, Alexander Kulik, and Bernd Froehlich. 2012. Finger and hand detection for multitouch interfaces based on maximally stable extremal regions. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces (ITS '12), 173--182. http://dx.doi.org/10.1145/2396636.2396663
[3]
Emilien Ghomi, Stéphane Huot, Olivier Bau, Michel Beaudouin-Lafon, and Wendy E. Mackay. 2013. Arpège: learning multitouch chord gestures vocabularies. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (ITS '13), 209--218. http://dx.doi.org/10.1145/2512349.2512795
[4]
Alix Goguey, Mathieu Nancel, Géry Casiez, and Daniel Vogel. 2016. The Performance and Preference of Different Fingers and Chords for Pointing, Dragging, and Object Transformation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4250--4261. http://dx.doi.org/10.1145/2858036.2858194
[5]
Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: Miniature Screen Text Entry via Finger Identification. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 59--70. http://dx.doi.org/10.1145/2858036.2858052
[6]
Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11, 1 (November 2009), 10--18. http://dx.doi.org/10.1145/1656274.1656278
[7]
Chris Harrison, Julia Schwarz, and Scott E. Hudson. 2011. TapSense: enhancing finger interaction on touch surfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11), 627--636. http://dx.doi.org/10.1145/2047196.2047279
[8]
Christian Holz and Patrick Baudisch. 2010. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 581--590. http://dx.doi.org/10.1145/1753326.1753413
[9]
Christian Holz and Patrick Baudisch. 2013. Fiberio: a touchscreen that senses fingerprints. In Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST '13), 41--50. http://dx.doi.org/10.1145/2501988.2502021
[10]
Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3011--3014. http://dx.doi.org/10.1145/2702123.2702518
[11]
Benjamin Lafreniere, Carl Gutwin, Andy Cockburn, and Tovi Grossman. 2016. Faster Command Selection on Touchscreen Watches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4663--4674. http://dx.doi.org/10.1145/2858036.2858166
[12]
Gierad Laput, Robert Xiao, Xiang 'Anthony' Chen, Scott E. Hudson, and Chris Harrison. 2014. Skin buttons: cheap, small, low-powered and clickable fixed-icon laser projectors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 389--394. http://dx.doi.org/10.1145/2642918.2647356
[13]
Luis A. Leiva, Alireza Sahami, Alejandro Catala, Niels Henze, and Albrecht Schmidt. 2015. Text Entry on Tiny QWERTY Soft Keyboards. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 669--678. http://dx.doi.org/10.1145/2702123.2702388
[14]
I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03), 754--755. http://dx.doi.org/10.1145/765891.765971
[15]
Nicolai Marquardt, Johannes Kiemer, and Saul Greenberg. 2010. What caused that touch? expressive interaction with a surface through fiduciary-tagged gloves. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10), 139--142. http://dx.doi.org/10.1145/1936652.1936680
[16]
Ian Oakley and Doyoung Lee. 2014. Interaction on the edge: offset sensing for small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 169--178. http://dx.doi.org/10.1145/2556288.2557138
[17]
Ian Oakley, DoYoung Lee, M. D. Rasel Islam, and Augusto Esteves. 2015. Beats: Tapping Gestures for Smart Watches. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 1237--1246. http://dx.doi.org/10.1145/2702123.2702226
[18]
Ian Oakley, Carina Lindahl, Khanh Le, DoYoung Lee, and MD. Rasel Islam. 2016. The Flat Finger: Exploring Area Touches on Smartwatches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4238--4249. http://dx.doi.org/10.1145/2858036.2858179
[19]
Simon T. Perrault, Eric Lecolinet, James Eagan, and Yves Guiard. 2013. Watchit: simple gestures and eyesfree interaction for wristwatches and bracelets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). 1451--1460. http://doi.acm.org/10.1145/2470654.2466192
[20]
Simon Rogers, John Williamson, Craig Stewart, and Roderick Murray-Smith. 2011. AnglePose: robust, precise capacitive touch tracking via 3d orientation estimation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 2575--2584. http://doi.acm.org/10.1145/1978942.1979318
[21]
Quentin Roy, Yves Guiard, Gilles Bailly, Éric Lecolinet, and Olivier Rioul. "Glass+ skin: An empirical evaluation of the added value of finger identification to basic single-touch interaction on touch screens." In Human-Computer Interaction, pp. 55--71. Springer International Publishing, 2015. http://dx.doi.org/10.1007/978-3-319-22723-8_5
[22]
T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making musclecomputer interfaces more practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 851--854. http://dx.doi.org/10.1145/1753326.1753451
[23]
Craig Stewart, Michael Rohs, Sven Kratz, and Georg Essl. 2010. Characteristics of pressure-based input for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 801--810. http://dx.doi.org/10.1145/1753326.1753444
[24]
Atsushi Sugiura and Yoshiyuki Koseki. 1998. A user interface using fingerprint recognition: holding commands and data objects on fingers. In Proceedings of the 11th annual ACM symposium on User interface software and technology (UIST '98). ACM, NY, NY, USA, 71--79. http://dx.doi.org/10.1145/288392.288575
[25]
Jingtao Wang and John Canny. 2004. FingerSense: augmenting expressiveness to physical pushing button by fingertip identification. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA '04), 1267--1270. http://dx.doi.org/10.1145/985921.986040
[26]
Feng Wang, Xiang Cao, Xiangshi Ren, and Pourang Irani. 2009. Detecting and leveraging finger orientation for interaction with direct-touch surfaces. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09), 2332. http://doi.acm.org/10.1145/1622176.1622182
[27]
Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. 2015. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2991--3000. http://dx.doi.org/10.1145/2702123.2702391
[28]
Hongyi Wen, Julian Ramos Rojas, and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3847--3851. http://dx.doi.org/10.1145/2858036.2858466
[29]
Robert Xiao, Julia Schwarz, and Chris Harrison. 2015. Estimating 3D Finger Angle on Commodity Touchscreens. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15), 47--50. http://dx.doi.org/10.1145/2817721.2817737
[30]
Hui-Shyong Yeo, Juyoung Lee, Andrea Bianchi, and Aaron Quigley. 2016. WatchMI: pressure touch, twist and pan gesture input on unmodified smartwatches. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 394--399. http://dx.doi.org/10.1145/2935334.2935375
[31]
Yang Zhang and Chris Harrison. 2015. Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15), 167--173. http://dx.doi.org/10.1145/2807442.2807480
[32]
Cheng Zhang, Junrui Yang, Caleb Southern, Thad E. Starner, and Gregory D. Abowd. 2016. WatchOut: extending interactions on a smartwatch with inertial sensing. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), 136143. http://dx.doi.org/10.1145/2971763.2971775

Cited By

View all
  • (2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
  • (2024)A Systematic Review of Human Activity Recognition Based on Mobile Devices: Overview, Progress and TrendsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.335759126:2(890-929)Online publication date: Oct-2025
  • (2023)Watch Your Language: Using Smartwatches to Support CommunicationProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608379(1-21)Online publication date: 22-Oct-2023
  • Show More Cited By

Index Terms

  1. TriTap: Identifying Finger Touches on Smartwatches

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. finger identification
    2. smartwatch
    3. touch contact profile

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)96
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 13 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
    • (2024)A Systematic Review of Human Activity Recognition Based on Mobile Devices: Overview, Progress and TrendsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.335759126:2(890-929)Online publication date: Oct-2025
    • (2023)Watch Your Language: Using Smartwatches to Support CommunicationProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608379(1-21)Online publication date: 22-Oct-2023
    • (2023)SonarAuth: Using Around Device Sensing to Improve Smartwatch Behavioral BiometricsAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610696(83-87)Online publication date: 8-Oct-2023
    • (2023)ThumbAirProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694746:4(1-30)Online publication date: 11-Jan-2023
    • (2023)Scene Graph Semantic Inference for Image and Text MatchingACM Transactions on Asian and Low-Resource Language Information Processing10.1145/356339022:5(1-23)Online publication date: 9-May-2023
    • (2023)Around-device finger input on commodity smartwatches with learning guidance through discoverabilityInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103105179:COnline publication date: 1-Nov-2023
    • (2023)MaterialSense: Estimating and utilizing material properties of contact objects in multi-touch interactionInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102985172(102985)Online publication date: Apr-2023
    • (2022)ThumbPitch: Enriching Thumb Interaction on Mobile Touchscreens using Deep LearningProceedings of the 34th Australian Conference on Human-Computer Interaction10.1145/3572921.3572925(58-66)Online publication date: 29-Nov-2022
    • (2022)Free as a Bird, but at What Cost?Proceedings of the ACM on Human-Computer Interaction10.1145/35467446:MHCI(1-24)Online publication date: 20-Sep-2022
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media