research-article

Supporting Healthy Grocery Shopping via Mobile Augmented Reality

Authors:

James Williamson,

Shivakant MishraAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 12, Issue 1s

Article No.: 16, Pages 1 - 24

https://doi.org/10.1145/2808207

Published: 21 October 2015 Publication History

Abstract

Augmented reality (AR) applications have recently become popular on modern smartphones. We explore the effectiveness of this mobile AR technology in the context of grocery shopping, in particular as a means to assist shoppers in making healthier decisions as they decide which grocery products to buy. We construct an AR-assisted mobile grocery-shopping application that makes real-time, customized recommendations of healthy products to users and also highlights products to avoid for various types of health concerns, such as allergies to milk or nut products, low-sodium or low-fat diets, and general caloric intake. We have implemented a prototype of this AR-assisted mobile grocery shopping application and evaluated its effectiveness in grocery store aisles. Our application's evaluation with typical grocery shoppers demonstrates that AR overlay tagging of products reduces the search time to find healthy food items, and that coloring the tags helps to improve the user's ability to quickly and easily identify recommended products, as well as products to avoid. We have evaluated our application's functionality by analyzing the data we collected from 15 in-person actual grocery-shopping subjects and 104 online application survey participants.

References

[1]

G. Agapito, B. Calabrese, I. Care, D. Falcone, P. H. Guzzi, N. Ielpo, T. Lamprinoudi, M. Milano, M. Simeoni, and M. Cannataro. 2014. Profiling basic health information of tourists: Towards a recommendation system for the adaptive delivery of medical certified nutrition contents. In Proceedings of the 2014 International Conference on High Performance Computing Simulation (HPCS). 616--620.

[2]

Junho Ahn, Mike Gartrell, James Williamson, and Richard Han. 2012. ARFusion: An indoor mobile augmented reality system supported by pedometry and context-awareness, an AR-assisted grocery shopping application. Institutional Review Board on the University of Colorado at Boulder 12-0102.

[3]

Junho Ahn and Richard Han. 2011. RescueMe: An indoor mobile augmented-reality evacuation system by personalized pedometry. In Proceedings of the IEEE Asia-Pacific Services Computing Conference (APSCC).

[4]

Eileen Anderson, Richard Wientt, Janet Wojcik, Sheila Wientt, and Todd Bowden. 2001. A computerized social cognitive intervention for nutrition behavior: Direct and mediated effects on fat, fiber, fruits, and vegetables, self-efficacy, and outcome expectations among food shoppers. Ann. Behav. Med. 23, 2, 88--100.

[5]

Amy Barton, Lynn Gilbert, Julaluk Baramee, and Theresa Granger. 2006. Cardiovascular risk in hispanic and non-hispanic preschoolers. National Institute of Health (May-June 2006).

[6]

Stephane Beauregard. 2007. Omnidirectional pedestrian navigation for first responders. In Proceedings of the 4th Workshop on Positioning, Navigation and Communication (WPNC'07).

[7]

J. Bobadilla, F. Ortega, A. Hernando, and A. GutiéRrez. 2013. Recommender systems survey. Know.-Based Syst. 46 (2013), 109--132.

Digital Library

[8]

Cristina Botella, Juani Breton-Lopez, Soledad Quero, Rosa Banos, Azucena Garcia-Palacios, Irene Zaragoza, and Alcaniz Raya. 2011. Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: A single case study. Comput. Hum. Behav. 27, 1, 217--227.

Digital Library

[9]

DanKam. 2010. DanKam AR application for color blindness. http://news.cnet.com/8301-27080_3-20026054-245.html.

[10]

DietGuideline 2010. U.S. Department of Agriculture. 2010 Dietary guidelines for Americans. http://www.cnpp.usda.gov/DGAs2010-PolicyDocument.htm. (2010).

[11]

Jill Freyne and Shlomo Berkovsky. 2010. Intelligent food planning: Personalized recipe recommendation. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI'10). 321--324.

Digital Library

[12]

Christoph Fuchs, Nils Aschenbruck, Peter Martini, and Monika Wieneke. 2011. A survey on indoor tracking for mission critical scenarios. Perva. Mobile Comput. 7, 1.

Digital Library

[13]

Subhashini Ganapathy, Glen Anderson, and Igor Kozintsev. 2011. MAR shopping assistant usage: Delay, error, and utility. In Proceedings of the IEEE Virtual Reality Conference (VR). 207--208.

Digital Library

[14]

Golfscape. 2014. Golfscape GPS AR range finder. http://golfscapeapp.com/.

[15]

GoogleDocs 2014. Google Docs' survey tool. http://support.google.com/docs/bin/answer.py?hl=en&answer=87809.

[16]

Levent Görgü, Abraham Campbell, Kealan McCusker, Mauro Dragone, Michael O'Grady, Noel O'Connor, and Greg O'Hare. 2010. FreeGaming: Mobile, collaborative, adaptive and augmented exergaming. In Proceedings of the 8th International Conference on Advances in Mobile Computing and Multimedia (MoMM'10). 173--179.

Digital Library

[17]

Yanying Gu, Anthony Lo, and Ignas Niemegeers. 2009. A survey of indoor positioning systems for wireless personal networks. IEEE Commun. Surv. Tutori. 11, 1, 13--32.

Digital Library

[18]

Ramón Hervás, Alberto Garcia-Lillo, and José Bravo. 2011. Mobile augmented reality based on the semantic web applied to ambient assisted living. In Ambient Assisted Living, Jos Bravo, Ramn Hervs, and Vladimir Villarreal (Eds.), Lecture Notes in Computer Science, vol. 6693, Springer Berlin / Heidelberg, 17--24.

[19]

Hwajung Hong, Hee Young Jeong, Rosa I. Arriaga, and Gregory D. Abowd. 2010. TriggerHunter: Designing an educational game for families with asthmatic children. In Proceedings of CHI EA'10. 3577--3582.

Digital Library

[20]

IQEngines. 2013. IQEngines: Image recognition and visual search. http://www.iqengines.com.

[21]

Vaiva Kalnikaite, Yvonne Rogers, Jon Bird, Nicolas Villar, Khaled Bachour, Stephen Payne, Peter Todd, Johannes Schoning, Antonio Kruger, and Stefan Kreitmayer. 2011. How to nudge in Situ: Designing Lambent devices to deliver salience information in supermarkets. In Proceedings of UbiComp'11.

Digital Library

[22]

David Katz, Valentine Njike, Zubaida Faridi, Lauren Rhee, Rebecca Reeves, and David Jenkins. 2009. The stratification of foods on the basis of overall nutritional quality: The overall nutritional quality index. Amer. J. Health Promot. (Nov.--Dec. 2009).

[23]

Stefan Ladstaetter, Patrick Luley, Alexander Almer, and Lucas Paletta. 2010. Multisensor data fusion for high accuracy positioning on mobile phones. In Proceedings of MobileHCI. 395--396.

Digital Library

[24]

Jia-Kuan Lin, Po-Hsun Cheng, Yen Su, et al. 2011. Augmented reality serious game framework for rehabilitation with personal health records. In Proceedings of the 13th IEEE International Conference on e-Health Networking Applications and Services (Healthcom). IEEE, 197--200.

[25]

T. Lobstein and S. Davies. 2009. Defining and labeling ‘healthy’ and ‘unhealthy’ food. Public Health Nutr. (2009).

[26]

Jennifer Mankoff, Gary Hsieh, Ho Chak Hung, Sharon Lee, and Elizabeth Nitao. 2002. Using low-cost sensing to support nutritional awareness. In Proceedings of the 4th International Conference on Ubiquitous Computing. Springer-Verlag, 371--376.

Digital Library

[27]

MechanicalTurk 2014. Amazon Mechanical Turk Survey Services. https://www.mturk.com/mturk/.

[28]

Cliona Ni Mhurchu, Tony Blakely, Joanne Wall, Anthony Rodgers, Yannan Jiang, and Jenny Wilton. 2007. Strategies to promote healthier food purchases: A pilot supermarket intervention study. Public Health Nut. 10, 06, 608--615.

[29]

Barry Mulrooney, Mairéad McDermott, and Nick Earley. 2006. NutraStick: Portable diet assistant. In CHI'06 extended abstracts on Human factors in computing systems (CHI EA'06). ACM, New York, 1855--1860.

Digital Library

[30]

Yoosoo Oh, Ahyoung Choi, and Woontack Woo. 2010. u-BabSang: A context-aware food recommendation system. J. Supercomput. 54, 1, 61--81.

Digital Library

[31]

PCAST. 2010. Realizing the full potential of health information technology to improve healthcare for americans: The path forward. http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-health-it-report.pdf. (2010).

[32]

M. Phanich, P. Pholkul, and S. Phimoltares. 2010. Food recommendation system using clustering analysis for diabetic patients. In Proceedings of the International Conference on Information Science and Applications (ICISA). 1--8.

[33]

QUIS 1987. Questionnaire for user interaction satisfaction. http://lap.umd.edu/quis/about.html.

[34]

SkyMap. 2011. Google Sky Map AR astronomy application. http://www.google.com/mobile/skymap/.

[35]

Richard Wientt, Eileen Smith Anderson-Bill, Patricia G. Bickley, Janet Walberg-Rankin, John F. Moore, and Michael Leahy. 1997. Nutrition for a Lifetime System ©: A multimedia system for altering food supermarket shoppers' purchases to meet nutritional guidelines. Comput. Hum. Behav. 13, 3, 371--392. 10.1016/S0747-5632(97)00015-0

[36]

WordLens. 2014. WordLens augmented reality language translation app for the iPhone. http://www.questvisual.com/.

Cited By

Dimou VStyliaras GSalomidis K(2024)AR-Based Food Traceability as a Means for Sustainable DevelopmentSustainability10.3390/su1607303716:7(3037)Online publication date: 5-Apr-2024
https://doi.org/10.3390/su16073037
Wolniak RStecuła KAydın B(2024)Digital Transformation of Grocery In-Store Shopping-Scanners, Artificial Intelligence, Augmented Reality and Beyond: A ReviewFoods10.3390/foods1318294813:18(2948)Online publication date: 18-Sep-2024
https://doi.org/10.3390/foods13182948
Altammami ADimitrova VPournaras E(2024)What you see, What you get? Mapping Inconsistencies of Sustainability Judgements among Experts and ConsumersProceedings of the 2024 International Conference on Information Technology for Social Good10.1145/3677525.3678695(443-452)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3677525.3678695
Show More Cited By

Index Terms

Supporting Healthy Grocery Shopping via Mobile Augmented Reality
1. Information systems
  1. Information systems applications
    1. Mobile information processing systems

Recommendations

Food Literacy while Shopping: Motivating Informed Food Purchasing Behaviour with a Situated Gameful App
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Establishing healthy eating patterns early in life is critical and has implications for lifelong health. Situated interventions are a promising approach to improve eating patterns. However, HCI research has emphasized calorie control and weight loss, ...
How far can Wearable Augmented Reality Influence Customer Shopping Behavior
MobiQuitous '20: MobiQuitous 2020 - 17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services

We investigate if providing shoppers with Augmented reality (AR) as a shopping tool can lead to an increase in purchase rate compared with conventional shopping applications. In a ”simulated shopping” study with two groups with a total number of 20 ...
Shopping with augmented reality: How wow-effect changes the equations!
Highlights
- More appropriate adjustment of AR features facilitates positive consumer behaviors.
Abstract
E-retailers have recently devoted novel forms of communication tools, such as augmented reality applications, to create a hedonic experience for customers and motivate them to purchase. AR has also obtained significant attention from ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 12, Issue 1s

Special Issue on Smartphone-Based Interactive Technologies, Systems, and Applications and Special Issue on Extended Best Papers from ACM Multimedia 2014

October 2015

317 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2837676

Editor:
Ralf Steinmetz
Technische Universität Darmstadt, Germany

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2015

Accepted: 01 June 2015

Revised: 01 April 2015

Received: 01 January 2015

Published in TOMM Volume 12, Issue 1s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
2,046
Total Downloads

Downloads (Last 12 months)198
Downloads (Last 6 weeks)13

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dimou VStyliaras GSalomidis K(2024)AR-Based Food Traceability as a Means for Sustainable DevelopmentSustainability10.3390/su1607303716:7(3037)Online publication date: 5-Apr-2024
https://doi.org/10.3390/su16073037
Wolniak RStecuła KAydın B(2024)Digital Transformation of Grocery In-Store Shopping-Scanners, Artificial Intelligence, Augmented Reality and Beyond: A ReviewFoods10.3390/foods1318294813:18(2948)Online publication date: 18-Sep-2024
https://doi.org/10.3390/foods13182948
Altammami ADimitrova VPournaras E(2024)What you see, What you get? Mapping Inconsistencies of Sustainability Judgements among Experts and ConsumersProceedings of the 2024 International Conference on Information Technology for Social Good10.1145/3677525.3678695(443-452)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1145/3677525.3678695
Yigitbas EMazur J(2024)Augmented and Virtual Reality for Diet and Nutritional Education: A Systematic Literature ReviewProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3652048(88-97)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1145/3652037.3652048
Strecker JWu JBektaş KVaslin CMayer S(2024)ShoppingCoach: Using Diminished Reality to Prevent Unhealthy Food Choices in an Offline Supermarket ScenarioExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650795(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650795
Wimer BSzymanski AMetoyer R(2024)Beyond Static Labels: Unpacking Nutrition Comprehension in the Digital AgeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642672(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642672
Doerr NLee BBaricova KSchmalstieg DSedlmair M(2024)Visual Highlighting for Situated Brushing and LinkingComputer Graphics Forum10.1111/cgf.1510543:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15105
Shin SBatch AButcher PRitsos PElmqvist N(2024)The Reality of the Situation: A Survey of Situated AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328554630:8(5147-5164)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3285546
Mandala VAkepati ARajkumar SR S(2024)The Future of Retail: Harnessing 5G Beyond Networks for Immersive Augmented Reality Shopping Environments2024 3rd International Conference on Artificial Intelligence For Internet of Things (AIIoT)10.1109/AIIoT58432.2024.10574630(1-6)Online publication date: 3-May-2024
https://doi.org/10.1109/AIIoT58432.2024.10574630
Xi NChen JGama FKorkeila HHamari J(2024)Acceptance of the metaverse: a laboratory experiment on augmented and virtual reality shoppingInternet Research10.1108/INTR-05-2022-033434:7(82-117)Online publication date: 26-Jan-2024
https://doi.org/10.1108/INTR-05-2022-0334
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents