exHAR: An Interface for Helping Non-Experts Develop and Debug Knowledge-based Human Activity Recognition Systems
Article No.: 11, Pages 1 - 30
Abstract
Human activity recognition (HAR) is crucial for ubiquitous computing systems. While HAR systems are able to recognize a predefined set of activities established during the development process, they often fail to handle users' unique ways of completing these activities and changes in their behavior over time, as well as different activities. Knowledge-based HAR models have been proposed to help individuals create new activity definitions based on common-sense rules, but little research has been done to understand how users approach this task. To investigate this process, we developed and studied how people interact with an explainable knowledge-based HAR development tool called exHAR. Our tool empowers users to define their activities as a set of factual propositions. Users can debug these definitions by soliciting explanations for model predictions (why and why-not) and candidate corrections for faulty predictions (what-if and how-to). After conducting a study to evaluate the effectiveness of exHAR in helping users design accurate HAR systems, we conducted a think-aloud study to better understand people's approach to debugging and personalizing HAR systems and the challenges they may encounter. Our findings revealed why some participants had inaccurate mental models of knowledge-based HAR systems and inefficient approaches to the debugging process.
References
[1]
Rakesh Agrawal, Ramakrishnan Srikant, et al. 1994. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, Vol. 1215. Santiago, Chile, 487--499.
[2]
Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, and Simon Lacoste-Julien. 2017. Joint discovery of object states and manipulation actions. In Proceedings of the IEEE International Conference on Computer Vision. 2127--2136. https://doi.ieeecomputersociety.org/10.1109/ICCV.2017.234
[3]
Hamza Amrani, Daniela Micucci, and Paolo Napoletano. 2021. Personalized models in human activity recognition using deep learning. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 9682--9688. https://doi.org/10.1109/ICPR48806.2021.9412140
[4]
Luca Arrotta, Gabriele Civitarese, and Claudio Bettini. 2022. DeXAR: Deep Explainable Sensor-Based Activity Recognition in Smart-Home Environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--30. https://doi.org/10.1145/3517224
[5]
Daniel Ashbrook and Thad Starner. 2010. MAGIC: a motion gesture design tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2159--2168. https://doi.org/10.1145/1753326.1753653
[6]
Franz Baader, Diego Calvanese, Deborah McGuinness, Peter Patel-Schneider, and Daniele Nardi. 2003. The description logic handbook: Theory, implementation and applications. Cambridge university press.
[7]
Nikola Banovic and John Krumm. 2018. Warming up to cold start personalization. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1--13. https://doi.org/10.1145/3161175
[8]
Abdelkareem Bedri, Diana Li, Rushil Khurana, Kunal Bhuwalka, and Mayank Goel. 2020. Fitbyte: Automatic diet monitoring in unconstrained situations using multimodal sensing on eyeglasses. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--12. https://doi.org/10.1145/3313831.3376869
[9]
Alberto Blanco-Justicia, Josep Domingo-Ferrer, Sergio Martinez, and David Sanchez. 2020. Machine learning explainability via microaggregation and shallow decision trees. Knowledge-Based Systems 194 (2020), 105532. https://doi.org/10.1016/j.knosys.2020.105532
[10]
Damien Bouchabou, Sao Mai Nguyen, Christophe Lohr, Benoit LeDuc, and Ioannis Kanellos. 2021. A survey of human activity recognition in smart homes based on IoT sensors algorithms: Taxonomies, challenges, and opportunities with deep learning. Sensors 21, 18 (2021), 6037. https://doi.org/10.3390/s21186037
[11]
Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI": uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. Proceedings of the ACM on Human-computer Interaction 3, CSCW (2019), 1--24. https://doi.org/10.1145/3359206
[12]
Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2021. Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1--7. https://doi.org/10.1145/3411763.3443435
[13]
Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Computing Surveys (CSUR) 54, 4 (2021), 1--40. https://doi.org/10.1145/3447744
[14]
Wenqiang Chen, Shupei Lin, Elizabeth Thompson, and John Stankovic. 2021. Sensecollect: We need efficient ways to collect on-body sensor-based human activity data! Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--27. https://doi.org/10.1145/3478119
[15]
Gabriele Civitarese, Claudio Bettini, Timo Sztyler, Daniele Riboni, and Heiner Stuckenschmidt. 2018. NECTAR: Knowledge-based collaborative active learning for activity recognition. In 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1--10. https://doi.org/10.1109/PERCOM.2018.8444590
[16]
Gabriele Civitarese, Timo Sztyler, Daniele Riboni, Claudio Bettini, and Heiner Stuckenschmidt. 2019. POLARIS: Probabilistic and ontological activity recognition in smart-homes. IEEE Transactions on Knowledge and Data Engineering 33, 1 (2019), 209--223. https://doi.org/10.1109/TKDE.2019.2930050
[17]
Fulvio Corno, Luigi De Russis, and Alberto Monge Roffarello. 2019. Empowering end users in debugging trigger-action rules. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13. https://doi.org/10.1145/3290605.3300618
[18]
Fulvio Corno, Luigi De Russis, and Alberto Monge Roffarello. 2019. My iot puzzle: Debugging if-then rules through the jigsaw metaphor. In International Symposium on End User Development. Springer, 18--33. https://doi.org/10.1007/978-3-030-24781-2_2
[19]
Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Jian Ma, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. 2022. Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100. International Journal of Computer Vision (IJCV) 130 (2022), 33--55. https://doi.org/10.1007/s11263-021-01531-2
[20]
Scott Davidoff, John Zimmerman, and Anind K Dey. 2010. How routine learners can support family coordination. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2461--2470. https://doi.org/10.1145/1753326.1753699
[21]
Anind K Dey, Raffay Hamid, Chris Beckmann, Ian Li, and Daniel Hsu. 2004. a CAPpella: programming by demonstration of context-aware applications. In Proceedings of the SIGCHI conference on Human factors in computing systems. 33--40. https://doi.org/10.1145/985692.985697
[22]
Anind K Dey, Timothy Sohn, Sara Streng, and Justin Kodama. 2006. iCAP: Interactive prototyping of context-aware applications. In International conference on pervasive computing. Springer, 254-271. https://doi.org/10.1007/11748625_16
[23]
Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General 144, 1 (2015), 114. https://doi.org/10.1037/xge0000033
[24]
Hyo Jin Do, Ha-Kyung Kong, Pooja Tetali, Jaewook Lee, and Brian P Bailey. 2023. To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1--23. https://doi.org/10.1145/3579532
[25]
Daniel A Epstein, Monica Caraway, Chuck Johnston, An Ping, James Fogarty, and Sean A Munson. 2016. Beyond abandonment to next steps: understanding and designing for life after personal informatics tool use. In Proceedings of the 2016 CHI conference on human factors in computing systems. 1109--1113. https://doi.org/10.1145/2858036.2858045
[26]
Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 80--89. https://doi.org/10.1109/DSAA.2018.00018
[27]
Takashi Hamatani, Moustafa Elhamshary, Akira Uchiyama, and Teruo Higashino. 2018. FluidMeter: Gauging the human daily fluid intake using smartwatches. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--25. https://doi.org/10.1145/3264923
[28]
Björn Hartmann, Leith Abdulla, Manas Mittal, and Scott R Klemmer. 2007. Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. In Proceedings of the SIGCHI conference on Human factors in computing systems. 145--154. https://doi.org/10.1145/1240624.1240646
[29]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778. https://doi.org/10.1109/CVPR.2016.90
[30]
Rim Helaoui, Mathias Niepert, and Heiner Stuckenschmidt. 2011. Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships. Pervasive and Mobile Computing 7, 6 (2011), 660--670. https://doi.org/10.1016/j.pmcj.2011.08.004
[31]
Rim Helaoui, Daniele Riboni, and Heiner Stuckenschmidt. 2013. A probabilistic ontological framework for the recognition of multilevel human activities. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 345--354. https://doi.org/10.1145/2493432.2493501
[32]
Jay Henderson, Tanya R Jonker, Edward Lank, Daniel Wigdor, and Ben Lafreniere. 2022. Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input Technique. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--24. https://doi.org/10.1145/3517262
[33]
Jonathan L Herlocker, Joseph A Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. 241--250. https://doi.org/10.1145/358916.358995
[34]
Pascal Hitzler, Markus Krötzsch, Bijan Parsia, Peter F Patel-Schneider, Sebastian Rudolph, et al. 2009. OWL 2 web ontology language primer. W3C recommendation 27, 1 (2009), 123.
[35]
Bernhard Hommel, Jochen Müsseler, Gisa Aschersleben, and Wolfgang Prinz. 2001. The theory of event coding (TEC): A framework for perception and action planning. Behavioral and brain sciences 24, 5 (2001), 849--878. https://doi.org/10.1017/s0140525x01000103
[36]
Ian Horrocks, Peter F Patel-Schneider, Harold Boley, Said Tabet, Benjamin Grosof, Mike Dean, et al. 2004. SWRL: A semantic web rule language combining OWL and RuleML. W3C Member submission 21, 79 (2004), 1--31. https://www.w3.org/submissions/SWRL/
[37]
HM Sajjad Hossain, MD Abdullah Al Haiz Khan, and Nirmalya Roy. 2018. DeActive: scaling activity recognition with active deep learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (2018), 1--23. https://doi.org/10.1145/3214269
[38]
Jan Humble, Andy Crabtree, Terry Hemmings, Karl-Petter Åkesson, Boriana Koleva, Tom Rodden, and Pär Hansson. 2003. "Playing with the bits" user-configuration of ubiquitous domestic environments. In International Conference on Ubiquitous Computing. Springer, 256--263. https://doi.org/10.1007/978-3-540-39653-6_20
[39]
Amy S Hwang, Khai N Truong, Jill I Cameron, Eva Lindqvist, Louise Nygård, and Alex Mihailidis. 2015. Co-designing ambient assisted living (AAL) environments: unravelling the situated context of informal dementia care. BioMed research international 2015 (2015). https://doi.org/10.1155/2015/720483
[40]
Jeya Vikranth Jeyakumar, Ankur Sarker, Luis Antonio Garcia, and Mani Srivastava. 2023. X-CHAR: A Concept-based Explainable Complex Human Activity Recognition Model. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 1 (2023), 1--28. https://doi.org/10.1145/3580804
[41]
Matthew Kay, Shwetak N Patel, and Julie A Kientz. 2015. How good is 85%? A survey tool to connect classifier evaluation to acceptability of accuracy. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 347--356. https://doi.org/10.1145/2702123.2702603
[42]
Sunnie SY Kim, Elizabeth Anne Watkins, Olga Russakovsky, Ruth Fong, and Andrés Monroy-Hernández. 2023. "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1--17. https://doi.org/10.1145/3544548.3581001
[43]
Amy J Ko and Brad A Myers. 2004. Designing the whyline: a debugging interface for asking questions about program behavior. In Proceedings of the SIGCHI conference on Human factors in computing systems. 151--158. https://doi.org/10.1145/985692.985712
[44]
Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces. 126--137. https://doi.org/10.1145/2678025.2701399
[45]
Ben Lafreniere, Tanya R. Jonker, Stephanie Santosa, Mark Parent, Michael Glueck, Tovi Grossman, Hrvoje Benko, and Daniel Wigdor. 2021. False Positives vs. False Negatives: The effects of recovery time and cognitive costs on input error preference. In The 34th Annual ACM Symposium on User Interface Software and Technology. 54--68. https://doi.org/10.1145/3472749.3474735
[46]
Rocco Langone, Alfredo Cuzzocrea, and Nikolaos Skantzos. 2020. Interpretable Anomaly Prediction: Predicting anomalous behavior in industry 4.0 settings via regularized logistic regression tools. Data & Knowledge Engineering 130 (2020), 101850. https://doi.org/10.1016/j.datak.2020.101850
[47]
Jisoo Lee, Luis Garduño, Erin Walker, and Winslow Burleson. 2013. A tangible programming tool for creation of context-aware applications. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. 391--400. https://doi.org/10.1145/2493432.2493483
[48]
Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. 2018. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32. https://doi.org/10.1609/aaai.v32i1.11771
[49]
Toby Jia-Jun Li, Yuanchun Li, Fanglin Chen, and Brad A Myers. 2017. Programming IoT devices by demonstration using mobile apps. In International Symposium on End User Development. Springer, 3--17. https://doi.org/10.1007/978-3-319-58735-6_1
[50]
Brian Y Lim and Anind K Dey. 2011. Design of an intelligible mobile context-aware application. In Proceedings of the 13th international conference on human computer interaction with mobile devices and services. 157--166. https://doi.org/10.1145/2037373.2037399
[51]
Brian Y Lim and Anind K Dey. 2013. Evaluating intelligibility usage and usefulness in a context-aware application. In International Conference on Human-Computer Interaction. Springer, 92--101. https://doi.org/10.1007/978-3-642-39342-6_11
[52]
Brian Y Lim, Anind K Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI conference on human factors in computing systems. 2119--2128. https://doi.org/10.1145/1518701.1519023
[53]
Zhipeng Luo and Milos Hauskrecht. 2019. Region-based active learning with hierarchical and adaptive region construction. In Proceedings of the 2019 SIAM International Conference on Data Mining. SIAM, 441--449. https://doi.org/10.1137/1.9781611975673.50
[54]
Nattaya Mairittha, Tittaya Mairittha, and Sozo Inoue. 2019. Optimizing activity data collection with gamification points using uncertainty based active learning. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 761--767. https://doi.org/10.1145/3341162.3345585
[55]
Alan Mazankiewicz, Klemens Böhm, and Mario Bergés. 2020. Incremental real-time personalization in human activity recognition using domain adaptive batch normalization. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1--20. https://doi.org/10.1145/3432230
[56]
Paul David McNicholas, Thomas Brendan Murphy, and M O'Regan. 2008. Standardising the lift of an association rule. Computational Statistics & Data Analysis 52, 10 (2008), 4712--4721. https://doi.org/10.1016/j.csda.2008.03.013
[57]
Lakmal Meegahapola, Florian Labhart, Thanh-Trung Phan, and Daniel Gatica-Perez. 2021. Examining the social context of alcohol drinking in young adults with smartphone sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--26. https://doi.org/10.1145/3478126
[58]
Alex Mihailidis, Joseph C Barbenel, and Geoff Fernie. 2004. The efficacy of an intelligent cognitive orthosis to facilitate handwashing by persons with moderate to severe dementia. Neuropsychological Rehabilitation 14, 1-2 (2004), 135--171. https://doi.org/10.1080/09602010343000156
[59]
Joshua Newn, Ryan M Kelly, Simon D'Alfonso, and Reeva Lederman. 2022. Examining and Promoting Explainable Recommendations for Personal Sensing Technology Acceptance. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--27. https://doi.org/10.1145/3550297
[60]
Mahsan Nourani, Chiradeep Roy, Tahrima Rahman, Eric D Ragan, Nicholas Ruozzi, and Vibhav Gogate. 2020. Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition. arXiv preprint arXiv:2005.02335 (2020). https://doi.org/10.48550/arXiv.2005.02335
[61]
Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115. https://doi.org/10.3390/s16010115
[62]
Aman Parnami, Apurva Gupta, Gabriel Reyes, Ramik Sadana, Yang Li, and Gregory D Abowd. 2016. Mogeste: A mobile tool for in-situ motion gesture design. In Proceedings of the 8th Indian Conference on Human Computer Interaction. 35--43. https://doi.org/10.1145/3014362.3014365
[63]
Betsy Phillips and Hongxin Zhao. 1993. Predictors of assistive technology abandonment. Assistive technology 5, 1 (1993), 36--45. https://doi.org/10.1080/10400435.1993.10132205
[64]
Pearl Pu and Li Chen. 2006. Trust building with explanation interfaces. In Proceedings of the 11th international conference on Intelligent user interfaces. 93--100. https://doi.org/10.1145/1111449.1111475
[65]
Priya Ranganathan, CS Pramesh, and Rakesh Aggarwal. 2017. Common pitfalls in statistical analysis: logistic regression. Perspectives in clinical research 8, 3 (2017), 148.
[66]
Parisa Rashidi and Diane J Cook. 2011. Ask me better questions: active learning queries based on rule induction. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. 904--912. https://doi.org/10.1145/2020408.2020559
[67]
Yili Ren, Sheng Tan, Linghan Zhang, Zi Wang, Zhi Wang, and Jie Yang. 2020. Liquid level sensing using commodity wifi in a smart home environment. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--30. https://doi.org/10.1145/3380996
[68]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135--1144. https://doi.org/10.1145/2939672.2939778
[69]
Daniele Riboni and Claudio Bettini. 2009. Context-aware activity recognition through a combination of ontological and statistical reasoning. In International Conference on Ubiquitous Intelligence and Computing. Springer, 39--53. https://doi.org/10.1007/978-3-642-02830-4_5
[70]
Daniele Riboni, Timo Sztyler, Gabriele Civitarese, and Heiner Stuckenschmidt. 2016. Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 1--12. https://doi.org/10.1145/2971648.2971691
[71]
Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, et al. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference on networked sensing systems (INSS). IEEE, 233--240. https://doi.org/10.1109/INSS.2010.5573462
[72]
Anna Rohrbach, Marcus Rohrbach, Wei Qiu, Annemarie Friedrich, Manfred Pinkal, and Bernt Schiele. 2014. Coherent multi-sentence video description with variable level of detail. In Pattern Recognition: 36th German Conference, GCPR 2014, Münster, Germany, September 2-5, 2014, Proceedings 36. Springer, 184--195. https://doi.org/10.1007/978-3-319-11752-2_15
[73]
Lena Rosenberg, Anders Kottorp, and Louise Nygård. 2012. Readiness for technology use with people with dementia: the perspectives of significant others. Journal of applied gerontology 31, 4 (2012), 510--530. https://doi.org/10.1177/0733464810396873
[74]
Chiradeep Roy, Mahesh Shanbhag, Mahsan Nourani, Tahrima Rahman, Samia Kabir, Vibhav Gogate, Nicholas Ruozzi, and Eric D Ragan. 2019. Explainable Activity Recognition in Videos. In IUI Workshops, Vol. 2.
[75]
Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1, 5 (2019), 206--215. https://doi.org/10.1038/s42256-019-0048-x
[76]
Cynthia Rudin and Joanna Radin. 2019. Why are we using black box models in AI when we don't need to. A lesson from an explainable AI competition. Harvard Data Science Review 1, 2 (2019), 1--9.
[77]
Roger C Schank and Robert P Abelson. 2013. Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Psychology press.
[78]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision. 618--626. https://doi.org/10.1109/ICCV.2017.74
[79]
Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. 2016. Hollywood in homes: Crowdsourcing data collection for activity understanding. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part 114. Springer, 510--526. https://doi.org/10.1007/978-3-319-46448-0_31
[80]
Geetika Singla, Diane J Cook, and Maureen Schmitter-Edgecombe. 2009. Tracking activities in complex settings using smart environment technologies. International journal of biosciences, psychiatry, and technology (IJBSPT) 1, 1 (2009), 25.
[81]
Dylan Slack, Sorelle A Friedler, Carlos Scheidegger, and Chitradeep Dutta Roy. 2019. Assessing the local interpretability of machine learning models. arXiv preprint arXiv:1902.03501 (2019).
[82]
Kehui Song, Xianyi Zeng, Ying Zhang, Julien De Jonckheere, Xiaojie Yuan, and Ludovic Koehl. 2021. An interpretable knowledge-based decision support system and its applications in pregnancy diagnosis. Knowledge-Based Systems 221 (2021), 106835. https://doi.org/10.1016/j.knosys.2021.106835
[83]
Kei Tanigaki, Tze Chuin Teoh, Naoya Yoshimura, Takuya Maekawa, and Takahiro Hara. 2022. Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--33. https://doi.org/10.1145/3550319
[84]
Son N Tran, Qing Zhang, Vanessa Smallbon, and Mohan Karunanithi. 2018. Multi-resident activity monitoring in smart homes: A case study. In 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 698--703. https://doi.org/10.1109/PERCOMW.2018.8480132
[85]
Khai N Truong, Elaine M Huang, and Gregory D Abowd. 2004. CAMP: A magnetic poetry interface for end-user programming of capture applications for the home. In International conference on ubiquitous computing. Springer, 143--160. https://doi.org/10.1007/978-3-540-30119-6_9
[86]
Chun-Hua Tsai and Peter Brusilovsky. 2019. Explaining recommendations in an interactive hybrid social recommender. In Proceedings of the 24th international conference on intelligent user interfaces. 391--396. https://doi.org/10.1145/3301275.3302318
[87]
Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Ke Huo, Yuanzhi Cao, and Karthik Ramani. 2020. CAPturAR: An augmented reality tool for authoring human-involved context-aware applications. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 328--341. https://doi.org/10.1145/3379337.3415815
[88]
Colin Ware. 2019. Information visualization: perception for design. Morgan Kaufmann.
[89]
Joseph P Wherton and Andrew F Monk. 2008. Technological opportunities for supporting people with dementia who are living at home. International Journal of Human-Computer Studies 66, 8 (2008), 571--586. https://doi.org/10.1016/j.ijhcs.2008.03.001
[90]
Tianwei Xing, Luis Garcia, Marc Roig Vilamala, Federico Cerutti, Lance Kaplan, Alun Preece, and Mani Srivastava. 2020. Neuroplex: learning to detect complex events in sensor networks through knowledge injection. In Proceedings of the 18th conference on embedded networked sensor systems. 489--502. https://doi.org/10.1145/3384419.3431158
[91]
Yangdi Xu, David Bull, and Dima Damen. 2017. Unsupervised long-term routine modelling using dynamic bayesian networks. In 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE, 1--8.
[92]
Yangdi Xu and Dima Damen. 2018. Human routine change detection using bayesian modelling. In 2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 1833--1838. https://doi.org/10.1109/DICTA.2017.8227502
[93]
Juan Ye, Graeme Stevenson, and Simon Dobson. 2014. USMART: An unsupervised semantic mining activity recognition technique. ACM Transactions on Interactive Intelligent Systems (TiiS) 4, 4 (2014), 1--27. https://doi.org/10.1145/2662870
[94]
Chien Wen Yuan, Nanyi Bi, Ya-Fang Lin, and Yuen-Hsien Tseng. [n. d.]. Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1--15. https://doi.org/10.1145/3544548.3580945
[95]
Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. 2014. Convolutional neural networks for human activity recognition using mobile sensors. In 6th international conference on mobile computing, applications and services. IEEE, 197--205. https://doi.org/10.4108/icst.mobicase.2014.257786
[96]
Lefan Zhang, Cyrus Zhou, Michael L Littman, Blase Ur, and Shan Lu. 2023. Helping Users Debug Trigger-Action Programs. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 4 (2023), 1--32. https://doi.org/10.1145/3569506
[97]
Tao Zhuo, Zhiyong Cheng, Peng Zhang, Yongkang Wong, and Mohan Kankanhalli. 2019. Explainable video action reasoning via prior knowledge and state transitions. In Proceedings of the 27th acm international conference on multimedia. 521--529. https://doi.org/10.1145/3343031.3351040
Index Terms
- exHAR: An Interface for Helping Non-Experts Develop and Debug Knowledge-based Human Activity Recognition Systems
Recommendations
EPARS: elderly physical activity reminder system using smartphone and wearable sensors
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable ComputersThe sedentary lifestyle of elderly people may cause chronic diseases, reduce muscle mass and mobility. Moreover, elderly physical inactivity increases the susceptibility to mood disorders, such as anxiety and depression, and can permanently lead to ...
Recognition of coupling-paired activities in daily life
HCCE '12: Proceedings of the 2012 Joint International Conference on Human-Centered Computer EnvironmentsRecognition of daily activities is one of major focuses in health care systems. But limited number of studies has been done on recognition of complex human activities in daily life. Therefore in this work an attempt has been done to recognize complex ...
Enabling Personalisation of Remote Elderly Assistant Applications
CHItaly '17: Proceedings of the 12th Biannual Conference on Italian SIGCHI ChapterOne of the goals of Ambient Assisted Living (AAL) solutions is to extend the time that elderly people can live independently in their preferred environments by using ICT technologies for personal healthcare. However, in order to be optimal, remote ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 March 2024
Published in IMWUT Volume 8, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 206Total Downloads
- Downloads (Last 12 months)206
- Downloads (Last 6 weeks)25
Reflects downloads up to 15 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in