article

Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships

Authors:

Mathias Niepert,

Heiner StuckenschmidtAuthors Info & Claims

Pervasive and Mobile Computing, Volume 7, Issue 6

Pages 660 - 670

https://doi.org/10.1016/j.pmcj.2011.08.004

Published: 01 December 2011 Publication History

Abstract

The majority of approaches to activity recognition in sensor environments are either based on manually constructed rules for recognizing activities or lack the ability to incorporate complex temporal dependencies. Furthermore, in many cases, the rather unrealistic assumption is made that the subject carries out only one activity at a time. In this paper, we describe the use of Markov logic as a declarative framework for recognizing interleaved and concurrent activities incorporating both input from pervasive lightweight sensor technology and common-sense background knowledge. In particular, we assess its ability to learn statistical-temporal models from training data and to combine these models with background knowledge to improve the overall recognition accuracy. We also show the viability and the benefit of exploiting both qualitative and quantitative temporal relationships like the duration of the activities and their temporal order. To this end, we propose two Markov logic formulations for inferring the foreground activity as well as each activities' start and end times. We evaluate the approach on an established dataset where it outperforms state-of-the-art algorithms for activity recognition.

References

[1]

Hu, D. Hao, Pan, S.J., Zheng, V.W., Liu, N.N. and Yang, Q., Real world activity recognition with multiple goals. In: UbiComp'08, ACM, New York, NY, USA. pp. 30-39.

[2]

M. Cirillo, F. Lazellotto, F. Pecora, A. Saffiotti, Monitoring domestic activities with temporal constraints and components, in: Proc. of the 5th Int. Conf. on Intelligent Environments, pp. 117-124.

[3]

R. Biswas, S. Thrun, K. Fujimura, Recognizing activities with multiple cues, in: Workshop on Human Motion, pp. 255-270.

[4]

S.D. Tran, L.S. Davis, Event modeling and recognition using Markov logic networks, in: Proceedings of the 10th European Conference on Compute Vision, pp. 610-623.

[5]

Buettner, M., Prasad, R., Philipose, M. and Wetherall, D., Recognizing daily activities with rfid-based sensors. In: Proceedings of the 11th International Conference on Ubiquitous Computing, ACM. pp. 51-60.

[6]

Patterson, D.J., Fox, D., Kautz, H. and Philipose, M., Fine-grained activity recognition by aggregating abstract object usage. In: Proceedings of the Ninth IEEE International Symposium on Wearable Computers, IEEE Computer Society.

Digital Library

[7]

Stikic, M. and Schiele, B., Activity recognition from sparsely labeled data using multi-instance learning. In: LoCA, Springer. pp. 156-173.

[8]

Gu, T., Chen, S., Tao, X. and Lu, J., An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data & Knowledge Engineering. v69. 533-544.

[9]

Jeon, H., Kim, T. and Choi, J., Ontology-based user intention recognition for proactive planning of intelligent robot behavior. In: MUE'08: Proceedings of the 2008 International Conference on Multimedia and Ubiquitous Engineering, IEEE Computer Society, Washington, DC, USA. pp. 244-248.

Digital Library

[10]

Galata, A., Johnson, N. and Hogg, D., Learning variable-length Markov models of behavior. Computer Vision and Image Understanding. 398-413.

[11]

T. Duong, H. Bui, D. Phung, S. Venkatesh, Activity recognition and abnormality detection with the switching hidden semi-Markov model, in: Computer Vision and Pattern Recognition, pp. 838-845.

Digital Library

[12]

Richardson, M. and Domingos, P., Markov logic networks. Machine Learning. v62. 107-136.

[13]

R. Helaoui, M. Niepert, H. Stuckenschmidt, A statistical-relational activity recognition framework for ambient assisted living systems, in: Ambient Intelligence and Future Trends-International Symposium on Ambient Intelligence.

[14]

R. Helaoui, M. Niepert, H. Stuckenschmidt, Recognizing interleaved and concurrent activities, in: Proceedings of the Ninth Annual IEEE International Conference on Pervasive Computing and Communications.

[15]

Kim, E., Helal, S. and Cook, D., Human activity recognition and pattern discovery. IEEE Pervasive Computing. v9. 48-53.

[16]

D.H. Hu, Q. Yang, Cigar: concurrent and interleaving goal and activity recognition, in: AAAI'08: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, pp. 1363-1368.

[17]

McCallum, A. and Sutton, C., An introduction to conditional random fields for relational learning. In: Getoor, L., Taskar, B. (Eds.), Introduction to Statistical Relational Learning, MIT Press.

[18]

J. Modayil, T. Bai, H. Kautz, Improving the recognition of interleaved activities, in: UbiComp'08: Proceedings of the 10th International Conference on Ubiquitous Computing, pp. 40-43.

[19]

D. Riboni, C. Bettini, Context-aware activity recognition through a combination of ontological and statistical reasoning, in: UIC'09, pp. 39-53.

[20]

D. Riboni, L. Pareschi, L. Radaelli, C. Bettini, Is ontology-based activity recognition really effective?, in: PerCom Workshops, pp. 427-431.

[21]

Getoor, L. and Taskar, B., Introduction to Statistical Relational Learning. 2007. MIT Press.

[22]

Koller, D. and Friedman, N., Probabilistic Graphical Models. 2009. MIT Press.

[23]

F.J. Och, H. Ney, Discriminative training and maximum entropy models for statistical machine translation, in: Proceedings of the 40th Meeting of the ACL.

[24]

S. Riedel, I. Meza-Ruiz, Collective semantic role labelling with Markov logic, in: Proceedings of CoNLL.

[25]

M. Niepert, C. Meilicke, H. Stuckenschmidt, A probabilistic-logical framework for ontology matching, in: Proceedings of the 24th AAAI Conference on Artificial Intelligence.

[26]

Creignou, N., Khanna, S. and Sudan, M., Complexity Classifications of Boolean Constraint Satisfaction Problems. 2001. SIAM.

[27]

D. Roth, W. Yih, Integer linear programming inference for conditional random fields, in: Proceedings of the International Conference on Machine Learning, pp. 736-743.

[28]

B. Taskar, Learning structured prediction models: a large-margin approach, Ph.D. Thesis, Stanford University, 2004.

[29]

S. Riedel, Improving the accuracy and efficiency of map inference for Markov logic, in: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 468-475.

[30]

M. Niepert, A delayed column generation strategy for exact k-bounded map inference in Markov logic networks, in: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence.

[31]

Schrijver, A., Theory of Linear and Integer Programming. 1998. Wiley & Sons.

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Safyan MQayyum ZSarwar SGarcía-Castro RAhmed M(2019)Ontology-driven semantic unified modelling for concurrent activity recognition (OSCAR)Multimedia Tools and Applications10.1007/s11042-018-6318-578:2(2073-2104)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1007/s11042-018-6318-5
Kolasa YBastogne TGeorges JKostkova PGrasso FCastillo CMejova YBosman AEdelstein M(2018)Simulation and Sensitivity Analysis of Sensors Network for Cardiac MonitoringProceedings of the 2018 International Conference on Digital Health10.1145/3194658.3194686(148-149)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3194658.3194686
Show More Cited By

Index Terms

Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships
1. Computing methodologies
  1. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Recognising activities of daily living from patterns of object use

To provide assistance and support to the elderly disabled and cognitively impaired, the recognition of their activities of daily living (ADL) must be accurate and precise with regards to the object use for the activity situations. Current ...
Recognizing Concurrent and Interleaved Activities in Social Interactions
DASC '11: Proceedings of the 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing

Social networks constitute an important research area where users are involved in social interactions and inform each other of activities they perform. In this paper a complex activity recognition system (SEMACT) is proposed, built and validated. ...
Latent-Dynamic Conditional Random Fields for recognizing activities in smart homes
Ambient and Smart Component Technologies for Human Centric Computing

As the number of elderly people in our society increases, the need of assistive technologies in home becomes urgent. Existing techniques allow elderly people to be better assisted through monitoring what goes on in smart homes and inferring their ...

Comments

Information & Contributors

Information

Published In

cover image Pervasive and Mobile Computing

Pervasive and Mobile Computing Volume 7, Issue 6

December, 2011

120 pages

ISSN:1574-1192

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2011.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 December 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Safyan MQayyum ZSarwar SGarcía-Castro RAhmed M(2019)Ontology-driven semantic unified modelling for concurrent activity recognition (OSCAR)Multimedia Tools and Applications10.1007/s11042-018-6318-578:2(2073-2104)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1007/s11042-018-6318-5
Kolasa YBastogne TGeorges JKostkova PGrasso FCastillo CMejova YBosman AEdelstein M(2018)Simulation and Sensitivity Analysis of Sensors Network for Cardiac MonitoringProceedings of the 2018 International Conference on Digital Health10.1145/3194658.3194686(148-149)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3194658.3194686
Orr CNugent CWang HZheng HKostkova PGrasso FCastillo CMejova YBosman AEdelstein M(2018)A Multi Agent Approach to Facilitate the Identification of Interleaved ActivitiesProceedings of the 2018 International Conference on Digital Health10.1145/3194658.3194684(126-130)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3194658.3194684
Nakahara KYamaguchi HHigashino THara TShigeno H(2016)In-home Activity and Micro-motion Logging Using Mobile Robot with KinectAdjunct Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing Networking and Services10.1145/3004010.3004027(106-111)Online publication date: 28-Nov-2016
https://dl.acm.org/doi/10.1145/3004010.3004027
Helaoui RRiboni DStuckenschmidt HMattern FSantini SCanny JLangheinrich MRekimoto J(2013)A probabilistic ontological framework for the recognition of multilevel human activitiesProceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing10.1145/2493432.2493501(345-354)Online publication date: 8-Sep-2013
https://dl.acm.org/doi/10.1145/2493432.2493501
Saguna SZaslavsky AChakraborty D(2013)Complex activity recognition using context-driven activity theory and activity signaturesACM Transactions on Computer-Human Interaction10.1145/249083220:6(1-34)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1145/2490832
Okeyo GChen LWang HSterritt R(2012)A knowledge-driven approach to composite activity recognition in smart environmentsProceedings of the 6th international conference on Ubiquitous Computing and Ambient Intelligence10.1007/978-3-642-35377-2_44(322-329)Online publication date: 3-Dec-2012
https://dl.acm.org/doi/10.1007/978-3-642-35377-2_44

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents