research-article

Double Attention Convolutional Neural Network for Sequential Recommendation

Authors:

Deqing ZouAuthors Info & Claims

ACM Transactions on the Web, Volume 16, Issue 4

Article No.: 19, Pages 1 - 23

https://doi.org/10.1145/3555350

Published: 08 December 2022 Publication History

Abstract

The explosive growth of e-commerce and online service has led to the development of recommender system. Aiming to provide a list of items to meet a user’s personalized need by analyzing his/her interaction history, recommender system has been widely studied in academic and industrial communities. Different from conventional recommender systems, sequential recommender systems attempt to capture the pattern of users’ sequential behaviors and the evolution of users’ preferences. Most of the existing sequential recommendation models only focus on user interaction sequence, but neglect item interaction sequence. An item interaction sequence also contains rich contextual information for capturing the item’s dynamic characteristic, since an item’s dynamic characteristic can be reflected by the users who interact with it in a period. Furthermore, existing dual sequential models use the same method to handle the user interaction sequence and item interaction sequence, and do not consider their different characteristics. Hence, we propose a novel Double Attention Convolution Neural Network (DACNN), which incorporates user interaction sequence and item interaction sequence into an integrated neural network framework. DACNN leverages the strength of attention mechanism to capture the temporary suitability and adopts CNN to extract local sequential features. Experimental evaluations on the real datasets show that DACNN outperforms the baseline approaches.

References

[1]

Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 6 (2005), 734–749.

Digital Library

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

[3]

Robert M. Bell and Yehuda Koren. 2007. Improved neighborhood-based collaborative filtering. In KDD Cup and Workshop at the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Citeseer, 7–14.

[4]

Veronika Bogina and Tsvi Kuflik. 2017. Incorporating dwell time in session-based recommendations with recurrent neural networks. In RecTemp@RecSys (CEUR Workshop Proceedings), Vol. 1922. CEUR-WS.org, 57–59.

[5]

Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In SIGIR. ACM, 335–344.

[6]

Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng, Xue Li, and Xiaofang Zhou. 2020. Sequence-aware factorization machines for temporal predictive analytics. In ICDE. IEEE, 1405–1416.

[7]

Florent Garcin, Christos Dimitrakakis, and Boi Faltings. 2013. Personalized news recommendation with context trees. In RecSys. ACM, 105–112.

[8]

Abhijeet Ghoshal, Syam Menon, and Sumit Sarkar. 2015. Recommendations using information from multiple association rules: A probabilistic approach. Inf. Syst. Res. 26, 3 (2015), 532–551.

Digital Library

[9]

Abhijeet Ghoshal and Sumit Sarkar. 2014. Association rules for recommendations with multiple items. INFORMS J. Comput. 26, 3 (2014), 433–448.

[10]

Yuyun Gong and Qi Zhang. 2016. Hashtag recommendation using attention-based convolutional neural network. In IJCAI. IJCAI/AAAI Press, 2782–2788.

[11]

Lei Guo, Hongzhi Yin, Qinyong Wang, Tong Chen, Alexander Zhou, and Nguyen Quoc Viet Hung. 2019. Streaming session-based recommendation. In KDD. ACM, 1569–1577.

[12]

Ruining He and Julian J. McAuley. 2016. Fusing similarity models with Markov chains for sparse sequential recommendation. In ICDM. IEEE Computer Society, 191–200.

[13]

Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural attentive item similarity model for recommendation. IEEE Trans. Knowl. Data Eng. 30, 12 (2018), 2354–2366.

Digital Library

[14]

Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. 2016. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In RecSys. ACM, 241–248.

[15]

Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, and Changsheng Xu. 2018. CSAN: Contextual self-attention network for user sequential recommendation. In ACM Multimedia. ACM, 447–455.

[16]

Mingi Ji, Weonyoung Joo, Kyungwoo Song, Yoon-Yeong Kim, and Il-Chul Moon. 2020. Sequential recommendation with relation-aware kernelized self-attention. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI’20, New York, NY, USA, February 7-12, 2020. AAAI Press, 4304–4311.

[17]

Thorsten Joachims, Dayne Freitag, and Tom M. Mitchell. 1997. Web Watcher: A tour guide for the world wide web. In IJCAI (1). Morgan Kaufmann, 770–777.

[18]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster).

[19]

Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. IEEE Computer 42, 8 (2009), 30–37.

Digital Library

[20]

Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In KDD. ACM, 1269–1278.

[21]

Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for non-negative matrix factorization. In NIPS. MIT Press, 556–562.

Digital Library

[22]

Chenliang Li, Xichuan Niu, Xiangyang Luo, Zhenzhong Chen, and Cong Quan. 2019. A review-driven neural model for sequential recommendation. In IJCAI. ijcai.org, 2866–2872.

[23]

Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In CIKM. ACM, 1419–1428.

[24]

Jian Liu, Pengpeng Zhao, Yanchi Liu, Jiajie Xu, Junhua Fang, Lei Zhao, Victor S. Sheng, and Zhiming Cui. 2019. Attention and convolution enhanced memory network for sequential recommendation. In DASFAA (2) (Lecture Notes in Computer Science), Vol. 11447. Springer, 333–349.

[25]

Kuan Liu, Xing Shi, Anoop Kumar, Linhong Zhu, and Prem Natarajan. 2016. Temporal learning and sequence modeling for a job recommender system. In RecSys. Challenge. ACM, 7:1–7:4.

[26]

Qiang Liu, Shu Wu, Diyi Wang, Zhaokang Li, and Liang Wang. 2016. Context-aware sequential recommendation. In ICDM. IEEE Computer Society, 1053–1058.

[27]

Yining Liu, Yong Liu, Yanming Shen, and Keqiu Li. 2018. Recommendation in a changing world: Exploiting temporal dynamics in ratings and reviews. ACM Trans. Web 12, 1 (2018), 3:1–3:20.

Digital Library

[28]

Samaneh Moghaddam, Mohsen Jamali, and Martin Ester. 2012. ETF: Extended tensor factorization model for personalizing prediction of review helpfulness. In WSDM. ACM, 163–172.

[29]

Raymond J. Mooney and Loriene Roy. 2000. Content-based book recommending using learning for text categorization. In ACM DL. ACM, 195–204.

[30]

Rong Pan, Yunhong Zhou, Bin Cao, Nathan Nan Liu, Rajan M. Lukose, Martin Scholz, and Qiang Yang. 2008. One-class collaborative filtering. In ICDM. IEEE Computer Society, 502–511.

[31]

Michael J. Pazzani, Jack Muramatsu, and Daniel Billsus. 1996. Syskill & Webert: Identifying interesting web sites. In AAAI/IAAI, Vol. 1. AAAI Press/The MIT Press, 54–61.

[32]

Jiarui Qin, Kan Ren, Yuchen Fang, Weinan Zhang, and Yong Yu. 2020. Sequential recommendation with dual side neighbor-based collaborative relation modeling. In WSDM. ACM, 465–473.

[33]

Ruihong Qiu, Zi Huang, Jingjing Li, and Hongzhi Yin. 2020. Exploiting cross-session information for session-based recommendation with graph neural networks. ACM Trans. Inf. Syst. 38, 3 (2020), 22:1–22:23.

Digital Library

[34]

Ruihong Qiu, Hongzhi Yin, Zi Huang, and Tong Chen. 2020. GAG: Global attributed graph neural network for streaming session-based recommendation. In SIGIR. ACM, 669–678.

[35]

Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In RecSys. ACM, 130–137.

[36]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. (2009), 452–461.

[37]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized Markov chains for next-basket recommendation. (2010), 811–820.

[38]

Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic matrix factorization. In NIPS. Curran Associates, Inc., 1257–1264.

[39]

Badrul Sarwar. 2001. Item-based collaborative filtering recommendation algorithms. In Proc. International World Wide Web Conference.

Digital Library

[40]

Sylvain Senecal and Jacques Nantel. 2004. The influence of online product recommendations on consumers’ online choices. Journal of Retailing 80, 2 (2004), 159–169.

[41]

Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929–1958.

Digital Library

[42]

Xiaoyuan Su and Taghi M. Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Adv. Artificial Intellegence 2009 (2009), 421425:1–421425:19.

[43]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In NIPS. 3104–3112.

[44]

Jiaxi Tang and Ke Wang. 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. In WSDM. ACM, 565–573.

[45]

Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet A. Orgun. 2019. Sequential recommender systems: Challenges, progress and prospects. In IJCAI. ijcai.org, 6332–6338.

[46]

Kasun Wickramaratna, Miroslav Kubat, and Kamal Premaratne. 2009. Predicting missing items in shopping carts. IEEE Trans. Knowl. Data Eng. 21, 7 (2009), 985–998.

Digital Library

[47]

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J. Smola, and How Jing. 2017. Recurrent recommender networks. In WSDM. ACM, 495–503.

[48]

Qitian Wu, Yirui Gao, Xiaofeng Gao, Paul Weng, and Guihai Chen. 2019. Dual sequential prediction models linking sequential recommendation and information dissemination. In KDD. ACM, 447–457.

[49]

Yuehong Wu and Juan Yang. 2021. Dual sequential recommendation integrating high-order collaborative relations via graph attention networks. In International Joint Conference on Neural Networks, IJCNN’21, Shenzhen, China, July 18-22, 2021. IEEE, 1–8.

[50]

Zhiang Wu, Changsheng Li, Jie Cao, and Yong Ge. 2020. On scalability of association-rule-based recommendation: A unified distributed-computing framework. ACM Trans. Web 14, 3 (2020), 13:1–13:21.

Digital Library

[51]

Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2021. Self-supervised hypergraph convolutional networks for session-based recommendation. In AAAI. AAAI Press, 4503–4511.

[52]

Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Jiajie Xu, Victor S. Sheng, Zhiming Cui, Xiaofang Zhou, and Hui Xiong. 2019. Recurrent convolutional neural network for sequential recommendation. In WWW. ACM, 3398–3404.

[53]

Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016. A dynamic recurrent model for next basket recommendation. In SIGIR. ACM, 729–732.

[54]

Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. 2018. Next item recommendation with self-attention. CoRR abs/1808.06414.

[55]

Sheng Zhang, Weihong Wang, James Ford, Fillia Makedon, and Justin D. Pearlman. 2005. Using singular value decomposition approximation for collaborative filtering. In CEC. IEEE Computer Society, 257–264.

[56]

Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, and Xiaofang Zhou. 2019. Feature-level deeper self-attention network for sequential recommendation. In IJCAI. ijcai.org, 4320–4326.

[57]

Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential click prediction for sponsored search with recurrent neural networks. In AAAI. AAAI Press, 1369–1375.

[58]

Chang Zhou, Jinze Bai, Junshuai Song, Xiaofei Liu, Zhengchao Zhao, Xiusi Chen, and Jun Gao. 2018. ATRank: An attention-based user behavior modeling framework for recommendation. In AAAI. AAAI Press, 4564–4571.

Cited By

Tang XChen LShi HLyu D(2024)DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge GraphsACM Transactions on Information Systems10.1145/365301542:5(1-23)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3653015
Liu YWalder CXie LLiu YBaeza-Yates RBonchi F(2024)Probabilistic Attention for Sequential RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671733(1956-1967)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671733
Han DHuang YLiu JLiao KLin K(2024)LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention BiasACM Transactions on Knowledge Discovery from Data10.1145/363262518:3(1-20)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3632625
Show More Cited By

Index Terms

Double Attention Convolutional Neural Network for Sequential Recommendation
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems

Recommendations

Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Top-N sequential recommendation models each user as a sequence of items interacted in the past and aims to predict top-N ranked items that a user will likely interact in a »near future». The order of interaction implies that sequential patterns play an ...
Improving Sequential Recommendation with Attribute-Augmented Graph Neural Networks
Advances in Knowledge Discovery and Data Mining
Abstract
Many practical recommender systems provide item recommendation for different users only via mining user-item interactions but totally ignoring the rich attribute information of items that users interact with. In this paper, we propose an attribute-...
Sirius: Sequential Recommendation with Feature Augmented Graph Neural Networks
Database Systems for Advanced Applications
Abstract
Many practical recommender systems recommend personalized items for different users by mining user-item interaction sequences. The interaction sequences, as a whole, imply the manifold collaborative relations among users and items. Further, from ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 16, Issue 4

November 2022

165 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3571715

Editor:
Ryen White
Microsoft Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 December 2022

Online AM: 09 August 2022

Accepted: 08 July 2022

Revised: 17 June 2022

Received: 11 July 2021

Published in TWEB Volume 16, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
363
Total Downloads

Downloads (Last 12 months)142
Downloads (Last 6 weeks)12

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tang XChen LShi HLyu D(2024)DHyper: A Recurrent Dual Hypergraph Neural Network for Event Prediction in Temporal Knowledge GraphsACM Transactions on Information Systems10.1145/365301542:5(1-23)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3653015
Liu YWalder CXie LLiu YBaeza-Yates RBonchi F(2024)Probabilistic Attention for Sequential RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671733(1956-1967)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671733
Han DHuang YLiu JLiao KLin K(2024)LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention BiasACM Transactions on Knowledge Discovery from Data10.1145/363262518:3(1-20)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3632625
Zhang RLiu NLiu LDong MOta KYan Z(2024)Performance Evaluation of Self-Similar GPS Scheduling: A Knowledge-Driven Dual-Task Deep Learning ApproachIEEE Transactions on Cognitive Communications and Networking10.1109/TCCN.2023.332898610:2(510-525)Online publication date: Apr-2024
https://doi.org/10.1109/TCCN.2023.3328986
Marish Kumar PPriya NDhilipkumar RSanthana Krishnan T(2024)Maximizing solar energy efficiency with efficient interleaved boost converter with Three-Level neutral point clamped inverter for Grid-Connected photovoltaic using hybrid approachSolar Energy10.1016/j.solener.2024.112855280(112855)Online publication date: Sep-2024
https://doi.org/10.1016/j.solener.2024.112855
Liu WZhang ZDing YWang B(2024)Global heterogeneous graph enhanced category-aware attention network for session-based recommendationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122907243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122907
Patel RThakkar PUkani V(2024)CNNRecEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108062133:PAOnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.108062
Li CIshak IIbrahim HZolkepli MSidi FLi C(2023)Deep Learning-Based Recommendation System: Systematic Review and ClassificationIEEE Access10.1109/ACCESS.2023.332335311(113790-113835)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3323353

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents