Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Interactive Recommendation with User-Specific Deep Reinforcement Learning

Interactive Recommendation with User-Specific Deep Reinforcement Learning In this article, we study a multi-step interactive recommendation problem for explicit-feedback recommender systems. Different from the existing works, we propose a novel user-specific deep reinforcement learning approach to the problem. Specifically, we first formulate the problem of interactive recommendation for each target user as a Markov decision process (MDP). We then derive a multi-MDP reinforcement learning task for all involved users. To model the possible relationships (including similarities and differences) between different users’ MDPs, we construct user-specific latent states by using matrix factorization. After that, we propose a user-specific deep Q-learning (UDQN) method to estimate optimal policies based on the constructed user-specific latent states. Furthermore, we propose Biased UDQN (BUDQN) to explicitly model user-specific information by employing an additional bias parameter when estimating the Q-values for different users. Finally, we validate the effectiveness of our approach by comprehensive experimental results and analysis. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Knowledge Discovery from Data (TKDD) Association for Computing Machinery

Interactive Recommendation with User-Specific Deep Reinforcement Learning

Download PDF
15 pages

Loading next page...
 
/lp/association-for-computing-machinery/interactive-recommendation-with-user-specific-deep-reinforcement-62GzI4xtkm

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2019 ACM
ISSN
1556-4681
eISSN
1556-472X
DOI
10.1145/3359554
Publisher site
See Article on Publisher Site

Abstract

In this article, we study a multi-step interactive recommendation problem for explicit-feedback recommender systems. Different from the existing works, we propose a novel user-specific deep reinforcement learning approach to the problem. Specifically, we first formulate the problem of interactive recommendation for each target user as a Markov decision process (MDP). We then derive a multi-MDP reinforcement learning task for all involved users. To model the possible relationships (including similarities and differences) between different users’ MDPs, we construct user-specific latent states by using matrix factorization. After that, we propose a user-specific deep Q-learning (UDQN) method to estimate optimal policies based on the constructed user-specific latent states. Furthermore, we propose Biased UDQN (BUDQN) to explicitly model user-specific information by employing an additional bias parameter when estimating the Q-values for different users. Finally, we validate the effectiveness of our approach by comprehensive experimental results and analysis.

Journal

ACM Transactions on Knowledge Discovery from Data (TKDD) – Association for Computing Machinery

Published: Oct 15, 2019

Keywords: Interactive recommendation

References

  • Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions
    Adomavicius, G.; Tuzhilin, A.
  • Tell me who I am: An interactive recommendation system
    Alon, N.; Awerbuch, B.; Azar, Y.; Patt-Shamir, B.
  • Finite-time analysis of the multiarmed bandit problem
    Auer, P.; Cesa-Bianchi, N.; Fischer, P.
  • Dynamic programming
    Bellman, R.
  • Bandit problems: Sequential allocation of experiments
    Berry, D. A.; Fristedt, B.
  • Stabilizing reinforcement learning in dynamic environment with application to online recommendation
    Chen, S.; Yu, Y.; Da, Q.; Tan, J.; Huang, H.; Tang, H.
  • Towards conversational recommender systems
    Christakopoulou, K.; Radlinski, F.; Hofmann, K.
  • Performance of recommender algorithms on top-n recommendation tasks
    Cremonesi, P.; Koren, Y.; Turrin, R.
  • Combining usage and content in an online recommendation system for music in the long tail
    Domingues, M. A.; Gouyon, F.; Jorge, A. M.; Leal, J. P.; Vinagre, J.; Lemos, L.; Sordo, M.
  • Multi-armed Bandit Allocation Indices
    Gittins, J.; Glazebrook, K.; Weber, R.
  • Fast matrix factorization for online recommendation with implicit feedback
    He, X.; Zhang, H.; Kan, M.; Chua, T.
  • Playlist recommendation based on reinforcement learning
    Hu, B.; Shi, C.; Liu, J.
  • Efficient thompson sampling for online matrix-factorization recommendation
    Kawale, J.; Bui, H. H.; Kveton, B.; Tran-Thanh, L.; Chawla, S.
  • Factorization meets the neighborhood: A multifaceted collaborative filtering model
    Koren, Y.
  • Advances in collaborative filtering
    Koren, Y.; Bell, R.
  • Matrix factorization techniques for recommender systems
    Koren, Y.; Bell, R.; Volinsky, C.
  • A contextual-bandit approach to personalized news article recommendation
    Li, L.; Chu, W.; Langford, J.; Schapire, R. E.
  • Reinforcement Learning for Robots Using Neural Networks
    Lin, L.
  • Human-level control through deep reinforcement learning
    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; Hassabis, D.
  • Exploiting temporal influence in online recommendation
    Pálovics, R.; Benczúr, A. A.; Kocsis, L.; Kiss, T.; Frigó, E.
  • Introduction to recommender systems handbook
    Ricci, F.; Rokach, L.; Shapira, B.
  • An MDP-based recommender system
    Shani, G.; Heckerman, D.; Brafman, R. I.
  • Mastering the game of Go with deep neural networks and tree search
    Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; Dieleman, S.; Grewe, D.; Nham, J.; Kalchbrenner, N.; Sutskever, I.; Lillicrap, T.; Leach, M.; Kavukcuoglu, K.; Graepel, T.; Hassabis, D.
  • Reinforcement Learning: An Introduction, Vol
    Sutton, R. S.; Barto, A. G.
  • Usage-based web recommendations: A reinforcement learning approach
    Taghipour, N.; Kardan, A.; Ghidary, S. S.
  • Personalized recommendation via parameter-free contextual bandits
    Tang, L.; Jiang, Y.; Li, L.; Zeng, C.; Li, T.
  • Explore-exploit in top-n recommender systems via gaussian processes
    Vanchinathan, H. P.; Nikolic, I.; De Bona, F.; Krause, A.
  • Exploration in interactive personalized music recommendation: A reinforcement learning approach
    Wang, X.; Wang, Y.; Hsu, D.; Wang, Y.
  • Q-learning
    Watkins, C. J.; Dayan, P.
  • Deep reinforcement learning for page-wise recommendations
    Zhao, X.; Xia, L.; Zhang, L.; Ding, Z.; Yin, D.; Tang, J.
  • Recommendations with negative feedback via pairwise deep reinforcement learning
    Zhao, X.; Zhang, L.; Ding, Z.; Xia, L.; Tang, J.; Yin, D.
  • Interactive collaborative filtering
    Zhao, X.; Zhang, W.; Wang, J.
  • DRN: A deep reinforcement learning framework for news recommendation
    Zheng, G.; Zhang, F.; Zheng, Z.; Xiang, Y.; Yuan, N. J.; Xie, X.; Li, Z.