Awesome Monte Carlo Tree Search Papers.

---

A curated list of Monte Carlo tree search papers with implementations from the following conferences/journals:

• Machine learning
  • NeurIPS
  • ICML
• Computer vision
  • CVPR
  • ICCV
• Natural language processing
  • ACL
• Data
  • KDD
• Artificial intelligence
  • AAAI
  • AISTATS
  • IJCAI
  • UAI
• Robotics
  • RAS
• Games
  • CIG

Similar collections about graph classification, gradient boosting, classification/regression trees, fraud detection, and community detection papers with implementations.

2023

• Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search (ICLR 2023)
  • Fangzheng Sun, Yang Liu, Jian-Xun Wang, Hao Sun
  • [Paper]

2022

• Finding Backdoors to Integer Programs: A Monte Carlo Tree Search Framework (AAAI 2022)

  • Elias B. Khalil, Pashootan Vaezipoor, Bistra Dilkina
  • [Paper]

• NSGZero: Efficiently Learning Non-exploitable Policy in Large-Scale Network Security Games with Neural Monte Carlo Tree Search (AAAI 2022)

  • Wanqi Xue, Bo An, Chai Kiat Yeo
  • [Paper]

• Solving Disjunctive Temporal Networks with Uncertainty under Restricted Time-Based Controllability Using Tree Search and Graph Neural Networks (AAAI 2022)

  • Kevin Osanlou, Jeremy Frank, Andrei Bursuc, Tristan Cazenave, Eric Jacopin, Christophe Guettier, J. Benton
  • [Paper]

• Qubit Routing Using Graph Neural Network Aided Monte Carlo Tree Search (AAAI 2022)

  • Animesh Sinha, Utkarsh Azad, Harjinder Singh
  • [Paper]

• Split Moves for Monte-Carlo Tree Search (AAAI 2022)

  • Jakub Kowalski, Maksymilian Mika, Wojciech Pawlik, Jakub Sutowicz, Marek Szykula, Mark H. M. Winands
  • [Paper]

• Procrastinated Tree Search: Black-Box Optimization with Delayed%2C Noisy and Multi-Fidelity Feedback (AAAI 2022)

  • Junxiong Wang, Debabrota Basu, Immanuel Trummer
  • [Paper]

• Enabling Arbitrary Translation Objectives with Adaptive Tree Search (ICLR 2022)

  • Wang Ling, Wojciech Stokowiec, Domenic Donato, Chris Dyer, Lei Yu, Laurent Sartran, Austin Matthews
  • [Paper]

• What's Wrong with Deep Learning in Tree Search for Combinatorial Optimization (ICLR 2022)

  • Maximili1an Böther, Otto Kißig, Martin Taraz, Sarel Cohen, Karen Seidel, Tobias Friedrich
  • [Paper]

• Anytime Capacity Expansion in Medical Residency Match by Monte Carlo Tree Search (IJCAI 2022)

  • Kenshi Abe, Junpei Komiyama, Atsushi Iwasaki
  • [Paper]

• Fast and Accurate User Cold-Start Learning Using Monte Carlo Tree Search (RECSYS 2022)

  • Dilina Chandika Rajapakse, Douglas Leith
  • [Paper]

2021

• Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search (AAAI 2021)

  • Li-Cheng Lan, Ti-Rong Wu, I-Chen Wu, Cho-Jui Hsieh
  • [Paper]

• Dec-SGTS: Decentralized Sub-Goal Tree Search for Multi-Agent Coordination (AAAI 2021)

  • Minglong Li, Zhongxuan Cai, Wenjing Yang, Lixia Wu, Yinghui Xu, Ji Wang
  • [Paper]

• Improved POMDP Tree Search Planning with Prioritized Action Branching (AAAI 2021)

  • John Mern, Anil Yildiz, Lawrence Bush, Tapan Mukerji, Mykel J. Kochenderfer
  • [Paper]

• Dynamic Automaton-Guided Reward Shaping for Monte Carlo Tree Search (AAAI 2021)

  • Alvaro Velasquez, Brett Bissey, Lior Barak, Andre Beckus, Ismail Alkhouri, Daniel Melcer, George K. Atia
  • [Paper]

• Single Player Monte-Carlo Tree Search Based on the Plackett-Luce Model (AAAI 2021)

  • Felix Mohr, Viktor Bengs, Eyke Hüllermeier
  • [Paper]

• Learning to Pack: A Data-Driven Tree Search Algorithm for Large-Scale 3D Bin Packing Problem (CIKM 2021)

  • Qianwen Zhu, Xihan Li, Zihan Zhang, Zhixing Luo, Xialiang Tong, Mingxuan Yuan, Jia Zeng
  • [Paper]

• Practical Massively Parallel Monte-Carlo Tree Search Applied to Molecular Design (ICLR 2021)

  • Xiufeng Yang, Tanuj Kr Aasawat, Kazuki Yoshizoe
  • [Paper]

• Convex Regularization in Monte-Carlo Tree Search (ICML 2021)

  • Tuan Dam, Carlo D'Eramo, Jan Peters, Joni Pajarinen
  • [Paper]

• Combining Tree Search and Action Prediction for State-of-the-Art Performance in DouDiZhu (IJCAI 2021)

  • Yunsheng Zhang, Dong Yan, Bei Shi, Haobo Fu, Qiang Fu, Hang Su, Jun Zhu, Ning Chen
  • [Paper]

2020

• Monte Carlo Tree Search in Continuous Spaces Using Voronoi Optimistic Optimization with Regret Bounds (AAAI 2020)

  • Beomjoon Kim, Kyungjae Lee, Sungbin Lim, Leslie Pack Kaelbling, Tomás Lozano-Pérez
  • [Paper]

• Neural Architecture Search Using Deep Neural Networks and Monte Carlo Tree Search (AAAI 2020)

  • Linnan Wang, Yiyang Zhao, Yuu Jinnai, Yuandong Tian, Rodrigo Fonseca
  • [Paper]
  • [Code]

• Monte-Carlo Tree Search in Continuous Action Spaces with Value Gradients (AAAI 2020)

  • Jongmin Lee, Wonseok Jeon, Geon-Hyeong Kim, Kee-Eung Kim
  • [Paper]
  • [Code]

• Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions (AISTATS 2020)

  • Lars Buesing, Nicolas Heess, Theophane Weber
  • [Paper]

• Watch the Unobserved: A Simple Approach to Parallelizing Monte Carlo Tree Search (ICLR 2020)

  • Anji Liu, Jianshu Chen, Mingze Yu, Yu Zhai, Xuewen Zhou, Ji Liu
  • [Paper]
  • [Code]

• Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains (ICML 2020)

  • Johannes Fischer, Ömer Sahin Tas
  • [Paper]
  • [Code]

• Sub-Goal Trees a Framework for Goal-Based Reinforcement Learning (ICML 2020)

  • Tom Jurgenson, Or Avner, Edward Groshev, Aviv Tamar
  • [Paper]

• Monte-Carlo Tree Search for Scalable Coalition Formation (IJCAI 2020)

  • Feng Wu, Sarvapali D. Ramchurn
  • [Paper]

• Generalized Mean Estimation in Monte-Carlo Tree Search (IJCAI 2020)

  • Tuan Dam, Pascal Klink, Carlo D'Eramo, Jan Peters, Joni Pajarinen
  • [Paper]

• Sparse Tree Search Optimality Guarantees in POMDPs with Continuous Observation Spaces (IJCAI 2020)

  • Michael H. Lim, Claire Tomlin, Zachary N. Sunberg
  • [Paper]

• Mix and Match: An Optimistic Tree-Search Approach for Learning Models from Mixture Distributions (NeurIPS 2020)

  • Matthew Faw, Rajat Sen, Karthikeyan Shanmugam, Constantine Caramanis, Sanjay Shakkottai
  • [Paper]

• Extracting Knowledge from Web Text with Monte Carlo Tree Search (WWW 2020)

  • Guiliang Liu, Xu Li, Jiakang Wang, Mingming Sun, Ping Li
  • [Paper]

2019

• ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search (AAAI 2019)

  • Shangtong Zhang, Hengshuai Yao
  • [Paper]
  • [Code]

• A Monte Carlo Tree Search Player for Birds of a Feather Solitaire (AAAI 2019)

  • Christian Roberson, Katarina Sperduto
  • [Paper]
  • [Code]

• Vine Copula Structure Learning via Monte Carlo Tree Search (AISTATS 2019)

  • Bo Chang, Shenyi Pan, Harry Joe
  • [Paper]
  • [Code]

• Noisy Blackbox Optimization using Multi-fidelity Queries: A Tree Search Approach (AISTATS 2019)

  • Rajat Sen, Kirthevasan Kandasamy, Sanjay Shakkottai
  • [Paper]
  • [Code]

• Reinforcement Learning Based Monte Carlo Tree Search for Temporal Path Discovery (ICDM 2019)

  • Pengfei Ding, Guanfeng Liu, Pengpeng Zhao, An Liu, Zhixu Li, Kai Zheng
  • [Paper]

• Monte Carlo Tree Search for Policy Optimization (IJCAI 2019)

  • Xiaobai Ma, Katherine Rose Driggs-Campbell, Zongzhang Zhang, Mykel J. Kochenderfer
  • [Paper]

• Subgoal-Based Temporal Abstraction in Monte-Carlo Tree Search (IJCAI 2019)

  • Thomas Gabor, Jan Peter, Thomy Phan, Christian Meyer, Claudia Linnhoff-Popien
  • [Paper]
  • [Code]

• Automated Machine Learning with Monte-Carlo Tree Search (IJCAI 2019)

  • Herilalaina Rakotoarison, Marc Schoenauer, Michèle Sebag
  • [Paper]
  • [Code]

• Multiple Policy Value Monte Carlo Tree Search (IJCAI 2019)

  • Li-Cheng Lan, Wei Li, Ting-Han Wei, I-Chen Wu
  • [Paper]

• Learning Compositional Neural Programs with Recursive Tree Search and Planning (NeurIPS 2019)

  • Thomas Pierrot, Guillaume Ligner, Scott E. Reed, Olivier Sigaud, Nicolas Perrin, Alexandre Laterre, David Kas, Karim Beguir, Nando de Freitas
  • [Paper]

2018

• Monte Carlo Methods for the Game Kingdomino (CIG 2018)

  • Magnus Gedda, Mikael Z. Lagerkvist, Martin Butler
  • [Paper]
  • [Code]
  • [Game Server]

• Reset-free Trial-and-Error Learning for Robot Damage Recovery (RAS 2018)

  • Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Jean-Baptiste Mouret
  • [Paper]
  • [Code]
  • [MCTS C++ Library]

• Memory-Augmented Monte Carlo Tree Search (AAAI 2018)

  • Chenjun Xiao, Jincheng Mei, Martin Müller
  • [Paper]

• Feedback-Based Tree Search for Reinforcement Learning (ICML 2018)

  • Daniel R. Jiang, Emmanuel Ekwedike, Han Liu
  • [Paper]

• Extended Increasing Cost Tree Search for Non-Unit Cost Domains (IJCAI 2018)

  • Thayne T. Walker, Nathan R. Sturtevant, Ariel Felner
  • [Paper]

• Three-Head Neural Network Architecture for Monte Carlo Tree Search (IJCAI 2018)

  • Chao Gao, Martin Müller, Ryan Hayward
  • [Paper]

• Bidding in Periodic Double Auctions Using Heuristics and Dynamic Monte Carlo Tree Search (IJCAI 2018)

  • Moinul Morshed Porag Chowdhury, Christopher Kiekintveld, Son Tran, William Yeoh
  • [Paper]

• Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search (NIPS 2018)

  • Zhuwen Li, Qifeng Chen, Vladlen Koltun
  • [Paper]

• M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search (NIPS 2018)

  • Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, Jianfeng Gao
  • [Paper]

• Single-Agent Policy Tree Search With Guarantees (NIPS 2018)

  • Laurent Orseau, Levi Lelis, Tor Lattimore, Theophane Weber
  • [Paper]

• Monte-Carlo Tree Search for Constrained POMDPs (NIPS 2018)

  • Jongmin Lee, Geon-hyeong Kim, Pascal Poupart, Kee-Eung Kim
  • [Paper]

2017

• An Analysis of Monte Carlo Tree Search (AAAI 2017)

  • Steven James, George Dimitri Konidaris, Benjamin Rosman
  • [Paper]

• Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation (AAAI 2017)

  • Jinzhuo Wang, Wenmin Wang, Ronggang Wang, Wen Gao
  • [Paper]

• Designing Better Playlists with Monte Carlo Tree Search (AAAI 2017)

  • Elad Liebman, Piyush Khandelwal, Maytal Saar-Tsechansky, Peter Stone
  • [Paper]

• Learning in POMDPs with Monte Carlo Tree Search (ICML 2017)

  • Sammie Katt, Frans A. Oliehoek, Christopher Amato
  • [Paper]

• Learning to Run Heuristics in Tree Search (IJCAI 2017)

  • Elias B. Khalil, Bistra Dilkina, George L. Nemhauser, Shabbir Ahmed, Yufen Shao
  • [Paper]

• Estimating the Size of Search Trees by Sampling with Domain Knowledge (IJCAI 2017)

  • Gleb Belov, Samuel Esler, Dylan Fernando, Pierre Le Bodic, George L. Nemhauser
  • [Paper]

• A Monte Carlo Tree Search Approach to Active Malware Analysis (IJCAI 2017)

  • Riccardo Sartea, Alessandro Farinelli
  • [Paper]

• Monte-Carlo Tree Search by Best Arm Identification (NIPS 2017)

  • Emilie Kaufmann, Wouter M. Koolen
  • [Paper]

• Thinking Fast and Slow with Deep Learning and Tree Search (NIPS 2017)

  • Thomas Anthony, Zheng Tian, David Barber
  • [Paper]

• Monte-Carlo Tree Search using Batch Value of Perfect Information (UAI 2017)

  • Shahaf S. Shperberg, Solomon Eyal Shimony, Ariel Felner
  • [Paper]

2016

• Using Domain Knowledge to Improve Monte-Carlo Tree Search Performance in Parameterized Poker Squares (AAAI 2016)

  • Robert Arrington, Clay Langley, Steven Bogaerts
  • [Paper]

• Monte Carlo Tree Search for Multi-Robot Task Allocation (AAAI 2016)

  • Bilal Kartal, Ernesto Nunes, Julio Godoy, Maria L. Gini
  • [Paper]

• Large Scale Hard Sample Mining with Monte Carlo Tree Search (CVPR 2016)

  • Olivier Canévet, François Fleuret
  • [Paper]

• On the Analysis of Complex Backup Strategies in Monte Carlo Tree Search (ICML 2016)

  • Piyush Khandelwal, Elad Liebman, Scott Niekum, Peter Stone
  • [Paper]

• Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games (IJCAI 2016)

  • Xiaoxiao Guo, Satinder P. Singh, Richard L. Lewis, Honglak Lee
  • [Paper]

• Monte Carlo Tree Search in Continuous Action Spaces with Execution Uncertainty (IJCAI 2016)

  • Timothy Yee, Viliam Lisý, Michael H. Bowling
  • [Paper]

• Learning Predictive State Representations via Monte-Carlo Tree Search (IJCAI 2016)

  • Yunlong Liu, Hexing Zhu, Yifeng Zeng, Zongxiong Dai
  • [Paper]

2015

• Efficient Globally Optimal Consensus Maximisation with Tree Search (CVPR 2015)

  • Tat-Jun Chin, Pulak Purkait, Anders P. Eriksson, David Suter
  • [Paper]

• Interplanetary Trajectory Planning with Monte Carlo Tree Search (IJCAI 2015)

  • Daniel Hennes, Dario Izzo
  • [Paper]

2014

• State Aggregation in Monte Carlo Tree Search (AAAI 2014)

  • Jesse Hostetler, Alan Fern, Tom Dietterich
  • [Paper]

• Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning (NIPS 2014)

  • Xiaoxiao Guo, Satinder P. Singh, Honglak Lee, Richard L. Lewis, Xiaoshi Wang
  • [Paper]

• Learning Partial Policies to Speedup MDP Tree Search (UAI 2014)

  • Jervis Pinto, Alan Fern
  • [Paper]

2013

• Monte Carlo Tree Search for Scheduling Activity Recognition (ICCV 2013)

  • Mohamed R. Amer, Sinisa Todorovic, Alan Fern, Song-Chun Zhu
  • [Paper]

• Convergence of Monte Carlo Tree Search in Simultaneous Move Games (NIPS 2013)

  • Viliam Lisý, Vojtech Kovarík, Marc Lanctot, Branislav Bosanský
  • [Paper]

• Bayesian Mixture Modelling and Inference based Thompson Sampling in Monte-Carlo Tree Search (NIPS 2013)

  • Aijun Bai, Feng Wu, Xiaoping Chen
  • [Paper]

2012

• Generalized Monte-Carlo Tree Search Extensions for General Game Playing (AAAI 2012)
  • Hilmar Finnsson
  • [Paper]

2011

• A Local Monte Carlo Tree Search Approach in Deterministic Planning (AAAI 2011)

  • Fan Xie, Hootan Nakhost, Martin Müller
  • [Paper]

• Real-Time Solving of Quantified CSPs Based on Monte-Carlo Game Tree Search (IJCAI 2011)

  • Satomi Baba, Yongjoon Joe, Atsushi Iwasaki, Makoto Yokoo
  • [Paper]

• Nested Rollout Policy Adaptation for Monte Carlo Tree Search (IJCAI 2011)

  • Christopher D. Rosin
  • [Paper]

• Variance Reduction in Monte-Carlo Tree Search (NIPS 2011)

  • Joel Veness, Marc Lanctot, Michael H. Bowling
  • [Paper]

• Learning Is Planning: Near Bayes-Optimal Reinforcement Learning via Monte-Carlo Tree Search (UAI 2011)

  • John Asmuth, Michael L. Littman
  • [Paper]

2010

• Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search (AAAI 2010)

  • Jeffrey Richard Long, Nathan R. Sturtevant, Michael Buro, Timothy Furtak
  • [Paper]

• Bayesian Inference in Monte-Carlo Tree Search (UAI 2010)

  • Gerald Tesauro, V. T. Rajan, Richard Segal
  • [Paper]

2009

• Monte Carlo Tree Search Techniques in the Game of Kriegspiel (IJCAI 2009)

  • Paolo Ciancarini, Gian Piero Favini
  • [Paper]

• Bootstrapping from Game Tree Search (NIPS 2009)

  • Joel Veness, David Silver, William T. B. Uther, Alan Blair
  • [Paper]

2008

• Direct Mining of Discriminative and Essential Frequent Patterns via Model-Based Search Tree (KDD 2008)
  • Wei Fan, Kun Zhang, Hong Cheng, Jing Gao, Xifeng Yan, Jiawei Han, Philip S. Yu, Olivier Verscheure
  • [Paper]

2007

• Bandit Algorithms for Tree Search (UAI 2007)
  • Pierre-Arnaud Coquelin, Rémi Munos
  • [Paper]

2006

• Properties of Forward Pruning in Game-Tree Search (AAAI 2006)

  • Yew Jin Lim, Wee Sun Lee
  • [Paper]

• Graph Branch Algorithm: An Optimum Tree Search Method for Scored Dependency Graph with Arc Co-Occurrence Constraints (ACL 2006)

  • Hideki Hirakawa
  • [Paper]

2005

• Game-Tree Search with Combinatorially Large Belief States (IJCAI 2005)
  • Austin Parker, Dana S. Nau, V. S. Subrahmanian
  • [Paper]

2003

• Solving Finite Domain Constraint Hierarchies by Local Consistency and Tree Search (IJCAI 2003)
  • Stefano Bistarelli, Philippe Codognet, Kin Chuen Hui, Jimmy Ho-Man Lee
  • [Paper]

2001

• Incomplete Tree Search using Adaptive Probing (IJCAI 2001)
  • Wheeler Ruml
  • [Paper]

1998

• KnightCap: A Chess Programm That Learns by Combining TD with Game-Tree Search (ICML 1998)
  • Jonathan Baxter, Andrew Tridgell, Lex Weaver
  • [Paper]

1988

• A Tree Search Algorithm for Target Detection in Image Sequences (CVPR 1988)
  • Steven D. Blostein, Thomas S. Huang
  • [Paper]

---

License

• CC0 Universal

---