v0.5.1

v0.5.1 🚀

Minor release with several QOL improvements

What's Changed

• Implement floyd on tmat_class atsp generation by @abcdhhhh in #226
• Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows by @dependabot in #210
• [DOCS] Modify url in example/1-quickstart.ipynb by @falconlee236 in #215
• TSPLIB and CVRPLIB testing notebooks
• Automatically enable sampling if num_samples>1
• Update Spec names according to torchrl>=0.6.0
• Now actions are automatically returned by default, no need to specify return_actions=True
• Fix edge cases for SDPA #228
• PCTSP distribution problem fix

New Contributors

• @falconlee236 made their first contribution in #215
• @abcdhhhh made their first contribution in #226

Full Changelog: v0.5.0...v0.5.1

v0.5.0

Major release: v0.5.0 is finally here! 🚀

We are proud to finally release our latest version, 0.5.0, after much work done for NeurIPS!
(Will our paper finally get accepted? 🤞)

Changelog

✨ Features

• New documentation released at rl4.co!
• Add SOTA FJSP environment @LTluttmann
• Add Improvement methods and respective environments MDP @yining043
  • N2S
  • DACT
  • NeuOpt
• Ade HetGNN model for the JFSP @LTluttmann
• Add L2D model @LTluttmann
• Add Multi-task VRP (MTVRP) environment
• Add temperature in NARGNN policies @Furffico
• Add multiple batch sizes for different dataset
• Local search support, DeepACO + Local search @hyeok9855
• Add MTPOMO, MVMoE model @RoyalSkye @FeiLiu36
• Supporting the meta learning trainer @jieyibi
• Supporting the improvement training @yining043
• Add graph problems: MCP and FLP @bokveizen
• New PPO versions:
  • Stepwise @LTluttmann
  • Improvement @yining043
• PolyNet support @ahottung
• Different distributions support + MDPOMO @jieyibi
• Add initial support for solvers API from RL4CO (MTVRP): PyVRP, OR-Tools, LKH3 @N-Wouda @leonlan
• Faster data logprobs collection: now we don't need to collect logprobs for unused trajectories, but we gather only logprobs for selected nodes by default, which decreases memory consumption
• Add Codecov to track the tests coverage

⚙️ Refactoring

• [Environment] Supporting generator_params arguments for environments, more modularized and flexible.
• Modularization of the Attention Model decoder’s QKV calculation for more flexibility @LTluttmann
• Refactor the MatNet encoder with the cross attention only needs to be calculated once @LTluttmann

📝 Documentation

• New documentation based on MkDocs
  • Fast search
  • Beautiful (we hope you'll like it!) new homepage
  • New API reference, about section
  • Ad-free website
  • Light/Dark mode
  • New about sections (licensing, citation)
  • ...and more!
• New tutorial on data distributions
• Miscellaneous: fix Colab links @wouterkool

🐛 Bug Fixes

• Fix the DeepACO’s log_heuristic calculation bug to raise the performance. @Furffico @henry-yeh
• Solve memory leakage during the autoregressive decoding @LTluttmann
• Python versioning: remove Python 3.8, compatibility with Python 3.12, and poetry support @ShuN6211
• Compatibility with tensordict>=0.5.0
• Memory leak in OP and PCTSP
• Fix A2C bug: optimize all parameters in module instead of only "policy" by default
• Fix double logging parameters, better logging in Wandb

v0.4.0

Major release: v0.4.0 is here! 🚀

This release adds several new features and major refactorings in both modeling and environment sides!

Changelog

✨ Features

• DeepACO + ACO @Furffico @henry-yeh
• Non-autoregressive (NAR) models and NARGNN @Furffico @henry-yeh
• Add modular environment data generator with support to new distributions @cbhua
• New decoding techniques based on the decoding strategy class @LTluttmann
  • Top-p (nucleus sampling)
  • Top-k
• Select start nodes functions @LTluttmann

⚙️ Refactoring

• Major modeling refactoring (summarized here). Now we categorize NCO approaches (which are not necessarily trained with RL!) into the following: 1) constructive (AR and NAR), 2) improvement, 3) transductive. This translates into code, which is now fully customizable. For instance, in constructive methods, now encoders / decoders can be fully replaced or removed in an abstract way!
• Major environment refactoring (summarized here): we further modularize the environments into components (logic under env, data generation under generator, and so on), with several components moved inside the RL4COEnvBase. Importantly, we introduce data generators that can be customized!
• Use Abstract classes if class should not be @ngastzepeda

📝 Documentation

• Hydra documentation and tutorial @LTluttmann
• New modularized examples under examples/
• Updated RL4CO structure in ReadTheDocs
• Move to MIT license with AI4CO for inclusiveness
• New RL4CO / AI4CO swag. You may also find them here!

🐛 Bug Fixes

• MatNet and FFSP bugfix @LTluttmann
• Best solution gathering from POMO @ahottung
• Tests now passing on MPS; compatibility with TorchRL pytorch/rl#2125
• Miscellaneuous @LTluttmann , @bokveizen , @tycbony

v0.3.3

New Routing Envs and more 🚀

Changelog

✨ Features

• Add CVRPTW Environment @ngastzepeda
  • Add Solomon instance / solution loader via vrplib
• Add basic Skill-VRP (SVRP) @ngastzepeda

📃 Documentation

• [Minor] improve decoding strategies documentation

🐛 Bug Fixes

• Avoid deepcopy bug by not saving intermediate steps of decoding strategy #123
• Allow passing select_start_nodes_fn and other kwargs in decoding strategies

v0.3.2

New Decoding Types and more 🚀

Changelog

Features

• Beam Search #109 #110 @LTluttmann
• Decoding type class #109 #110 @LTluttmann

Documentation

• Add (simple , API work in progress!) tutorial notebooks for TSPLib and CVRPLib #84
• Add decoding strategies notebook @LTluttmann + small fix @Haimrich

Optimization

• torch.no_grad to torch.inference_mode
• Faster testing

Bug Fixes

• Batch size initialization @ngastzepeda
• Bump up naming to align with 0.4.0 release of TorchRL
• MatNet bug fix #108

v0.3.1

QoL and BugFixes 🚀

Changelog

• Better multi start decoding #102
  • Add modular select_start_nodes function for POMO
  • Improve efficiency of multistart function
  • Add testing and selection function for more envs
  • Fix OP selecting too far away nodes in POMO
  • Automatic multistart, no need to manually choose beforehand when running POMO
• Fix CVRP capacity bug @ngastzepeda #105
• Add critic init embedding support
• Fix data generation and add better docs #106
• Better dataset handling: add dataset choice; use low CPU usage dataset by default
• Better solution plotting and better quickstart notebook #103
• Library winter cleanup
• Miscellaneous minor fixes here and there

v0.3.0

Faster Library, Python 3.11 and new TorchRL support, Envs, Models, Multiple Dataloaders, and more 🚀

Faster Library, new Python 3.11 and TorchRL

• Update to latest TorchRL #72, solving several issues as #95 #97 (also see this)
• Benchmarking:
  • Up to 20% speedup in training epochs thanks to faster TensorDict and new env updates
  • Almost instant data generation (avoid list comprehension, e.g. from ~20 seconds to <1 second per epoch!)
• Python 3.11 now available #97

New SMTWTP environment

• Add new scheduling problem: Single Machine Total Weighted Tardiness Problem environment as in DeepACO @henry-yeh

New MatNet model

• Add MatNet version for square matrices (faster implementation ideal for routing problems)
• Should be easy to implement scheduling from here

Multiple Dataloaders

• Now it is possible to have multiple dataloaders, with naming as well!
  • For example, to track generalization during training

Miscellaneous

• Fix POMO shapes @hyeok9855 , modularizing PPO etc
• Fix precion bug for PPO
• New AI4CO transfer!

v0.2.3

Add FlashAttention2 support ⚡

• Add FlashAttention2 support as mentioned here
• Remove old wrapper for half() precision since Lightning already deals with this
• Fix scaled_dot_product_attention implementation in PyTorch < 2.0
• Minor fixes

v0.2.2

QoL: New Baseline, Testing Search Methods, Downloader, Miscellanea 🚀

Changelog

• Add mean baseline @hyeok9855
• Add testing for search methods
• Move downloader to external repo, extra URL as backup for DPP
• Small bug fix for duplicate args
• Add more modular data generation
• Suppress extra warning in automatic_optimization
• Minor doc cleaning

v0.2.1

QoL, Better documentation, Bug Fixes 🚀

• Add RandomPolicy class
• Control max_steps for debugging purposes during decoding
• Better documentation, add tutorials, and references #88 @bokveizen
• Set bound to < Python 3.11 for the time being #90 @hyeok9855
• Log more info by default in PPO
• precompute_cache method can now accept td as well
• If Trainer is supplied with gradient_clip_val and manual_optimization=False, then remove gradient clipping (e.g. for PPO)
• Fix test data size following training and not test by default