-
-
Notifications
You must be signed in to change notification settings - Fork 2k
Timeline
What a year! What started out as a very diffuse outlook has converged to an exciting future for PyMC3: We decided to stick with Theano as our backend, while turning it into a next-generation graph computation engine.
This means that the library you know as PyMC3 will prevail, albeit with some breaking changes coming up.
Let's start with some basics:
- With PyMC3 v3.10.0 we switched the backend from
Theano v1.0.5toTheano-PyMC v1.0.11.Theano-PyMCis our fork that we are currently refactoring. - We are continuing to refactor
Theano-PyMCfor an easier development, while making some incremental releases. - In parallel, we're keeping the PyMC3 master branch up to date and are currently working towards a
3.11.0release.3.11.0will include some breaking changes such as treating(1,)shapes as vectors and dropping Python 3.6 support.
You could say that 3.10 ---> 3.11 -?-> 3.12 is a "rough ride", but the biggest change is still coming up:
A new RandomVariable Op was recently merged into Theano-PyMC. By switching the inner workings of PyMC3 from its pm.Distribution class to the new RandomVariable, we will open the door to many exciting possibilities. It will also dramatically simplify the inner workings of PyMC3, solving most if not all shape problems and allowing us to delete huge chunks of internal code.
Theano-PyMC will be renamed to Aesara and get a 2.0 release.
The rename of Theano to Aesara and the switch of the PyMC3 internals to RandomVariable mandate a PyMC3 4.0 release.
Yeah, about that...
Even though the internals of PyMC3 4.0 will be quite different to PyMC3 3.x, we can keep the user-facing API largely identical. Custom distributions will have to be refactored, but most models will work just fine! The same is true for commonly used functions for sampling and prior/posterior predictive.
We only have the capacity to maintain and develop one PyMC library.
Superseding 3.x with 4.0 focuses users and developers to that one library.
As mentioned earlier, we're continuing on a slightly faster release cycle from 3.10 onwards. Here we'll focus on the following:
- Outsourcing docs to
pymc-examples - Taking out deprecated functionality and streamlining the API to facilitate switching to
4.0 - Not touching obsolete parts that will be replaced/removed for
RandomVariable-based models.
The switch to RandomVariable and Aesara 2.0 is prepared on a v4 branch on the main repo. This way the PR and Issue numbering remains compatible, and we can do a "The big one" PR into master to bump us to 4.0.
- Some things are still
ToDoin the backend. See Milestones. - We'll have to temporarily phase out some PyMC3 submodules such as
glm,gp,odeand specialized step methods such asSMCorMLDA. This is just so we can push forward with the core functionality and gradually phase them back in (e.g. with4.1).
The JAX linker is already available in current Theano-PyMC releases.
It is not a blocker w.r.t. RandomVariable and PyMC 4.0, which means that improvements to it are independent of this timeline!
Switching the inner workings of PyMC3 while keeping large parts of its user-facing API unaffected is a bit like open heart surgery. It requires good planning, concentration and swift execution, but brings great benefits compared to the alternatives.
We manage this through milestones on PyMC3 and Theano-PyMC.
- Everything in the
vNextmilestone has topmost priority. - Issues in milestones later than
vNextmay depend on things from a previous milestone. - Issues outside of milestones are considered "backlog". They don't block a release, but fixing them is still an important contribution.
We're transplanting the inner organs of PyMC3. It's tricky, but with 4.0 your favorite library will supercharge modern probabilistic programming.
As of May 1st, 2020 this page intends to give a diffuse estimate about where PyMC3 is heading in the near and mid-term future.
The PyMC3 project is alive and well, with new features coming in and people picking it up for new projects.
At the same time we're monitoring how the world of probabilistic programming is evolving and we're preparing to stabilize PyMC3 in the long term.
In November 2018 we began this by factoring out plotting, stats and diagnostics to ArviZ, and we're now trying to make PyMC3 more maintainable and easier to productionize.
This involves improving our documentation, release management and working on PyMC3's internals to make it easier to fix bug or incompatibilities.
At some point in the future (with lots of uncertainty), there will inevitably be an incompatibility that we can't fix, which is why some development effort is also directed towards new exciting projects such as PyMC4.