Time varying intercept #628

ulfaslak · 2024-04-15T13:59:54Z

This PR adds a time varying coefficient for the intercept. It's near-identical to #598, except it does not add support for time varying media contributions (this, we will add later).

Description

The API is now:

mmm = DelayedSaturatedMMM(
    date_column=my_date_column_name,
    channel_columns=my_media_column_names,
    time_varying_intercept=True,            # <-- This is new
)

Moreover, the model_config can now contain arguments for the time varying prior:

mmm = DelayedSaturatedMMM(
    ...
    model_config={
        ...,
        "tvp_kwargs": {
            "m": 200,
            "L": 100,  # If left to None, is estimated from the data (supporting 1 year of future OOS prediction)  
            "eta_lam": 1,
            "ls_mu": 52,  # If left to None, is estimated from the data (supporting 1 year of future OOS prediction)    
            "ls_sigma": 10,
            "cov_func": myCovFunc,  # If left to None, default Matern52 is used (good default).
        },
    }
)

Changes are also made to the plotting and data setter functions, to support out of sample predictions.

Checklist

Checked that the pre-commit linting/style checks pass
Included tests that prove the fix is effective or that the new feature works
Added necessary documentation (docstrings and/or example notebooks)
If you are a pro: each commit corresponds to a relevant logical change

Modules affected

MMM
CLV

Type of change

📚 Documentation preview 📚: https://pymc-marketing--628.org.readthedocs.build/en/628/

…varying-prior

review-notebook-app · 2024-04-15T14:00:00Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

codecov · 2024-04-15T14:13:08Z

Codecov Report

Attention: Patch coverage is 94.52055% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 91.78%. Comparing base (e63929a) to head (f0e59e6).

Files	Patch %	Lines
pymc_marketing/mmm/base.py	83.33%	3 Missing ⚠️
pymc_marketing/mmm/delayed_saturated_mmm.py	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #628      +/-   ##
==========================================
+ Coverage   91.73%   91.78%   +0.05%     
==========================================
  Files          22       24       +2     
  Lines        2322     2386      +64     
==========================================
+ Hits         2130     2190      +60     
- Misses        192      196       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

wd60622

I think the API change is fine. Just some comments to package up the code a bit more with extendability and reusability in mind in the future. Hopefully then being available for other cases and useful even with upcoming API changes

Would love to hear the thoughts

pymc_marketing/mmm/utils.py

pymc_marketing/mmm/tvp.py

pymc_marketing/mmm/delayed_saturated_mmm.py

pymc_marketing/mmm/base.py

pymc_marketing/mmm/delayed_saturated_mmm.py

juanitorduz

@ulfaslak Super nice PR and fun to review! I left some small comments :)

pymc_marketing/mmm/base.py

pymc_marketing/mmm/delayed_saturated_mmm.py

pymc_marketing/mmm/tvp.py

pymc_marketing/mmm/utils.py

juanitorduz · 2024-04-16T09:55:48Z

@ulfaslak can you please the notebook name in https://raw.githubusercontent.com/pymc-labs/pymc-marketing/main/docs/source/notebooks/index.md so that it gets passed tot he docs 🙏

juanitorduz · 2024-04-16T10:34:03Z

@ulfaslak you might also wanna update the branch as we solved an issue in #633 🙏

review-notebook-app · 2024-04-16T10:42:09Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:09Z
----------------------------------------------------------------

can you add black lines so that the items are spaced correctly :)

review-notebook-app · 2024-04-16T10:42:10Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:10Z
----------------------------------------------------------------

Would you mind adding

warnings.filterwarnings("ignore")

az.style.use("arviz-darkgrid")
plt.rcParams["figure.figsize"] = [12, 7]
plt.rcParams["figure.dpi"] = 100

%load_ext autoreload
%autoreload 2
%config InlineBackend.figure_format = "retina"

To make the style consistent :)

review-notebook-app · 2024-04-16T10:42:11Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:10Z
----------------------------------------------------------------

can you explain a bit what the do operator does for users who are not familiar with it?
Can you comment on the choice of the true_params?Think about someone who just knows the concepts from the MMM example
add x and y labels to the plot :)

review-notebook-app · 2024-04-16T10:42:12Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:11Z
----------------------------------------------------------------

Line #30.    plt.title("Target")

can we use the fig, ax - plt.subplots() syntax?

review-notebook-app · 2024-04-16T10:42:12Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:12Z
----------------------------------------------------------------

Line #7.        sampler_config={"chains": 2, "draws": 10, "tune": 10, "nuts_sampler": "numpyro"},

Just 10 draws?

review-notebook-app · 2024-04-16T10:42:13Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:13Z
----------------------------------------------------------------

Can you please give more explanation to these plots (titles, x and y labels ?) :D

Also, final words on the key takeaways :)

…varying-intercept

ulfaslak · 2024-04-16T12:49:25Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-16T10:42:12Z ----------------------------------------------------------------

Line #7. sampler_config={"chains": 2, "draws": 10, "tune": 10, "nuts_sampler": "numpyro"},
Just 10 draws?

ya that was for debugging. I'm revising this whole notebook and making it nice now :). Thanks for all the comments!

If provided `X` to `_data_setter` was *not* the immediate sequence following the training set, the time_index would we wrong. With this fix, the `date["time_index"]` which gets set upon providing a new `X`, is inferred from the `self.date_column` column of provided `X` (by comparing it to same date column in the training data `self.X`).

ulfaslak · 2024-04-30T21:00:22Z

There is one test failing tests/mmm/test_tvp.py::test_calling_without_default_args

I'll address this tomorrow.

SO CLOSE!!

juanitorduz · 2024-04-30T21:05:36Z

That would be an interesting observation indeed!

View entire conversation on ReviewNB

review-notebook-app · 2024-04-30T21:07:02Z

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-04-30T21:07:01Z
----------------------------------------------------------------

These zoom in plots are great! thanks for adding them !!!

ulfaslak commented on 2024-05-01T07:36:06Z
----------------------------------------------------------------

I'll start using ax.annotate ALL THE TIME from now on 😈

juanitorduz · 2024-04-30T21:09:49Z

There is one test failing tests/mmm/test_tvp.py::test_calling_without_default_args

I'll address this tomorrow.

SO CLOSE!!

The notebook looks amazing! There are a couple of minor comments open in the notebook, but they are not blockers, IMO (it would be nice to test if, in this case, the out-of-sample forecast accuracy breaks after one length scale). So, once you are happy with the notebook and you fix the test, we can merge this amazing addition! :)

juanitorduz · 2024-04-30T21:10:31Z

(tomorrow is 1st of May so maybe you do it on Thursday and enjoy the day :D )

cetagostini · 2024-04-30T21:36:11Z

I love the new introduction!

View entire conversation on ReviewNB

ulfaslak · 2024-05-01T07:35:15Z

Maaa maybe. I think in reality things are more complicated than that and we can't make an observation that simple. The default length-scale is 2 years (104.36 when data is weekly, 730.5 when it is daily)... When I lower length-scale in the last example, to one year, OOS uncertainty actually get smaller. I could try to dig into the relationship between LS and OOS performance, but the finding and nuanced answer would probably be *it depends*, and then—to bake that into this notebook—would require a new section. That might clutter the narrative and would take me a bunch of time = slow down release. So while interesting that you note this, I am against adding it for now (could be material for another time though, let's take note).

View entire conversation on ReviewNB

ulfaslak · 2024-05-01T07:36:07Z

I'll start using ax.annotate ALL THE TIME from now on 😈

View entire conversation on ReviewNB

juanitorduz · 2024-05-01T07:37:05Z

Maaa maybe. I think in reality things are more complicated than that and we can't make an observation that simple. The default length-scale is 2 years (104.36 when data is weekly, 730.5 when it is daily)... When I lower length-scale in the last example, to one year, OOS uncertainty actually get smaller. I could try to dig into the relationship between LS and OOS performance, but the finding and nuanced answer would probably be it depends, and then—to bake that into this notebook—would require a new section. That might clutter the narrative and would take me a bunch of time = slow down release. So while interesting that you note this, I am against adding it for now (could be material for another time though, let's take note).

View entire conversation on ReviewNB

You are right! Let's skip this for this first iteration :)

ulfaslak · 2024-05-01T07:54:49Z

(tomorrow is 1st of May so maybe you do it on Thursday and enjoy the day :D )

Appreciate the gesture 🙏. I'm already two cups of coffee deep though, so I might as well get it wrapped up 😅

ulfaslak · 2024-05-01T08:42:04Z

@juanitorduz @wd60622 @carlosagostini OK I think it's done now. Fixed the test and did some last minute copy fixes using grammarly (lotta typos when writing in free hand in code editor). Letting the CI spinners converge now, and hopefully we can merge today on May 1st 🎉.

juanitorduz · 2024-05-01T09:13:48Z

@juanitorduz @wd60622 @carlosagostini OK I think it's done now. Fixed the test and did some last minute copy fixes using grammarly (lotta typos when writing in free hand in code editor). Letting the CI spinners converge now, and hopefully we can merge today on May 1st 🎉.

How do you see Grammarly on notebooks? Or do you convert it to markdown?

juanitorduz · 2024-05-01T09:17:50Z

Fantastic 🚀 !

@wd60622 do you have any additional comments :) ?

wd60622

Looks good! Epic

ulfaslak · 2024-05-01T09:41:35Z

@juanitorduz @wd60622 @carlosagostini OK I think it's done now. Fixed the test and did some last minute copy fixes using grammarly (lotta typos when writing in free hand in code editor). Letting the CI spinners converge now, and hopefully we can merge today on May 1st 🎉.

How do you see Grammarly on notebooks? Or do you convert it to markdown?

I have Grammarly as a Chrome extension, and then I just load the notebook in the browser the old school way.

* Add time-varying prior functionality to DelayedSaturatedMMM * resolve wd's comments * resolve failing pre-commits * add tvp_kwargs to model_config * fix typo * replace softplus * resolve minor review comments * Add option to supply `ax` to `plot_posterior_predictive` * bugfix: time_index was not set correctly for OOS If provided `X` to `_data_setter` was *not* the immediate sequence following the training set, the time_index would we wrong. With this fix, the `date["time_index"]` which gets set upon providing a new `X`, is inferred from the `self.date_column` column of provided `X` (by comparing it to same date column in the training data `self.X`). * Clean up example notebook * Make utility function `transform_1d_array` * 'tvp_kwargs' -> 'intercept_tvp_kwargs' * move `infer_time_index` into utils * add tests for new utils * small fixes (found in tests) * add tests to cover all added cases * fix ruff check * update typehints * resolve review comments * refactor model logic for tv intercept * address review comment for util test * . * fix documentation link * change variable name * fix hsgp_dims * update time_varying_prior to be centered on 1 * review fixes * fix broken test * add final tests * fix coverage issues * Update tests/mmm/test_tvp.py Co-authored-by: Will Dean <[email protected]> * Update pymc_marketing/mmm/tvp.py Co-authored-by: Will Dean <[email protected]> * Update tests/mmm/test_tvp.py Co-authored-by: Will Dean <[email protected]> * Update tests/mmm/test_tvp.py Co-authored-by: Will Dean <[email protected]> * significant improvements to notebook * fix heading * update notebook to make it EVEN better * update legend, add watermark * fix intro * fix broken test * copy sweep with grammarly --------- Co-authored-by: Will Dean <[email protected]>

ulfaslak and others added 8 commits March 19, 2024 13:47

Add time-varying prior functionality to DelayedSaturatedMMM

faaba0f

resolve wd's comments

3ce5ac4

Merge branch 'main' into time-varying-prior

dac69da

Merge branch 'main' of github.com:pymc-labs/pymc-marketing into time-…

ec316b4

…varying-prior

resolve failing pre-commits

7e8aee7

merge previous PR into this

b0aaa2d

add tvp_kwargs to model_config

16f9414

fix typo

8ab0532

wd60622 reviewed Apr 15, 2024

View reviewed changes

juanitorduz reviewed Apr 16, 2024

View reviewed changes

replace softplus

c06c06d

juanitorduz added docs Improvements or additions to documentation enhancement New feature or request MMM labels Apr 16, 2024

ulfaslak added 2 commits April 16, 2024 13:09

resolve minor review comments

b70cd06

Merge branch 'main' of github.com:pymc-labs/pymc-marketing into time-…

9b1287b

…varying-intercept

ulfaslak added 3 commits April 17, 2024 12:47

Add option to supply ax to plot_posterior_predictive

e01f9ac

Clean up example notebook

83e7103

fix intro

d3ced36

ulfaslak added 2 commits May 1, 2024 10:18

fix broken test

f8c0b3c

copy sweep with grammarly

3f88589

Merge branch 'main' into time-varying-intercept

f0e59e6

juanitorduz approved these changes May 1, 2024

View reviewed changes

wd60622 approved these changes May 1, 2024

View reviewed changes

juanitorduz merged commit f0097e2 into main May 1, 2024
10 checks passed

juanitorduz deleted the time-varying-intercept branch May 1, 2024 09:38

This was referenced May 1, 2024

Add tv intrecept to readme #661

Merged

TV intercept notebook: fix title level #663

Merged

wd60622 mentioned this pull request May 20, 2024

How to add random walk to pymc model config #696

Closed

wd60622 mentioned this pull request Jun 3, 2024

Add HSGP prior #415

Closed

ulfaslak restored the time-varying-intercept branch September 8, 2024 11:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Time varying intercept #628

Time varying intercept #628

ulfaslak commented Apr 15, 2024 •

edited by juanitorduz

Loading

review-notebook-app bot commented Apr 15, 2024

codecov bot commented Apr 15, 2024 •

edited

Loading

wd60622 left a comment •

edited

Loading

juanitorduz left a comment

juanitorduz commented Apr 16, 2024

juanitorduz commented Apr 16, 2024

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

ulfaslak commented Apr 16, 2024

ulfaslak commented Apr 30, 2024

juanitorduz commented Apr 30, 2024

review-notebook-app bot commented Apr 30, 2024 •

edited

Loading

juanitorduz commented Apr 30, 2024

juanitorduz commented Apr 30, 2024

cetagostini commented Apr 30, 2024

ulfaslak commented May 1, 2024

ulfaslak commented May 1, 2024

juanitorduz commented May 1, 2024

ulfaslak commented May 1, 2024

ulfaslak commented May 1, 2024

juanitorduz commented May 1, 2024

juanitorduz commented May 1, 2024

wd60622 left a comment

ulfaslak commented May 1, 2024

Time varying intercept #628

Time varying intercept #628

Conversation

ulfaslak commented Apr 15, 2024 • edited by juanitorduz Loading

Description

Checklist

Modules affected

Type of change

review-notebook-app bot commented Apr 15, 2024

codecov bot commented Apr 15, 2024 • edited Loading

Codecov Report

wd60622 left a comment • edited Loading

Choose a reason for hiding this comment

juanitorduz left a comment

Choose a reason for hiding this comment

juanitorduz commented Apr 16, 2024

juanitorduz commented Apr 16, 2024

review-notebook-app bot commented Apr 16, 2024 • edited Loading

review-notebook-app bot commented Apr 16, 2024 • edited Loading

review-notebook-app bot commented Apr 16, 2024 • edited Loading

review-notebook-app bot commented Apr 16, 2024 • edited Loading

review-notebook-app bot commented Apr 16, 2024 • edited Loading

review-notebook-app bot commented Apr 16, 2024 • edited Loading

ulfaslak commented Apr 16, 2024

ulfaslak commented Apr 30, 2024

juanitorduz commented Apr 30, 2024

review-notebook-app bot commented Apr 30, 2024 • edited Loading

juanitorduz commented Apr 30, 2024

juanitorduz commented Apr 30, 2024

cetagostini commented Apr 30, 2024

ulfaslak commented May 1, 2024

ulfaslak commented May 1, 2024

juanitorduz commented May 1, 2024

ulfaslak commented May 1, 2024

ulfaslak commented May 1, 2024

juanitorduz commented May 1, 2024

juanitorduz commented May 1, 2024

wd60622 left a comment

Choose a reason for hiding this comment

ulfaslak commented May 1, 2024

ulfaslak commented Apr 15, 2024 •

edited by juanitorduz

Loading

codecov bot commented Apr 15, 2024 •

edited

Loading

wd60622 left a comment •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 16, 2024 •

edited

Loading

review-notebook-app bot commented Apr 30, 2024 •

edited

Loading