Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media transformation sampling & plotting methods #734

Merged
merged 12 commits into from
Jun 12, 2024

Conversation

wd60622
Copy link
Contributor

@wd60622 wd60622 commented Jun 11, 2024

Description

This adds new methods for sampling and plotting the media transformations. They are:

  • sample_prior: sample the parameter distributions for the transformation
  • sample_curve: sample the curve given a set of parameters. These can be prior or posterior
  • plot_curve: Plot curve HDI and n random curve samples. Uses plot_curve_hdi and plot_curve_samples methods

This creates a nice pre-modeling workflow of:

import matplotlib.pyplot as plt

saturation = ...
prior = saturation.sample_prior()
curve = saturation.sample_curve(parameters=prior)
saturation.plot_curve(curve)
plt.show()

And also model analysis

mmm = ...
mmm.fit(X, y)

# Curve can now has additional dimensions like channel
curves = mmm.saturation.sample_curve(mmm.fit_result)

Related Issue

Checklist

Modules affected

  • MMM
  • CLV

Type of change

  • New feature / enhancement
  • Bug fix
  • Documentation
  • Maintenance
  • Other (please specify):

📚 Documentation preview 📚: https://pymc-marketing--734.org.readthedocs.build/en/734/

@wd60622 wd60622 added enhancement New feature or request MMM labels Jun 11, 2024
Copy link

codecov bot commented Jun 11, 2024

Codecov Report

Attention: Patch coverage is 91.05691% with 11 lines in your changes missing coverage. Please review.

Project coverage is 93.48%. Comparing base (271966b) to head (1361413).
Report is 166 commits behind head on main.

Files with missing lines Patch % Lines
pymc_marketing/mmm/components/base.py 89.81% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #734      +/-   ##
==========================================
- Coverage   93.60%   93.48%   -0.12%     
==========================================
  Files          27       27              
  Lines        2722     2841     +119     
==========================================
+ Hits         2548     2656     +108     
- Misses        174      185      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@juanitorduz juanitorduz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments :)

pymc_marketing/mmm/components/adstock.py Outdated Show resolved Hide resolved
ax: plt.Axes | None = None,
hdi_kwargs: dict | None = None,
plot_kwargs: dict | None = None,
) -> plt.Axes:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually return the figure... shall we keep that convention (no strong opinion)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am leaning toward that as well.
I am adding the support to sample the curves based on parameter distributions that could include other dimensions (i.e. channel) and think support for plotting
i.e. 3 channels in the dims would lead to figure with 3 axes each with their saturation curve.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok! No strong preference. Just think about it from a user perspective :)

@wd60622 wd60622 marked this pull request as draft June 11, 2024 14:19
@wd60622
Copy link
Contributor Author

wd60622 commented Jun 11, 2024

Still need to add:

  • plot as grid based on the coordinates of the curves

@wd60622 wd60622 marked this pull request as ready for review June 12, 2024 10:19
@wd60622
Copy link
Contributor Author

wd60622 commented Jun 12, 2024

This should be good to go @juanitorduz
Let me know your thoughts

The plots adapt to the "Curve" that is used as argument. For instance, unless specified, the priors will have only chain, draw coords as dims wouldn't be known / are all the same

transformation = LogisticSaturation()

parameters= transformation.sample_prior()
curve = transformation.sample_curve(parameters)
transformation.plot_curve(curve)
plt.show()

single-saturation

coords = {"channel": ["A", "B", "C"]}
parameters= transformation.sample_prior(coords=coords) # This could be from posterior with additional coords for transformation
curve = transformation.sample_curve(parameters)
transformation.plot_curve(curve)
plt.show()

multiple-saturations

@juanitorduz
Copy link
Collaborator

juanitorduz commented Jun 12, 2024

Two tests are failing:

FAILED tests/mmm/test_delayed_saturated_mmm.py::test_plotting_media_transform_workflow[adstock]
FAILED tests/mmm/test_delayed_saturated_mmm.py::test_plotting_media_transform_workflow[saturation]

@wd60622
Copy link
Contributor Author

wd60622 commented Jun 12, 2024

Two tests are failing:

FAILED tests/mmm/test_delayed_saturated_mmm.py::test_plotting_media_transform_workflow[adstock]
FAILED tests/mmm/test_delayed_saturated_mmm.py::test_plotting_media_transform_workflow[saturation]

Somewhere the mmm is being modified which makes the tests sensitive to the order of tests. This should be fixed but probably not best thing to have order matter

Reference: Before and After

@juanitorduz
Copy link
Collaborator

@wd60622 this looks great! I just left a question :)

@juanitorduz
Copy link
Collaborator

We have these bad guys failing

FAILED tests/mmm/components/test_base.py::test_new_transformation_plot_curve[coords0-1]
FAILED tests/mmm/components/test_base.py::test_new_transformation_plot_curve[coords1-3]

:D

Copy link
Collaborator

@juanitorduz juanitorduz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update the branch and merge! Cool work!

@juanitorduz juanitorduz merged commit 6fe03dc into main Jun 12, 2024
9 of 10 checks passed
@juanitorduz juanitorduz deleted the media-sampling-methods branch June 12, 2024 15:06
@juanitorduz juanitorduz added this to the 0.7.0 milestone Jun 13, 2024
twiecki pushed a commit that referenced this pull request Sep 10, 2024
* add plotting methods

* add tests for new methods

* saturation support for additional variable dims

* consolidate the logic of sampling

* change warning

* workflow from a fitted model

* change order of tests

* suggestion to use names

* because of new data

---------

Co-authored-by: Juan Orduz <[email protected]>
twiecki pushed a commit that referenced this pull request Sep 10, 2024
* add plotting methods

* add tests for new methods

* saturation support for additional variable dims

* consolidate the logic of sampling

* change warning

* workflow from a fitted model

* change order of tests

* suggestion to use names

* because of new data

---------

Co-authored-by: Juan Orduz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request MMM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Media transforms prior sampling & curve plotting
2 participants