Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize Acceptance Rate Plot by ESS #345

Merged
merged 4 commits into from
Sep 11, 2020

Conversation

Havaldar
Copy link

@Havaldar Havaldar commented Sep 8, 2020

Tackles issue #202

  • Defaults to plotting acceptance rate by population size
  • Uses flag to replace population size by effective sample size

Example plot (with acceptance rate default and ESS plot)
example

@yannikschaelte yannikschaelte changed the base branch from master to develop September 9, 2020 12:21
Copy link
Member

@yannikschaelte yannikschaelte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, that looks (except one thing) really nice. Could you also add a test case to https://github.com/ICB-DCM/pyABC/blob/master/test/visualization/test_visualization.py#L53? I.e. just run the same code with the new option enabled. I know that this is not a proper unit test, but for visualization that is all we currently do.

pop_sizes.append(np.array(
history.get_nr_particles_per_population().values[1:]))

if normalize_by_ess:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is one error here: Not samples should be replaced by ESS, but pop_sizes should be replaced by ESS, t.t. the resulting rations are usually <= 1. So, just move this code block up a bit ;) Sorry, if I miscommunicated that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update the docstring accordingly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

@Havaldar
Copy link
Author

Havaldar commented Sep 9, 2020

made all the changes! (Thank you so much for the review :) )

Copy link
Member

@yannikschaelte yannikschaelte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

(And in your plot I see that we need to enforce integer values on the x axis when plotting populations. Not sure how to best do that -- needs to be integer, but should not be all integers in case many time points exist. Leaving that for future work :) .)

@@ -74,7 +74,8 @@ def test_acceptance_rates_trajectory():
histories, labels, yscale='log', rotation=76)
_, ax = plt.subplots()
pyabc.visualization.plot_acceptance_rates_trajectory(
histories, labels, yscale='log10', rotation=76, size=(10, 5), ax=ax)
histories, labels, yscale='log10', rotation=76, size=(10, 5), ax=ax
normalize_by_ess=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing: Could you once run this with normalize_by_ess=False, and once with True? So as to test both modes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@yannikschaelte
Copy link
Member

Great! As the tests do not seem to run, I will change base and from there do a PR do develop.

@yannikschaelte yannikschaelte changed the base branch from develop to ess_aware September 11, 2020 08:53
@yannikschaelte yannikschaelte merged commit 8a9b990 into ICB-DCM:ess_aware Sep 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants