BinderHub as a backend

[jtpio]

TLDR: It would be great for the Voila Gallery to use a BinderHub (mybinder.org) as a backend.

BinderHub as a backend for the gallery

The proposal is as follows:

• The Voila Gallery is nothing more but a collection of links to pre-built Binders
• It uses the BinderHub API to spawn the examples
• Static website built with Jekyll
• Hosted on GitHub Pages
• The examples stay in a yaml file (or multiple files)

Demo

Here is a repo that already implements this idea. The end result is relatively simple and minimal:

• Live at: https://voila-gallery.jtp.io
• Source: https://github.com/jtpio/voila-gallery-binderhub

Why

It's simpler, and simpler to operate.

This would fix #57, #55, #31, potentially #49 and close voila-gallery/voila-gallery.org-deploy#1.

For #21, we would make voila-gallery.org point to voila-dashboards.github.io and add a CNAME file that contains voila-gallery.org.

Also since making the example work on Binder is part of the steps to add it to the gallery, it sounds reasonable to launch that Binder image directly.

TLJH plugin

We can keep the TLJH plugin for the users who want to deploy the gallery on their own on a single machine using TLJH.

This proposed change is for the public facing gallery, with mybinder.org as the BinderHub. But the BinderHub host could technically be changed to something else in the config.

The repo for the public facing gallery would be https://github.com/voila-dashboards/voila-gallery.org and the content would be the same as https://github.com/jtpio/voila-gallery-binderhub.

The current repo would be renamed to https://voila-dashboards/tljh-voila-gallery, and the gallery.yaml simplified (less examples). The tljh_voila_gallery package could also be published to pypi.org.

Adding a new example

Pretty much the same as before:

• Same steps as described in https://github.com/voila-dashboards/gallery#contributing-new-examples (adapted for the new repo)
• When a PR is open, CI checks the repo has been built on mybinder.org with the ref (trigger a build otherwise)
• Merging into master deploys to GitHub pages automatically

Pros

• Simpler setup
• Static web site
• Hosted on GitHub pages
• We can still keep the plugin for those who want to deploy the gallery on one machine using TLJH
• The gallery can have links to dashboards deployed on Heroku
• Opening a PR to add a new example will make sure the ref has been built by Binder before it can be merged
• Very low maintenance (no need to deploy changes and manage servers)
• Save $100/month that could be invested in the OVH Binder cluster instead

Cons

• Slightly longer to launch the examples
• Puts the load on mybinder.org? The number of concurrent users isn't high so that might not be such an issue. Also the gallery will only link to repos that have already been built (pinned commit hash)

Transition

Here are the steps to transition voila-gallery.org to this setup:

• Move https://github.com/jtpio/voila-gallery-binderhub to https://github.com/voila-dashboards/voila-gallery.org
• Update README.md, CONTRIBUTING.md and the steps to add new examples
• Add CNAME
• Point the voila-gallery.org DNS entry to voila-dashboards.github.io
• Add a CI script to validate the new examples are built on mybinder.org

I would be happy to hear other opinions on this proposal!

cc @pbugnion @yuvipanda @SylvainCorlay

Also pinging @betatim since it's related to mybinder.org (event though this is not a lot of traffic)

[betatim]

Sounds good to me. Can different voila gallery entries use the same repo? Then you could perform two optimisations:

1. reuse an already running binder when the user clicks on an example that uses the same repo (the second example would start more or less instant)
2. get faster launch times because chances are higher that someone else launched the same repo already

[SylvainCorlay]

Transition

Here are the steps to transition voila-gallery.org to this setup:

• Move https://github.com/jtpio/voila-gallery-binderhub to https://github.com/voila-dashboards/voila-gallery.org
• Update README.md, CONTRIBUTING.md and the steps to add new examples
• Add CNAME
• Point the voila-gallery.org DNS entry to voila-dashboards.github.io
• Add a CI script to validate the new examples are built on mybinder.org

I would be happy to hear other opinions on this proposal!

I am +1 on this plan. This all sounds really reasonable.

Quick note on the hosting on github pages, since voila-dashboards.github.io is already used for the team compass, we may want to use voila-gallery.github.io instead.

[SylvainCorlay]

Do we want to only allow for one version of a binder to be on the gallery? (i.e. should people be able to update the ref for an accepted dashboard without approval from the gallery maintainers)

[jtpio]

Thanks @betatim for your input!

Sounds good to me. Can different voila gallery entries use the same repo?

Yes they can.

Then you could perform two optimisations:

1. reuse an already running binder when the user clicks on an example that uses the same repo (the second example would start more or less instant)

2. get faster launch times because chances are higher that someone else launched the same repo already

Reusing binders sounds like a great idea. And indeed we get much faster launch times (since there is nothing to launch).

But that also means keeping track of the url and the token? And making sure that binder is up before redirecting.

[jtpio]

Quick note on the hosting on github pages, since voila-dashboards.github.io is already used for the team compass, we may want to use voila-gallery.github.io instead.

I was under the impression that GitHub handles the redirect like this:

• voila-dashboards.github.io doesn't have a CNAME so https://voila-dashboards.github.io will redirect to its GH pages
• if voila-gallery.org points to voila-dashboards.github.io, then GitHub would check repos under https://github.com/voila-dashboards and find the one for which the CNAME matches voila-gallery.org

Would need to double check that. Otherwise happy to use voila-gallery.github.io too.

Do we want to only allow for one version of a binder to be on the gallery? (i.e. should people be able to update the ref for an accepted dashboard without approval from the gallery maintainers)

But if they update the ref, that means the gallery should point to the new one?

It would be great if they could update it independently yes. We would need to make sure the binder is built before exposing it in the gallery though.

[jtpio]

Transition

...

* Move https://github.com/jtpio/voila-gallery-binderhub to https://github.com/voila-dashboards/voila-gallery.org

For the transition, instead of just moving the POC repo, we should extract the files that we need, so we keep the git history and the contributors are still credited in the new repo.