Building an end-to-end data pipeline with Meltano + dbt + Metriql + Superset

[buremba]

You'll build an end-to-end modern data stack using Meltano + dbt + Metriql and Superset and show how well these tools work together. You'll come up with a possible use case and relevant dataset to demonstrate the use of these tools.

We will rn Meltano with Github Actions just like how I run dbt using Github workflow so that the end-to-end pipeline can be reproducible by the people who would like to try it out on their data. I can guide you on that once we get in that step.

We can also have a button similar to the Metriql Demo data repo that spins up Superset in addition to Metriql on Heroku (I have a sample setup for it) so that within one click they can have everything in place easily!

If you can work with the marketing data, we can use our sample data for Facebook or Google Adwords. That way, people can fork the repo, change their credentials and API keys, and end up on the same reporting screen but on top of their data. (https://hub.meltano.com/extractors/facebook)

[bashyroger]

+1 for this but I'd also suggest an alternative where the Extract-load (EL) component meltano is replaced with https://airbyte.io/
Rationale: Airbyte has much more community traction right now, based on 4000+ github stars for Airbyte for VS 500+ gitlab stars for Meltano

[buremba]

@bashyroger, we have plans to write blog posts about Airbyte and Fivetran as well. I just created #23, thanks!