-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Wei-Chun, Chang <[email protected]>
- Loading branch information
1 parent
3af8e62
commit 6fb722f
Showing
1 changed file
with
25 additions
and
17 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,13 +2,15 @@ | |
title: 5 Minutes Tutorial | ||
icon: material/school | ||
--- | ||
|
||
!!! Note | ||
|
||
Recce Cloud is currently in **private alpha** and scheduled for general availability later this year. [Sign up](../../index.md#signup) to the Recce newsletter to be notified, or email [[email protected]](mailto:[email protected]) to join our design partnership program for early access. | ||
|
||
**Jaffle Shop** is an example project officially provided by dbt-labs. This document uses [jaffle_shop_duckdb](https://github.com/dbt-labs/jaffle_shop_duckdb) to enable you to start using **recce cloud** from scratch within five minutes. | ||
|
||
## Clone Jaffle Shop to your Private Repositroy | ||
|
||
1. [Create a private repository](https://github.com/new) in your github account. | ||
1. Clone the **Jaffle Shop** dbt data project | ||
```shell | ||
|
@@ -19,22 +21,25 @@ icon: material/school | |
``` | ||
git remote set-url origin [email protected]:<owner>/<repo>.git | ||
``` | ||
1. Push to your new created repository. | ||
1. Push to your new created repository. | ||
``` | ||
git push | ||
``` | ||
|
||
## Authorize the repository to the Recce Cloud. | ||
|
||
1. Go to the [recce cloud](https://cloud.datarecce.io/). If it is your first time to login, please click the **Continue with Github** and authorize your github account to the **Recce Cloud** GitHub App. | ||
![alt text](../../assets/images/recce-cloud/sign-in.png) | ||
![alt text](../../assets/images/recce-cloud/sign-in-authorize.png) | ||
![alt text](../../assets/images/recce-cloud/sign-in.png) | ||
![alt text](../../assets/images/recce-cloud/sign-in-authorize.png) | ||
1. Click the **Install** button to install **Recce Cloud** github app to your personal or organization account. | ||
![alt text](../../assets/images/recce-cloud/app-install.png) | ||
1. In the app installation page in GitHub, authorize the new created repository to the app. | ||
![alt text](../../assets/images/recce-cloud/app-install-authorize.png) | ||
![alt text](../../assets/images/recce-cloud/app-install.png) | ||
1. In the app installation page in GitHub, authorize the new created repository to the app. | ||
![alt text](../../assets/images/recce-cloud/app-install-authorize.png) | ||
1. Then it will show up all the authorized repositories. | ||
![alt text](../../assets/images/recce-cloud/repo-list.png) | ||
![alt text](../../assets/images/recce-cloud/repo-list.png) | ||
|
||
## Prepare the base environment | ||
|
||
1. Prepare virtual env | ||
```shell | ||
python -m venv venv | ||
|
@@ -66,7 +71,7 @@ icon: material/school | |
- package: dbt-labs/audit_helper | ||
version: 0.12.0 | ||
- package: data-mie/dbt_profiler | ||
version: 0.8.2 | ||
version: 0.8.2 | ||
``` | ||
and run | ||
``` | ||
|
@@ -98,14 +103,17 @@ icon: material/school | |
``` | ||
|
||
## Prepare the review state for the PR | ||
|
||
As a PR author, you can prepare the recce review state and persist it in the recce cloud. | ||
|
||
1. Checkout a branch | ||
|
||
``` | ||
git checkout -b feature/recce-getting-started | ||
``` | ||
|
||
1. Prepare development environment. First, edit an existing model `./models/staging/stg_payments.sql`. | ||
|
||
```diff | ||
... | ||
|
@@ -119,24 +127,26 @@ As a PR author, you can prepare the recce review state and persist it in the rec | |
from source | ||
) | ||
``` | ||
|
||
run on development environment. | ||
|
||
```shell | ||
dbt seed | ||
dbt run | ||
dbt docs generate | ||
``` | ||
|
||
1. Commit the change | ||
|
||
``` | ||
git add models/staging/stg_payments.sql | ||
git add models/staging/stg_payments.sql | ||
git commit -m 'Update the model' | ||
git push | ||
``` | ||
|
||
1. Create a pull request for this branch in your github repository. | ||
1. Prepare a [github token](https://github.com/settings/tokens) in your account. You have to provide the `repo` permission. | ||
![alt text](../../assets/images/recce-cloud/github-token.png) | ||
|
||
1. Ensure you have configured these environment variables. | ||
```shell | ||
export GITHUB_TOKEN=<github-token> | ||
|
@@ -147,20 +157,21 @@ As a PR author, you can prepare the recce review state and persist it in the rec | |
recce server --cloud | ||
``` | ||
Open the link http://0.0.0.0:8000, you can see the lineage diff | ||
![](../assets/images/jaffle-shop/jaffle_shop_lineage.png) | ||
![](../../assets/images/jaffle-shop/jaffle_shop_lineage.png) | ||
1. Switch to the **Query** tab, add this query | ||
```sql | ||
select * from {{ ref("orders") }} order by 1 | ||
``` | ||
Add the primary key `order_id` and click the `Run Diff` button | ||
![alt text](../../assets/images/recce-cloud/query-diff.png){: .shadow} | ||
![alt text](../../assets/images/recce-cloud/query-diff.png){: .shadow} | ||
1. Click the `+` button to add the query result to checklist | ||
1. You can find that there are three checks in the **Checks** page | ||
1. Terminate the server. It would store the state to the recce cloud. | ||
1. In the GitHub PR page, we can find a failed check for this PR. This is because not all checks are approved. | ||
![alt text](../../assets/images/recce-cloud/pr-checks-wo-approved.png){: .shadow} | ||
## Review the PR | ||
As a PR author, you can review the PR by using the state stored in the recce cloud. If the checks are all good, you can approve them. | ||
1. Checkout the PR branch | ||
|
@@ -173,18 +184,15 @@ As a PR author, you can review the PR by using the state stored in the recce clo | |
export RECCE_STATE_PASSWORD=mypassword | ||
``` | ||
1. Run the recce server | ||
``` | ||
``` | ||
recce server --cloud --review | ||
``` | ||
1. You can see the lineage diff and the checklist prepared by the PR author. | ||
1. Approve all the checks if everything looks good to you | ||
![alt text](../../assets/images/recce-cloud/checks.png){: .shadow} | ||
1. Go back to the GitHub PR page, you can find that the recce check is marked as passed. | ||
![alt text](../../assets/images/recce-cloud/pr-checks-all-approved.png){: .shadow} | ||
!!! Note | ||
In this tutorial, we use duckdb as the warehouse, which is a file-based warehouse. The reviewer needs to have the same duckdb file to run the query. | ||
In this tutorial, we use duckdb as the warehouse, which is a file-based warehouse. The reviewer needs to have the same duckdb file to run the query. |