Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lab Guide #2 - Deploy an Extract-Load pipeline using Singer Taps on ECS #86

Open
8 of 11 tasks
aaronsteers opened this issue Apr 3, 2020 · 0 comments
Open
8 of 11 tasks

Lab Guide #2 - Deploy an Extract-Load pipeline using Singer Taps on ECS #86

aaronsteers opened this issue Apr 3, 2020 · 0 comments

Comments

@aaronsteers
Copy link
Contributor

Configure Extracts (4:00-6:00, approx. 2m):

  • Copy Pardot credentials file (tap-pardot-config.json) to the data/taps/.secrets folder (15s)
  • Open data/taps/data.select, delete Salesforce refs, update rules to include all columns on Pardot accounts and opportunities (45s)
  • Install dataops tools: pip3 install slalom.dataops (15s)
  • Run s-tap plan pardot to update Pardot extract plan (15s)
  • Review the updated plan file (30s)

Run a Sync Test (10:30-14:30, approx. 4m):

  • Switch to git tab, browse through all changes (30s)
  • Copy-paste and run the provided AWS User Switch command so aws-cli can locate our credentials (15s)
  • Copy-paste and run the provided Sync command to execute the Pardot sync in ECS (60s)
  • Click on the provided Logging URL link to open Cloudwatch logs in a browser (15s)
  • Wait for the ECS task to start (1-2m)
  • Stop the timer once rows are extracted (DONE!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@aaronsteers