readme.md

Atlas Hub | A simple extract, batch job and script scheduler.

Website • Demo • Documentation • Chat

Atlas Hub is a task scheduling tool for getting data from a source and depositing it in a destination - sql servers to SFTP servers.

🤔 What Can It Do?

Tasks can run at any time and on any schedule.

Possibilities are almost unlimited....

• Run .bat files on windows servers over ssh
• Read, modify and resend files over FPT or SFTP
• ZIP and send data
• Run raw python code to process data and send
• Run code from FTP/SFTP/SAMB or web source
• Send output data embedded or attached to email
• Parameterize sql on the project or task level
• Parameterize file names with date parameters
• Export data as text, csv, excel, delimited, or as a blob
• Encrypt data before sending
• Pull and send data from FTP/SFTP/SAMB/SSH
• Run SSH commands to monitor remote servers

🏃 Start It Up

Atlas Hub can be run locally. We use pyenv and poetry to manage the project dependencies. Assuming you will too -

pyenv local 3.9.0
poetry install
npm install

# have you already created a database "atlas_hub_dev" and updated the config files?
FLASK_APP=web
flask db init
flask db migrate
flask db upgrade
flask cli seed
# if you want some basic demo information added
flask cli seed_demo

Finally, use the npm script to start up the website

npm start

🧪 Testing

Tests require a running postgresql and redis instances.

Start up a demo sql, sftp and ftp servers with docker:

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=@Passw0rd>" -p 1433:1433 --name sql1 -h sql1  -d mcr.microsoft.com/mssql/server:2017-latest
docker run -p 23:22 -d emberstack/sftp --name sftp
docker run -d --name ftpd_server -p 21:21 -p 30000-30009:30000-30009 -e FTP_USER_NAME=demo -e FTP_USER_PASS=demo -e FTP_USER_HOME=/home/demo -e "PUBLICHOST=localhost" -e "ADDED_FLAGS=-d -d" stilliard/pure-ftpd

Final, simply run the tests with tox.

🚀 Install

Atlas Hub is built for linux and only takes three commands to install.

🎁 Contributing

Contributions are welcome! Please open an issue describing an issue or feature.

This repository uses commitizen. Commit code changes for pr's with npm run commit.

🏆 Credits

Atlas was originally created by the Riverside Healthcare Analytics team. See the credits <https://www.atlas.bi/about/>_ for more details.

🔧 Tools

Special thanks to a few other tools used here.