README.md

gut-data-version-control-

The objective of this project is used dvc in ML and Data Science projects. For this we'll store large files in MinIO.

MinIO's Setup

1.Install Go sudo passwd root

su root

apt update

wget -c https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz

tar xvf go1.14.2.linux-amd64.tar.gz

sudo chown -R root:root ./go

sudo mv go /usr/local

sudo echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile

source /etc/profile

rm go1.14.2.linux-amd64.tar.gz

go version

2. Install MinIO server

cd ~

wget https://dl.min.io/server/minio/release/linux-amd64/minio

sudo useradd --system minio --shell /sbin/nologin

sudo usermod -L minio

sudo chage -E0 minio

sudo mv minio /usr/local/bin

sudo chmod +x /usr/local/bin/minio

sudo chown minio:minio /usr/local/bin/minio

sudo touch /etc/default/minio

sudo echo 'MINIO_ACCESS_KEY="minio"' >> /etc/default/minio

sudo echo 'MINIO_VOLUMES="/usr/local/share/minio/"' >> /etc/default/minio

sudo echo 'MINIO_OPTS="-C /etc/minio --address :9000"' >> /etc/default/minio

sudo echo 'MINIO_SECRET_KEY="gutteste"' >> /etc/default/minio

sudo mkdir /usr/local/share/minio

sudo mkdir /etc/minio

sudo chown minio:minio /usr/local/share/minio

sudo chown minio:minio /etc/minio

3. Install initialize script Minio's systemd

cd ~

wget https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service

vim minio_service

[delete all and paste the code available in /setup/minio_service.py]

sudo mv minio.service /etc/systemd/system

4.Firewall's configuration

cd ~

sudo ufw default deny incoming

sudo ufw default allow outgoing

sudo ufw allow ssh

sudo ufw allow 9000

sudo ufw allow http

sudo ufw allow https

sudo ufw enable

sudo ufw allow 80

sudo ufw allow 443

sudo ufw status verbose

5. Start Minio's service

cd ~ sudo systemctl daemon-reload

sudo systemctl enable minio

sudo systemctl start minio

sudo systemctl status minio

q (to quit)

6. Install Minio Customer (MC)

cd ~

wget https://dl.min.io/client/mc/release/linux-amd64/mc

chmod +x mc

7. Generates TLS certificates to Minio service (http - https)

cd ~

sudo apt instalar software -properties-common

sudo add-apt-repository universe

sudo apt update

sudo apt install certbot

sudo certbot certonly --standalone -d minio.your_domain

sudo cp /etc/letsencrypt/live/minio.your_domain_name/privkey.pem /etc/minio/certs/

private.key sudo cp /etc/letsencrypt/live/minio.your_domain_name/fullchain.pem /etc/minio/certs/public.crt

sudo chown minio: minio /etc/minio/certs/private.key

sudo chown minio: minio /etc/minio/certs/public.crt

sudo systemctl restart minio