-
Notifications
You must be signed in to change notification settings - Fork 35
4.0.0 Vireo Deployment AWS
This deployment was successfully tested on a AWS t2.small. A t2.micro did not have enough memory for java to create a jar file. Installations may vary on other systems.
A docker deployment is available at https://github.com/TexasDigitalLibrary/Vireo4Docker
The vireo install will need PostgreSQL. The H2 database can be used for testing but PostgreSQL will be needed for persistence across reboots. MySQL is not yet supported.
The base system will need npm, nodejs, grunt and maven. Nodejs is needed for processing assets.
The base system will also need git but this typically comes with the base linux system.
Vireo will also need java 1.8 developer kit.
Edits to src/main/resources/application.yaml will also be needed.
Finally the install should use a proxy web server such as Apache2 or Nginx.
sudo bash --login
Call apt-get update to be ready for other installs.
yum update
git --version
Typically it is already part of the base linux system but it may need to be explicitly installed. If so:
yum install git
wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
yum install -y apache-maven
yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel
There may have been an older java installed with the base system so find the newly installed java
sudo alternatives --config java
#select the correct number for java 1.8
java -version
export JAVA_HOME=/usr/lib/jvm/java-1.8.0
Set up an account under which the deploy will be performed. For these instructions we'll use the account named 'vireo.'
Add user 'vireo' and then enter a requested initial password.
adduser vireo4
passwd vireo4
Give user 'vireo4' root level privileges:
visudo
Add the following line beneath the similar line for root:
vireo4 ALL=(ALL) ALL
Exit editor using ':wq'
Alternatively you can use the vireo4 account with:
sudo -u vireo4 bash
This is the most system specific problem for deployments. This section may be different for later versions of ubuntu or AWS variants.
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
nvm --version
nvm install node
npm install -g grunt-cli
Put
export JAVA_HOME=/usr/lib/jvm/java-1.8.0
and
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
in ~/.bashrc
Uploaded documents are stored in the operating systems directory structure. Create a [assets_store] directory such as /opt/vireo for these files.
sudo mkdir /opt/vireo/
sudo chown vireo4 /opt/vireo/
sudo chgrp vireo4 /opt/vireo/
Sign in as vireo4. Create a directory in vireo4 such as /home/vireo4/etd/ in which to clone Vireo:
mkdir /home/vireo4/etd/
cd /home/vireo4/etd/
git clone https://github.com/TexasDigitalLibrary/Vireo.git
cd Vireo
Use the default master branch or change to the latest point release branch of Vireo4
git checkout 4.0.x
Try to start vireo4 using spring boot to verify that it has everything it needs to build and run.
mvn clean spring-boot:run -Drun.arguments=console
After several minutes of downloads and processing you will see a generate> prompt. If you get this far then your system is set up properly. Use Control-C or other means to kill this process. If it failed review previous steps and verify the version of OS you are using. Most problems are related to getting the right npm and node versions for the OS.
If you point your browser to your URL or IP address followed by the port number, e.g. http://example.edu:9000, you will see a poorly formatted vireo page. This is OK. The presentation of this page will be remedied by changes to application.yaml further down.
yum install postgresql postgresql-server
sudo service postgresql initdb
sudo service postgresql start
psql --version
sudo su - postgres
psql
This will give you a 'postgres=#' prompt.
Create role 'vireodb', give privileges, and create database vireo.
postgres=# CREATE ROLE vireodb WITH LOGIN PASSWORD '[your_vireo_postgres_password]';
postgres=# ALTER ROLE vireodb CREATEDB;
postgres=# CREATE DATABASE vireo;
postgres=# GRANT ALL PRIVILEGES ON DATABASE vireo TO vireodb;
postgres=# \q
Exit from the postgres account so you will be back in the vireo4 account.
exit
Edit /var/lib/pgsql9/data/pg_hba.conf Change lines: # "local" is for Unix domain socket connections only local all all peer # IPv4 local connections: host all all 127.0.0.1/32 ident to # "local" is for Unix domain socket connections only local all all md5 # IPv4 local connections: local all all md5
Restart postgres
service postgresql restart
Test with
psql [dbname] [username]
psql vireo vireodb
A few values will be set at the command line but some should be set in the file:
Set a port. For this install we will change port 9000 to port 8080 as we'll be running with embedded tomcat:
server:
#port: 9000
port: 8080
Set the database access by commenting out the H2 section and uncommenting the platform, url, and driverClassName in the PostgreSQL entry. Modify as needed for your particular setup (i.e. you elected a different port number):
spring:
datasource:
platform: postgresql
url: jdbc:postgresql://localhost:5432/vireo
driverClassName: org.postgresql.Driver
Also under datasource, set the database role name and password (as set in the Install PostreSQL section):
username: vireodb
password: [your_vireo_postgres_password]
Comment/uncomment the jpa entry to select the correct database-platform for PostgreSQL:
jpa:
database-platform: org.hibernate.dialect.PostgreSQLDialect
In the same section change:
hibernate.ddl-auto: create-drop
To:
hibernate.ddl-auto: update
Set the application URL with your url and port number, such as 'http://etd.example.edu:8080':
app:
url: [your url and port number]
Note that the values which appear under info: build: will be filled either via the pom.xml or the command line.
Finally, set the email relay and addresses:
email:
host: [your_smtp_relay] #e.g. smtp-relay.example.edu
from: [your_outbound_address] #e.g. [email protected]
replyTo: [your_replyto_address] #e.g. [email protected]
For app:url: if you don't yet know your url you can find the ip address with either curl http://checkip.amazonaws.com or curl http://icanhazip.com
If you want to give users some sample accounts to experiment with, you will need to start vireo enabled with a command line interface. Use the -Drun.argumets=console flag, which only works when started with spring-boot.
mvn clean spring-boot:run -Drun.arguments=console
Once you get the prompt:
generate> generate 3
generate> accounts 3
generate> admin_accounts 3
If you sign back in to the database you can see the accounts:
sudo su - postgres
psql
postgres=# \c vireo
postgres=# SELECT * FROM weaver_users;
postgres=# \q
exit
mvn clean package -Dproduction -Dassets.uri=file:[assets_path]
#e.g. mvn clean package -Dproduction -Dassets.uri=file:/opt/vireo
Run it with the following command using the appropriate war file name based on [version-tag]:
nohup java -jar target/vireo-4.0.0-[version-tag].jar &
e.g. nohup java -jar target/vireo-4.0.0-RC.jar &
Create file /etc/init.d/vireo4 which contains:
#!/bin/sh
SERVICE_NAME=vireo4
PATH_TO_JAR=/home/vireo4/etd/Vireo/target/vireo-4.0.0-RC.war
PID_PATH_NAME=/tmp/vireo4-pid
case $1 in
start)
echo "Starting $SERVICE_NAME ..."
if [ ! -f $PID_PATH_NAME ]; then
nohup sudo -u vireo4 java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is already running ..."
fi
;;
stop)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stopping ..."
kill $PID;
echo "$SERVICE_NAME stopped ..."
rm $PID_PATH_NAME
else
echo "$SERVICE_NAME is not running ..."
fi
;;
restart)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stopping ...";
kill $PID;
echo "$SERVICE_NAME stopped ...";
rm $PID_PATH_NAME
echo "$SERVICE_NAME starting ..."
nohup sudo -u vireo4 java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is not running ..."
fi
;;
esac
Set the permissions:
sudo chmod 0764 vireo4
Start with the command:
service vireo4 start
To test:
sudo reboot
Your URL can be found on the AWS console under EC2 listings. It is of the form 'ec2-[IP ADDRESS].compute-1.amazonaws.com e.g. ec2-111-222-333-444.compute-1.amazonaws.com
ec2-3-222-186-66.compute-1.amazonaws.com
This example uses nginx.
sudo yum install -y nginx
We'll need a certificate. You should use your institution's key, but for now we'll use a test key:
sudo cd /etc/pki/tls/private
sudo openssl genrsa -out custom.key
sudo chown root.root custom.key
sudo chmod 600 custom.key
sudo openssl req -new -key custom.key -out csr.pem
``
Follow the prompts
'Common Name' must be the url such as
The next command will provide a 'not available' error unless it is immediately preceded by an apt-get update:
sudo apt-get update sudo apt-get install python-certbot-nginx
Edit /etc/nginx/sites-available/default so that your URL replaces '_' after server_name such as:
server_name [your_url];
Create the certificates for nginx:
sudo certbot --nginx -d [your_url_with_no_protocol] e.g. sudo certbot --nginx -d etd.example.edu
You will be asked several questions via prompts. The third one is for redirecting HTTP traffic, respond with a '2' if you want all HTTP traffic redirected to HTTPS (recommended).
Try pointing your browser to your url with no port. you will see an nginx page.
Go back and edit default
sudo vi /etc/nginx/sites-enabled/default
You will see that the earlier certbot command added, port 443 and ssl_certificate related entries to default.
ec2-3-222-186-66.compute-1.amazonaws.com
Comment out:
#root /var/www/html;
and change <i>location</i> so it looks like:
location / { proxy_pass http://127.0.0.1:8080/; }
Since we are using a different protocol and port, go back and edit src/main/resources/application.yaml to change your <i>app:url:</i>:
app: url: https://etd.example.edu
Stop the vireo4 service, rebuild the war file, and restart the vireo4 service.
You may also need to clean your browser's cache, especially if you see the vireo page with no formatting or other assets.
Nginx should have picked up the location changes. If not then restart the service it with <i>systemctl</i>:
systemctl restart nginx
# Done!
You should be able to access vireo4 at your such as <i>https://etd.example.edu</i>
# Post Install
## Register
Register your account on the vireo4 website.
## Grant privileges
There are 4 types of accounts: Admin, Manager, Reviewer, and Student. The Admin account will manage the site for the whole institution.
You may need to manually set an initial Administrative permission on an account. This account can then be used to set permissions on other accounts through the user interface.
sudo su - postgres psql postgres=# UPDATE weaver_users SET role='ROLE_ADMIN' WHERE id=[user_id_in_weaver_users_table_you_want_to_make_admin]; postgres=#\q exit
## Database backups
While signed in under the unix vireo account create a directory for the backups and create a crontab entry
mkdir /home/vireo/backups/ crontab -e
Create an entry such as:
0 1 * * * pg_dump vireo > /home/vireo/backups/vireo_backup_date +\%Y_\%m_\%dT\%H:\%M:\%S.sql
This creates a file in your backups directory with a date stamp such as:
vireo_backup_2019_05_13T09:53:47.sql
To restore use:
sudo -u postgres psql postgres=# DROP DATABASE vireo; postgres=# CREATE DATABASE vireo;
postgres=# \q
sudo -u postgres psql -U postgres vireo < /tmp/vireo_backup_2019_05_13T09:53:47.sql nohup java -jar target/vireo-4.0.0-SNAPSHOT.jar &
# Miscellaneous
If you are having trouble accessing your database, verify the port on which which PostgreSQL is running. The default is 5432.
sudo -u postgres psql postgres=# SELECT * FROM pg_settings WHERE name = 'port'; postgres=# \q exit
If you prefer running vireo4 as a jar file this can be generated by changing one line in the </i>pom.xml</i> file:
Edit pom.xml to use jar instead of war. Change:
war
to
jar