Skip to content

EuropeanaRS installation in a production environment

Jump to bottom Edit New page
Aldo edited this page Nov 4, 2015 · 8 revisions

How to install and deploy a EuropeanaRS instance in a production environment

  1. Prerequisites

  2. Initial configuration

    1. Gaining root access
    2. Updating your system
    3. Setting the default locale

  3. Installing the production environment

    1. Installing base packages
    2. Installing dependencies
    3. Installing ruby
    4. Installing Sphinx
    5. Installing postgres
    6. Passenger Install
    7. Allowing Apache access
    8. Enabling Apache mods

  4. Installing EuropeanaRS

    1. Creating the folder structure
    2. Downloading Sources
    3. Configuring database access
    4. Configuring EuropeanaRS settings
      1. Application Config
      2. Deployment Environment Config
      3. Apache Config

  5. Startup Scripts

    1. Startup For EuropeanaRS
    2. Startup For Sphinx
    3. Database and log Deployment
    4. Cron Task
    5. Convenience Symlink

  6. Final Deployment

Prerequisites

  • A computer with:
    • Ubuntu 14.04
    • Internet connection

Initial Configuration

Gaining root access

We will begin by ensuring our default user, from now on {username}, has root permission by adding it to the sudoers list:

sudo adduser {username} sudo

You will be prompted for the root password.

Updating your system

Next, we will ensure Ubuntu packages are updated to the latest version. For that purpose we will update the package database and the global system:

sudo apt-get update; sudo apt-get dist-upgrade

Setting the default locale

To ensure a sane fallback locale, execute the following commands:

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
sudo locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales
sudo update-locale LC_ALL="en_US.utf-8"

##Installing base packages

###Installing dependencies The following packages are needed dependencies for our environment, and can be installed by executing:

sudo apt-get install git-core libxml2-dev nodejs libxslt1-dev libmysqlclient-dev libsqlite3-dev  libpq-dev make apache2-threaded-dev apache2-mpm-worker libcurl4-openssl-dev

###Installing ruby First, we'll import the ruby repo gpg key:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

Next, we'll install a local ruby version 2.2.1 by rvm method

curl -L https://get.rvm.io | bash -s stable --ruby --autolibs=enable --auto-dotfiles
source /home/{username}/.rvm/scripts/rvm
echo 'gem: --no-rdoc --no-ri' >> ~/.gemrc 
gem install bundler
echo "export rvmsudo_secure_path=1">> /home/{username}/.bashrc

###Installing Sphinx EuropeanaRS requires sphinx 2.2.10 or higher. The easiest way to install a valid and compliant version of sphinx is to download and install the corresponding deb package:

wget http://sphinxsearch.com/files/sphinxsearch_2.2.10-release-0ubuntu12~precise_amd64.deb
sudo dpkg -i sphinxsearch_2.2.10-release-0ubuntu12~precise_amd64.deb

###Installing postgres EuropeanaRS requires postgres as database management system. To install it:

sudo apt-get install postgresql postgresql-contrib

To assign a password to the postgres user:

sudo -u postgres psql postgres
posgres#> ALTER USER postgres with password '{your_db_password}';
posgres#> \q

To create the europeanars_production database:

sudo -u postgres createdb europeanars_production

Passenger Install

To use the passenger app server for ruby, we'll install and configure it with the following commands:

sudo apt-get install libapache2-mod-passenger
sudo a2enmod passenger
sudo service apache2 restart

Installing EuropeanaRS

Creating the folder structure

We will create the folder structure by executing the following commands:

sudo mkdir /u
sudo chown {username}:root /u
mkdir /u/apps
mkdir /u/apps/europeanars
mkdir /u/apps/europeanars/shared/
mkdir /u/apps/europeanars/shared/log
mkdir /u/apps/europeanars/shared/pids

Downloading Sources

Next, we'll download the source code from git into /u/apps/europeanars by executing the following commands:

cd /u/apps/europeanars
git clone https://github.com/agordillo/EuropeanaRS current

Configuring database access

We will create the database file in /u/apps/europeanars/shared/database.yml , which will include the settings for accessing the postgresql database previously configured.

cp /u/apps/europeanars/current/config/database.yml.example /u/apps/europeanars/shared/database.yml

The /u/apps/europeanars/shared/database.yml file should have the following content:

production:
  adapter: postgresql
  database: europeanars_production
  encoding: utf8
  host: localhost
  username: postgres
  password: {your_db_password}

Allowing Apache access

To allow remote access to our server, we'll remove the deny all directives in /etc/apache2/apache2.conf by removing the following lines:

<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>
<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>
<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

Configuring EuropeanaRS settings

Application Config

The configuration of a EuropeanaRS instance can be specified through the application_config.yml file.

cp /u/apps/europeanars/current/config/application_config.yml.example /u/apps/europeanars/shared/application_config.yml

Edit the /u/apps/europeanars/shared/application_config.yml file to specify the settings of your EuropeanaRS instance.
For more information about how to edit this file, please check Setting up a EuropeanaRS instance.

production:
  secret_token: ""     # Execute 'bundle exec rake secret' to generate a new secret_token and copy it here
  europeana:
    api_key: ""        #Request an API key here: http://labs.europeana.eu/api/introduction
    private_key: ""
    oauth2_client_id: ""
    oauth2_client_key: ""
    oai_pmh_key: ""    #Request an access key for the OAI-PMH service here: http://labs.europeana.eu/api/oai-pmh-access-key
    # my_europeana: "mimic" #To use a mimic of the MyEuropeana API instead of the real one. This should be used only for development.
  facebook: #Get Facebook credentials for your app from https://developers.facebook.com/
    app_id: ""
    app_secret: ""
  mail:
    no_reply_mail: "[email protected]"
    main_mail: "[email protected]"
    type: "SMTP" #Type can be "SENDMAIL" or "SMTP" (default)
    # gmail_credentials:
    #   username: ""
    #   password: ""
  settings: #settings for the Recommender System
    resource_types: ["TEXT","VIDEO","SOUND","IMAGE","3D"]
    max_text_length: 50 #Maximun length in characters of the texts that will be taking into account when measuring semantic distances. High values could lead to performance issues.
    max_user_los: 2 #Maximum number of LOs of the users to be processed on the RS. High values could lead to performance issues.
    europeanars_database:
      max_preselection_size: 500 #Maximun number of Learning Objects retrieved by the search engine in the preselection phase of the recommendations. High values could lead to performance issues.
    europeana_database:
      max_preselection_size: 500 #Maximun number of Learning Objects retrieved from Europeana in the preselection phase. High values could lead to performance issues.
  default_settings: #Default settings for the Recommender System
    database: "EuropeanaRS" #database used for generate the recommendations. Accepted values: "EuropeanaRS", "Europeana", "All"
    query: ""
    preselection_filter_resource_type: "true"
    preselection_filter_languages: "true"
    europeanars_database:
      preselection_size: 500
    europeana_database:
      preselection_size: 500
  europeanars_api:
    require_key: "true" #if false, no app key would be required when using the EuropeanaRS API.
  weights:
    default_rs:
      los_score: 0.4
      us_score: 0.4
      quality_score: 0.1
      popularity_score: 0.1
    default_los:
      title: 0.2
      description: 0.15
      language: 0.5
      year: 0.15
    default_us:
      language: 0.5
      los: 0.5
    popularity:
      visit_count: 0.5
      like_count: 0.5
  filters:
    default_rs:
      los_score: 0
      us_score: 0
      quality_score: 0
      popularity_score: 0
    default_los:
      title: 0
      description: 0
      language: 0
      year: 0
    default_us:
      language: 0
      los: 0

Apache Config

To configure our web server, we need to create the file /etc/apache2/sites-available/europeanars.conf, with the following content:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName XXXX.org
        ServerAlias www.XXXX.org
        ServerAlias m.XXXX.org
        DocumentRoot /u/apps/europeanars/current/public/
        <Directory /u/apps/europeanars/current/public/>
                AllowOverride all
                Options -MultiViews
                AllowOverride FileInfo
        </Directory>
        ErrorLog ${APACHE_LOG_DIR}/error.log
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Next, we will create symbolic links to the apache configuration files:

sudo rm /etc/apache2/sites-enabled/000-default.conf
sudo ln -s /etc/apache2/sites-available/europeanars.conf /etc/apache2/sites-enabled/europeanars.conf

Startup Scripts

To enable automatic startup for EuropeanaRS and sphinx services we'll follow the next steps. Coming soon...

Final steps in the production machine

To finish the installation:

cp  /u/apps/europeanars/shared/database.yml /u/apps/europeanars/current/config/database.yml
cp  /u/apps/europeanars/shared/application_config.yml /u/apps/europeanars/current/config/application_config.yml
cd /u/apps/europeanars/current
bundle install --without development test
bundle exec rake db:populate:install_production RAILS_ENV=production
cd ..
sudo rm -rf /u/apps/europeanars/current/
sudo chown {username}:www-data /u/apps/europeanars/shared/log/searchd.production.pid
sudo chmod 777 /u/apps/europeanars/shared/log/searchd.production.pid

Set up development machine for deployment

The next steps are taken in a development machine.
To install a development environment you can read the following instructions: EuropeanaRS Installation

In the development machine, we'll customize the production environment based on our deployment file in europeanars/config/deploy/{environment}.yml. The europeanars/config/deploy/example.yml contains all the information that should be filled in:

message: "Deploying to EuropeanaRS"
repository: [email protected]:agordillo/EuropeanaRS.git
branch: master
server_url: 127.0.0.1 #put here the server URL (e.g. example.com)
username: ubuntu
keys: /home/username/.ssh/ssh-keypair #remove this line if you don't want to use ssh keys

Finally, we can deploy our EuropeanaRS application by executing:

bundle exec cap deploy DEPLOY={environment}

This way, if we define a production environment in europeanars/config/deploy/production.yml, we can deploy it with the following command:

bundle exec cap deploy DEPLOY=production

This way, we can specify several production environments for the same application in our development machine.

Add a custom footer
Add a custom sidebar
Clone this wiki locally