Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL integration #11

Merged
merged 7 commits into from
Mar 7, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ sympa_template_lists: []
# ssl: true
# queue_transport: sympa
# bouncequeue_transport: sympabounce

## Database variables
sympa_db_type: mysql # mysql or PostgreSQL
sympa_install_db_package: False # yes for this role to install mysql or postgres, no if installed elsewhere
sympa_db_name: sympa
sympa_db_host: localhost
sympa_db_port: 3306
Expand Down Expand Up @@ -113,4 +117,5 @@ sympa_default_home: home
sympa_edit_list: owner
sympa_ldap_force_canonical_email: 1
sympa_review_page_size: 25
sympa_webserver_type: Other # 'Other' or 'Apache 2'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets use lowercase strings without a space

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this variables is there to set a debconf parameter of select type. When you install sympa manually, and after checking the debconf db with debconf-get-selections it clearly proposes "Other" and "Apache 2".

I did some test, pre-setting the parameter with:

echo "sympa wwsympa/webserver_type select <value>" | debconf-set-selections

trying with <value> set to Other, other, apache2, Apache 2, etc.

Without changing debconf priorities, when setting the value to Other it silently preserves this value. When setting other values, it silently overwrites to the default value Apache 2.

So I think that there, we have to stick to the exact spelling proposed for the two debconf select options.

BTW, I struggled a lot with sympa debconf parameter. Sympa has a quite peculiar behaviour: most of the parameters are low o medium priority (so they aren't asked for), several are "internal", and most of all, they seem to be heavily overwritten at installation with default values, eventually built from linux hostname.

sympa_web_page_title: Mailing lists service
67 changes: 37 additions & 30 deletions tasks/main.yml
Original file line number Diff line number Diff line change
@@ -1,36 +1,41 @@
- name: Set database to mysql
debconf:
name: sympa
question: sympa/database-type
value: mysql
vtype: select
---
## Sympa configuration main tasks file

- name: Set webserver to other
debconf:
name: sympa
question: wwsympa/webserver_type
value: Other
vtype: select
- name: Check that "debconf" and "debconf-utils" are installed
apt:
name:
- debconf
- debconf-utils
state: latest
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this requirements' check, as suggested by Ansible documentation, even if it seems to work without debconf-utils package.


- name: MySQL/MariaDB configuration
include_tasks: mysql.yml
when: sympa_db_type | lower == 'mysql'

- name: "Set dbconfig-install to no"
- name: PostgreSQL configuration
include_tasks: postgresql.yml
when: sympa_db_type | lower == 'postgresql'

- name: Set debconf options for sympa
debconf:
name: sympa
question: sympa/dbconfig-install
value: no
vtype: boolean
changed_when: false

- name: Create a new database with name '{{ sympa_db_name }}'
mysql_db:
name: "{{ sympa_db_name }}"
encoding: utf8

- name: Create DB user '{{ sympa_db_user }}'
mysql_user:
name: "{{ sympa_db_user }}"
password: "{{ sympa_db_password | mandatory }}"
priv: '{{ sympa_db_name }}.*:ALL,GRANT'
no_log: True
name: "{{ item.name }}"
question: "{{ item.question }}"
value: "{{ item.value }}"
vtype: "{{ item.vtype }}"
loop:
## set the database type
- { name: 'sympa', question: 'sympa/database-type', value: '{{ sympa_db_type }}' , vtype: select }
## Configure the database at sympa installation with dbconfig
- { name: 'sympa', question: 'sympa/dbconfig-install', value: 'true' , vtype: boolean }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want this setting? Arent we configuring the database on our own

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried again: for PostgreSQL, when I set this to false, sympa installation fails with the message:

Database sympa defined in sympa.conf has not the right structure or is unreachable. verify db_xxx parameters in sympa.conf

(and I tried again with postgresql or PostgreSQL for sympa parameter db_type, and that's the same).

For MySQL, when I tried yesterday there was no difference wether I put true or false. I'll try to figure out what is the difference between PostgreSQL and MySQL installation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm too puzzled with this dbconfig parameter. I think my initial interpretation was right: seeing sympa.conf file, sympa tries an upgrade but, with PostreSQL, fails when running:

# /usr/lib/sympa/bin/sympa.pl --health_check

One strange thing is that I don't find any reference to db_passwd in database drivers code. dbconfig must do something that I don't identify, that solves this issue.

In order to avoid staying stucked, I propose you to set sympa/dbconfig-install to true for PostgreSQL and to maintain it set to false for MySQL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible solution here. I'll test again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible solution here. I'll test again.

Eureka! Yes, it did!

## Set the sympa database name
- { name: 'sympa', question: 'sympa/db/dbname', value: '{{ sympa_db_name }}' , vtype: string }
## Set the sympa database user
- { name: 'sympa', question: 'sympa/db/app-user', value: '{{ sympa_db_user }}' , vtype: string }
## Use the localhost for database (and not a distant host to define)
- { name: 'sympa', question: 'sympa/remote/host', value: 'localhost' , vtype: select }
## Set the webserver type
- { name: 'sympa', question: 'wwsympa/webserver_type', value: '{{ sympa_webserver_type }}' , vtype: select }


- name: Create sympa dir
file:
Expand Down Expand Up @@ -173,3 +178,5 @@
minute: "0"
hour: "2"
job: "/usr/lib/sympa/bin/sympa.pl --reload_list_config"

...
52 changes: 52 additions & 0 deletions tasks/mysql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
## MySQL configuration for sympa

- name: Check MySQL installation
block:
- name: Gather installed packages
package_facts:
manager: auto

- name: Check if mysql server package is installed
assert:
that: >
ansible_facts.packages['default-mysql-server'] is defined or
ansible_facts.packages['mysql-server'] is defined or
ansible_facts.packages['mariadb-server-10.1'] is defined
## The last condition could be replaced by a jinja2 json_query filter to match any version
fail_msg: "No mysql package found. We stop, because we can't install sympa without its database. Sorry."
success_msg: "mysql package found. We can continue!"
when: not sympa_install_db_package

- name: Install MySQL packages
apt:
name:
- default-mysql-server
- python-dev
- default-libmysqlclient-dev
state: present
when: sympa_install_db_package

- name: Install pip, if not yet installed
apt:
name: python-pip
state: present

- name: install mysqlclient pip module, if not yet installed
pip:
name: mysqlclient
state: present

- name: Create a new database with name '{{ sympa_db_name }}'
mysql_db:
name: "{{ sympa_db_name }}"
encoding: utf8

- name: Create DB user '{{ sympa_db_user }}'
mysql_user:
name: "{{ sympa_db_user }}"
password: "{{ sympa_db_password | mandatory }}"
priv: '{{ sympa_db_name }}.*:ALL,GRANT'
no_log: True

...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are those dots for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See hereafter.

62 changes: 62 additions & 0 deletions tasks/postgresql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
## PostgreSQL installation and configuration for sympa

- name: Check PostgreSQL installation
block:
- name: Gather installed packages
package_facts:
manager: auto

- name: Check if postgresql server package is installed
assert:
that: ansible_facts.packages['postgresql'] is defined
fail_msg: "No postgresql package found. We stop, because we can't install sympa without its database. Sorry."
success_msg: "postgresql package found. We can continue!"
when: not sympa_install_db_package

- name: Install PostgreSQL
apt:
state: present
update_cache: yes
cache_valid_time: 3600
name:
- postgresql
- postgresql-contrib
- libpq-dev
- python-psycopg2
- dbconfig-pgsql
when: sympa_install_db_package
tags: postgresql

- name: Create sympa database
become: true
become_user: postgres
postgresql_db:
name: "{{ sympa_db_name }}"
encoding: UTF-8
lc_collate: es_UY.UTF-8
lc_ctype: es_UY.UTF-8
template: template0
state: present
tags: postgresql

- name: Create sympa user with access to the database
become: true
become_user: postgres
postgresql_user:
db: "{{ sympa_db_name }}"
name: "{{ sympa_db_user }}"
password: "{{ sympa_db_password | mandatory }}"
priv: ALL
state: present
tags: postgresql

## Debconf keys for sympa database

- name: Define sympa postgresql database's password through debconf
debconf:
name: sympa
question: 'sympa/pgsql/app-pass'
value: '{{ sympa_db_password | mandatory }}'
vtype: password
...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are those dots for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's YAML syntax for the end of a YAML section. It's not needed by Ansible, but I prefer to stick to the upstream syntax over Ansible pidgin.

2 changes: 1 addition & 1 deletion templates/sympa.conf.j2
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ parsed_family_files {{ sympa_parsed_family_files }}
## db_type
## Type of the database (mysql|Pg|Oracle|Sybase|SQLite)
## Be careful to the case
db_type mysql
db_type {{ sympa_db_type }}

## db_name
## Name of the database
Expand Down