Source code

Athena service roles are installed via athena-services command. To add new service it is necessary to add ansible role in ~/git/athena/athena-ansible/src/main/docker/roles/ directory.

Ports

In case if service needs external connectivity it is necessary to define service port in ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

User IDs

Ansible role has to provision dedicated Host User in case if it deploys docker service to make sure it runs with non-privileged user.

Since it is not possible to map user Id in Host operating system to User Id inside docker container it is necessary to use same User Id in both Host and Docker container. User IDs are defined in ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

Defaults

All role defaults (such as DB name, Docker image name, etc) are defined in ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

Docker image

To add new Docker image it is necessary to add new project in ~/git/athena/athena-docker/ and add this new project as module in ~/git/athena/athena-docker/pom.xml

Playbook

To make sure new service role is executed during athena-services run it is necessary to add it in ~/git/athena/athena-ansible/src/main/docker/services.yml. To understand in which zone service has to be deployed please refer to Security zones documentation.

Service discovery in WAF and ELB

If service has to be discovered by WAF or by any other services it is necessary to register it in Consul

Common service tags:

  • http - all services that are tagged with http will be discovered by WAF and exposed as https://<service>-<owner>-<env>.<route53domain>, for example wordpress service for owner Athena in environment DEV is exposed as https://wordpress-athena-dev.athenapaas.com
  • elb - all services that are tagged with elb are exposed as publicly available sites via AWS ELB as https://<service>-<owner>-<env>-public.<route53domain>, for example wordpress service for owner Athena in environment PROD is exposed as https://wordpress-athena-prod-public.athenapaas.com and http://wordpress-athena-prod-public.athenapaas.com. To make it available as www.athenapaas.com it is necessary to cname it.

Dependencies

If service requires external dependencies such as database or a certain service API it can consume them by service name, for example to connect to postgres service it can be referred to as postgres.service.consul

Infrastructure

If service requires infrastructure changes such as opening of a new port it is necessary to do changes in infrastructure playbooks. For example to open port it is necessary to add it to ~/git/athena/athena-ansible/src/main/docker/roles/vpc/tasks/security.yml

Example service

Source code

  • Role directory: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/

  • Role tasks directory: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/tasks

  • Main role task: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/tasks/main.yml


- assert:
    that:
      - wordpress_uid is defined
      - wordpress_service_type is defined
      - wordpress_db_name is defined
      - wordpress_port is defined
      - wordpress_distributed is defined

- name: create wordpress user
  user:
    system: yes
    createhome: yes
    shell: /bin/false
    uid: "{{wordpress_uid}}"
    name: wordpress
  become: yes
  register: uid

- name: find DB
  set_fact:
    db: "{{item}}"
    db_id: "{{item.target.split('.')[0]}}"
    db_user: "{{vpc_name}}"
    db_endpoint:
      Address: "{{item.target}}"
      Port: "{{item.port}}"
  with_items: "{{lookup('dig','_mysql._'+wordpress_service_type+'.service.consul./SRV','flat=0',wantlist=True)}}"
  delegate_to: 127.0.0.1

- name: create wordpress db
  local_action:
    module: mysql_db
    collation: "utf8_general_ci"
    encoding: utf8
    name: "{{ wordpress_db_name }}"
    login_user: "{{ db_user }}"
    login_host: "{{ db_endpoint.Address }}"
    login_password: "{{ lookup('password' ,lookup('env','ANSIBLE_DATA')+'/passwords/rds/'+db_id) }}"
    login_port: "{{ db_endpoint.Port }}"
  run_once: true

- name: create wordpress db user
  local_action:
    module: mysql_user
    name: "{{ wordpress_db_name }}"
    login_user: "{{ db_user }}"
    login_host: "{{ db_endpoint.Address }}"
    login_password: "{{ lookup('password' ,lookup('env','ANSIBLE_DATA')+'/passwords/rds/'+db_id) }}"
    password: "{{ lookup('password' ,lookup('env','ANSIBLE_DATA')+'/passwords/rds/db/'+db_id+'/'+wordpress_db_name + ' chars=ascii_letters,digits,hexdigits')}}"
    login_port: "{{ db_endpoint.Port }}"
    priv: "{{wordpress_db_name}}.*:ALL"
    host: "%"
  run_once: true

- name: create wordpress glusterfs directory for data volume
  file: path=/var/data/glusterfs/volumes/wordpress/var/www/html state=directory mode=755 owner=wordpress group=wordpress
  become: yes
  when: wordpress_distributed

- name: create wordpress local directory
  file: path=/var/data/wordpress/local/wp-admin state=directory mode=755 owner=wordpress group=wordpress
  become: yes
  when: wordpress_distributed

- name: launch distributed docker wordpress data image
  docker:
    name: wordpress-data
    image: "{{wordpress_image}}"
    state: present
    insecure_registry: yes
    command: /bin/true
    volumes:
      - /var/data/glusterfs/volumes/wordpress/var/www/html:/var/www/html
      - /var/data/wordpress/local:/var/www/local
    pull: always
  become: yes
  when: wordpress_distributed

- name: launch docker wordpress data image
  docker:
    name: wordpress-data
    image: "{{wordpress_image}}"
    state: present
    insecure_registry: yes
    command: /bin/true
    pull: always
  become: yes
  when: not wordpress_distributed

- name: launch docker wordpress image
  docker:
    name: wordpress
    labels:
      AthenaServiceName: "wordpress"
      AthenaLogType: generic1
    image: "{{wordpress_image}}"
    state: reloaded
    net: "{{docker_network_name}}"
    detach: true
    insecure_registry: yes
    restart_policy: always
    pull: always
    volumes_from:
      - wordpress-data
    env:
      WORDPRESS_DB_HOST: "{{ db_endpoint.Address }}"
      WORDPRESS_DB_NAME: "{{ wordpress_db_name }}"
      WORDPRESS_DB_USER: "{{ wordpress_db_name }}"
      WORDPRESS_DB_PASSWORD: "{{ lookup('password' ,lookup('env','ANSIBLE_DATA')+'/passwords/rds/db/'+db_id+'/'+wordpress_db_name) }}"
    ports:
      - "{{ wordpress_port }}:80"
  become: yes

- name: upload site root htaccess file
  template:
    dest: "/var/data/wordpress/local/.htaccess"
    force: yes
    src: htaccess-root.j2
    owner: wordpress
    group: wordpress
  become: yes
  when: wordpress_distributed

- name: upload deny folder access htaccess file
  template:
    dest: "/var/data/wordpress/local/wp-admin/.htaccess"
    force: yes
    src: htaccess-deny.j2
    owner: wordpress
    group: wordpress
  become: yes
  when: wordpress_distributed

- name: upload wp-config.php
  template:
    dest: "/var/data/wordpress/local/wp-config.php"
    force: yes
    src: wp-config.php.j2
    owner: wordpress
    group: wordpress
  become: yes
  when: wordpress_distributed

- name: remove wordpress authoring capability
  docker:
    name: busybox
    image: busybox
    log_driver: json-file
    detach: False
    command: >
      sh -c '
      rm /var/www/html/.htaccess &&
      ln -s /var/www/local/.htaccess /var/www/html/.htaccess &&
      rm /var/www/html/wp-admin/.htaccess &&
      ln -s /var/www/local/wp-admin/.htaccess /var/www/html/wp-admin/.htaccess &&
      rm /var/www/html/wp-config.php &&
      ln -s /var/www/local/wp-config.php /var/www/html/wp-config.php
      '
    volumes_from:
      - wordpress-data
  become: yes
  when: wordpress_distributed

- name: check if http service is up
  local_action: "wait_for host={{ inventory_hostname }} port={{wordpress_port}} delay=1 timeout=300"
  when: wordpress_service_type == 'Backoffice'

- name: register service
  consul:
    service_name: "wordpress"
    service_port: "{{wordpress_port}}"
    script: "nc -vz {{ec2_private_ip_address}} {{wordpress_port}}"
    interval: "5s"
    port: "{{consul_api_port}}"
    tags:
      - http

- name: register service
  consul:
    service_name: "wordpresselb"
    service_port: "{{wordpress_port}}"
    script: "nc -vz {{ec2_private_ip_address}} {{wordpress_port}}"
    interval: "5s"
    port: "{{consul_api_port}}"
    tags:
      - elb
  when: wordpress_service_type == 'Public'

  • Role template directory: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/templates

  • Apache htaccess config to deny unwanted access to sensitive wordpress APIs: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/templates/htaccess-deny.j2


{% if wordpress_service_type == 'Public' %}

SetEnvIfExpr "%{REQUEST_URI} =~ m#/wp-admin/admin-ajax\.php$# && %{QUERY_STRING} =~ m#^action=inwave_color.*$#" uriok=1
Order Deny,Allow
Deny from all
Allow from env=uriok

{% endif %}

  • Apache htaccess config to redirect to index.php and HTTPS: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/templates/htaccess-root.j2

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

{% if wordpress_service_type == 'Public' %}
<IfModule mod_rewrite.c>

RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

RewriteCond %{REQUEST_URI} wp-login
RewriteRule . / [R,L]
</IfModule>

Options -Indexes
{% endif %}

  • Wordpress configuration: ~/git/athena/athena-ansible/src/main/docker/roles/wordpress/templates/wp-config.php.j2

<?php

$_SERVER['HTTPS'] = 'on';

{% if wordpress_service_type == 'Backoffice' %}

define('WP_HOME','https://wordpress-{{vpc_name|lower}}-{{vpc_env|lower}}.{{route53_domain}}');
define('WP_SITEURL','https://wordpress-{{vpc_name|lower}}-{{vpc_env|lower}}.{{route53_domain}}');
define('WP_SITEURL','https://wordpress-{{vpc_name|lower}}-{{vpc_env|lower}}.{{route53_domain}}');

{% endif %}

define('DB_NAME', '{{wordpress_db_name}}');
define('DB_USER', '{{wordpress_db_name}}');
define('DB_PASSWORD', '{{ lookup('password' ,lookup('env','ANSIBLE_DATA')+'/passwords/rds/db/'+db_id+'/'+wordpress_db_name) }}');
define('DB_HOST', '{{db_endpoint.Address}}');
define('DB_CHARSET', 'utf8');
define('DB_COLLATE', '');

define('AUTH_KEY',         'ju]*1?<ZP-6t?A7W:!%LcR@u{2^11tUHF*r(_c^&eZ(B/t+)}QaB1*FwpP+nd');
define('SECURE_AUTH_KEY',  'DvNyRXby/M;F$DUkavj(`p[`ur;ny=dw!jRG]pYC+[q*>wLn|}8qu~l$#Y8:]');
define('LOGGED_IN_KEY',    'IZP[n9+0@c+R6Au;!PR41,Ts.%%xSA`B8PBVwUC~] RU(h;F!$+IF|%_EQnCb');
define('NONCE_KEY',        'mOijX?G|VW3[ub,/Z$%l1[|j&4l(OW{^Bbl@`P>@a+R|g+gybmBB^yn{s<CNd');
define('AUTH_SALT',        'a_<>e!r!bmw%ofd<Ef|X:BCnbKZteIzE*Nq+/pE308Z/qc|L,8e>_Nbemt_PK');
define('SECURE_AUTH_SALT', 'kI^re]34J`g**L>A|.-kNaVKVJ3=_>_S=1~c%&B2A9>joD=~gL-(.NSSZ(K(c');
define('LOGGED_IN_SALT',   '@snE:4S->w+F&o(?L0.gj,*QZem3/%mt!-*0Rj+0ypXG%}AA6-T!c aiM/YAJ');
define('NONCE_SALT',       'Yfk;+C+MI#A8PK8m+_b%8s2iv-sSUa,-?sCn|gEF=Aa%8M?vpY]y^@8b-7X.0');

$table_prefix  = 'whmcs_';

define('WP_POST_REVISIONS', 3);
define('EMPTY_TRASH_DAYS', 10);
define('AUTOSAVE_INTERVAL', 160);

define('WP_DEBUG', false);

if ( !defined('ABSPATH') )
  define('ABSPATH', dirname(__FILE__) . '/');

require_once(ABSPATH . 'wp-settings.php');

Ports

  • In ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

wordpress_port: 10980

User IDs

  • In ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

wordpress_uid: 10017

Defaults

  • In ~/git/athena/athena-ansible/src/main/docker/roles/defaults/defaults/main.yml

# Wordpress database name
wordpress_db_name: wordpress 

# Deploy Wordpress service
deploy_service_wordpress: false

# Deploy wordpress on multiple nodes
wordpress_distributed: false

# Wordpress version
wordpress_image: "{{docker_registry_host}}/athena-wordpress:{{platform_version}}"

# Docker images backup list (with other images removed)
docker_images_backup_list:
  - athena-wordpress:{{platform_version}}


Docker image

  • In ~/git/athena/athena-docker/pom.xml:
  <modules>
...
    <module>wordpress</module>
...
  </modules>
  • Service docker project directory ~/git/athena/athena-docker/wordpress/

  • Gitignore file ~/git/athena/athena-docker/wordpress/.gitignore

target/
  • Maven build specification ~/git/athena/athena-docker/wordpress/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.knowledgeprice.athena.docker</groupId>
  <artifactId>athena-wordpress</artifactId>
  <version>2.508-SNAPSHOT</version>
  <packaging>pom</packaging>
  <name>Athena Wordpress</name>

  <parent>
    <groupId>com.knowledgeprice.athena</groupId>
    <artifactId>athena-docker</artifactId>
    <version>2.508-SNAPSHOT</version>
  </parent>

</project>
  • Service docker source directory ~/git/athena/athena-docker/wordpress/src/main/docker

  • Service Dockerfile ~/git/athena/athena-docker/wordpress/src/main/docker/Dockerfile

FROM wordpress

MAINTAINER rihards.freimanis@knowledgeprice.com

ENV WORDPRESS_UID=10017

RUN usermod -u ${WORDPRESS_UID} www-data && \
    groupmod -g ${WORDPRESS_UID} www-data && \
    chown -R www-data:www-data /usr/src/wordpress

RUN sed -i '/AccessFileName .htaccess/a RemoteIPHeader X-Forwarded-For' /etc/apache2/apache2.conf

RUN sed -i 's/LogFormat "%h %l %u/LogFormat "%a %l %u/g' /etc/apache2/apache2.conf

RUN a2enmod remoteip

VOLUME /var/www/local
  • Local development run shell file: ~/git/athena/athena-docker/wordpress/src/main/docker/run.sh

if [ -z "$1" ]; then VERSION="0.0.1-SNAPSHOT"; else VERSION="$1"; fi

echo "Running with version=${VERSION}"

set -e

export TEST_ENV=nft
export WORDPRESS_DB_PASSWORD=$(cat $HOME/git/test/ansible-data-${TEST_ENV}/passwords/rds/db/test${TEST_ENV}dbmysql94/wordpress)


# Image name
IMG=registry-athena-dev.athenapaas.com/athena-wordpress:${VERSION}

if [ "$VERSION" == "0.0.1-SNAPSHOT" ];
then docker build --rm -t ${IMG} . ;
fi

docker ps -a | grep wordpress-data | cut -f 1 -d ' ' | xargs docker rm -v && \
docker create --name=wordpress-data ${IMG} && \
docker run \
  -p 10980:80 \
  --volumes-from wordpress-data \
  -e "WORDPRESS_DB_HOST=mysql.service.consul" \
  -e "WORDPRESS_DB_USER=wordpress" \
  -e "WORDPRESS_DB_PASSWORD=${WORDPRESS_DB_PASSWORD}" \
  -e "WORDPRESS_DB_NAME=wordpress" \
  --rm ${IMG}

Playbook

  • In Services playbook ~/git/athena/athena-ansible/src/main/docker/services.yml

# Wordpress Gluster Volume host discovery
- hosts: Public:Backoffice
  user: "{{host_user}}"
  roles:
    - 
      role: defaults
      tags: 
        - glusterfs
    - 
      role: set-service
      service_name: wordpress
      service_port: "{{gluster_daemon_port}}"
      service_tag: gluster
      tags: 
        - glusterfs
    - 
      role: glusterfs-volume
      glusterfs_name: wordpress
      tags: 
        - glusterfs


# Deploy worpress on public nodes
- hosts: Public
  user: "{{host_user}}"
  roles:
    - 
      role: defaults
      tags: 
        - wordpress
    - 
      role: wordpress
      wordpress_service_type: Public
      tags: 
        - wordpress


# Main Backoffice play (with other Backoffice roles not included)
- hosts: Backoffice
  user: "{{host_user}}"
  roles:
    - 
      role: defaults
      tags: 
...
        - wordpress 
...
    -
       role: wordpress
      wordpress_service_type: Backoffice
      tags: 
        - wordpress
...

Service discovery in WAF and ELB

If deployed in PROD environment service will be available as https://wordpress-athena-prod.athenapaas.com (WAF - content editing), https://wordpress-athena-prod-public.athenapaas.com and http://wordpress-athena-prod-public.athenapaas.com (ELB - published content)

Dependencies

Wordpress consumes deployed AWS RDS DB instance via mysql.service.consul (discovered via looking up mysql consul service tag)

Infrastructure

  • In ~/git/athena/athena-ansible/src/main/docker/roles/vpc/tasks/security.yml

- name: create Backoffice security group
  local_action:
    module: ec2_group
    name: "{{vpc_name}}{{vpc_env}}Backoffice"
    description: "Backoffice security group"
    vpc_id: "{{vpc.vpc_id}}"
    region: "{{vpc_region}}"
    purge_rules: true
    rules:
...
      - proto: tcp 
        from_port: "{{wordpress_port}}"
        to_port: "{{wordpress_port}}"
        cidr_ip: "{{ vpc_cidr }}.0.0/16"
...

- name: create Public security group
  local_action:
    module: ec2_group
    name: "{{vpc_name}}{{vpc_env}}Public"
    description: "Public security group"
    vpc_id: "{{vpc.vpc_id}}"
    region: "{{vpc_region}}"
    purge_rules: true
    rules:
...
      - proto: tcp 
        from_port: "{{wordpress_port}}"
        to_port: "{{wordpress_port}}"
        cidr_ip: "{{ vpc_cidr }}.0.0/16"
...