* [Creating a cookbook for our CI server](#creating-a-cookbook-for-our-ci-server)
* [Adding a Test Machine](#adding-a-test-machine)
* [Automates the application deployment](#automates-the-application-deployment)
* [Deployment automation in Jenkins](#deployment-automation-in-jenkins)
* [Logging](#logging)
* [Monitoring](#monitoring)
* [Cloud infrastructure](#cloud-infrastructure)

# Creating a cookbook for our CI server

## Requirements

* [Chef Development Kit (ver. 0.11.0)](https://downloads.chef.io/chef-dk/)

## Project setup

```
git clone https://github.com/joebew42/dropwizard-sample-app.git
cd dropwizard-sample-app/cookbooks/
```

## Create the ci cookbook

```
chef generate cookbook ci
```

This will create a `scaffold` of the cookbook.

### Our first recipe

*recipes/default.rb*

```
include_recipe 'java'
```

How can I try it ? [`test-kitchen`](https://github.com/test-kitchen/test-kitchen) (aka kitchen) is a tool used to run integration tests ([`serverspec`]()) against a machine.

The kitchen workflow can described as follows:

* create
* setup
* converge
* verify
* destroy

Let's initialize kitchen files and directories:

```
kitchen init
```

*.kitchen.yml*

```
---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  client_rb:
    file_cache_path: '/var/chef/cache'

platforms:
  - name: ubuntu/trusty64
    driver:
      vagrantfile_erb: Vagrantfile

suites:
  - name: default
    run_list:
      - recipe[ci::default]
    attributes:
```

*Vagrantfile*

```
Vagrant.configure('2') do |config|
  config.vm.box = 'ubuntu/trusty64'
  config.vm.box_check_update = false
  config.vm.network :private_network, ip: '192.168.33.33'

  config.berkshelf.enabled = false

  if Vagrant.has_plugin?("vagrant-omnibus")
    config.omnibus.chef_version = 'latest'
  end

  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope = :box
    config.cache.auto_detect = false
    config.cache.enable :apt
    config.cache.enable :yum
    config.cache.enable :gem
    config.cache.enable :chef_gem
    config.cache.enable :generic, { :cache_dir => "/var/chef/cache" }
  end

  config.vm.provider "virtualbox" do |vb|
    vb.memory = 1024
  end
end
```

Now run `kitchen create` in order to create the machine used to run integration tests against

```
kitchen create
```

Apply chef cookbook:

```
kitchen converge
```

Ops! Some errors are thrown :/

### Fix our recipe first

Adding [`java`](https://supermarket.chef.io/cookbooks/java) as cookbook dependecy by putting this line in `metadata.rb`:

```
depends 'java', '~> 1.39.0'
```

Put some java's specific attributes in `attributes/java.rb`

```
default['java']['jdk_version'] = '7'
```

Now try to run a converge:

```
kitchen converge
```

Java is installed! :)

### How to verify that Java is installed ?

Here we use [`serverspec`](http://serverspec.org/), that is a set of test helpers that can be executed against each kind of machine (manually or automatically provisioned). Useful for *smoke tests*.

Create our first integration test that verifies if java is correctly installed:

*test/integration/default/serverspec/default_spec.rb*

```
require 'serverspec'

set :backend, :exec

describe 'ci::default' do
  describe command('/usr/bin/java -version') do
    its(:stderr) { should contain('1.7') }
  end
end

```
Execute the verification with:

```
kitchen verify
```

All (just one) tests passes! :D

### Put all things together

We want to install `jenkins` and configure it (ssh keys, directories, etc...)

#### Jenkins cookbook

Add the [`jenkins cookbook`](https://supermarket.chef.io/cookbooks/jenkins) as dependency in `metadata.rb`:

```
depends 'java', '~> 1.39.0'
depends 'jenkins', '~> 2.4.1'
```

Update berkshelp dependencies with `berks update`.

Put the `jenkins::master` in default.rb recipe:

```
include_recipe 'java'
include_recipe 'jenkins::master'
```

run the converge with `kitchen converge`

Visits: `http://192.168.33.33:8080` :D

#### Just one simple integration test

*test/integration/default/serverspec/default_spec.rb*

```
require 'serverspec'

set :backend, :exec

describe 'ci::default' do
  describe command('/usr/bin/java -version') do
    its(:stderr) { should contain('1.7') }
  end

  describe port(8080) do
    it { should be_listening }
  end
end
```

Run verification: `kitchen verify`

#### Complete the recipe

We are going to complete the recipe by installing `maven` and `mysql`

*metadata.rb*

```
depends 'java', '~> 1.39.0'
depends 'jenkins', '~> 2.4.1'
depends 'maven', '~> 2.1.1'
depends 'mysql', '~> 6.1.2'
depends 'mysql2_chef_gem', '~> 1.0.2'
depends 'database', '~> 4.0.9'
```

run `berks update`

*recipes/default.rb*

```
include_recipe 'java'
include_recipe 'maven'
include_recipe 'jenkins::master'

jenkins_plugin 'git'
jenkins_plugin 'greenballs'
jenkins_plugin 'junit'
jenkins_plugin 'jobConfigHistory'
jenkins_plugin 'delivery-pipeline-plugin' do
  notifies :restart, 'service[jenkins]', :delayed
end

package 'git'

mysql_service 'test' do
  port '3306'
  version '5.5'
  initial_root_password 'root'
  action [:create, :start]
end

mysql2_chef_gem 'default' do
  action [:install]
end

mysql_database 'db_notes_test' do
  connection(
    :host     => '127.0.0.1',
    :username => 'root',
    :password => 'root'
  )
  action :create
end
```

*test/integration/default/serverspec/default_spec.rb*

```
require 'serverspec'

set :backend, :exec

describe 'ci::default' do
  describe command('/usr/bin/java -version') do
    its(:stderr) { should contain('1.7') }
  end

  describe port(8080) do
    it { should be_listening }
  end
  
  describe service('mysql-test') do
    it { should be_enabled }
    it { should be_running }
  end
end
```

## Adding the CI machine in the root Vagrantfile

*Vagrantfile*

```
...
  config.vm.define "ci" do |ci|
    ci.vm.hostname = "ci"
    ci.vm.network :private_network, ip: '192.168.33.101'

    ci.vm.provider 'virtualbox' do |vb|
      vb.memory = 1024
    end

    ci.vm.provision :chef_zero, install: true do |chef|
      chef.verbose_logging
      chef.nodes_path = 'cookbooks'
      chef.file_cache_path = '/var/chef/cache'
      chef.add_recipe 'ci::default'
      chef.json = {}
    end
  end
...
```

And then adds the cookbook `ci` as dependecy

*Berksfile*

```
source 'https://supermarket.chef.io'

cookbook 'sample-app', path: './cookbooks/sample-app'
cookbook 'ci', path: './cookbooks/ci'
```

run `berks update`

# Adding a Test machine

The `test` machine is used to simulate a production like environment. We can use to test deploy task or as a `staging` phase.

The cookbook used to proviion the test machine is the same used for `dev` environment, with small changes.

```
cd cookbooks/sample-app
```

We have to modify the `default` recipe:

*recipes/default.rb*

```
...
['db_notes', 'db_notes_test'].each do |database_name|
  mysql_database database_name do
    connection(
      :host     => '127.0.0.1',
      :username => 'root',
      :password => 'root'
    )
    action :create
  end
end
...
```

In a `test` environment we don't need a database used for integration, so we can continue by extracting the list `['db_notes', 'db_notes_test']` as attribute of the cookbook, in order to assign new values programmatically during the provision.

*atributes/default.rb*

```
default['java']['jdk_version'] = '7'

default['databases'] = ['db_notes', 'db_notes_test']
```

*recipes/default.rb*

```
...
node['databases'].each do |database_name|
  mysql_database database_name do
    connection(
      :host     => '127.0.0.1',
      :username => 'root',
      :password => 'root'
    )
    action :create
  end
end
...
```

Now we can add a new `test` machine in our root Vagrantfile

*Vagrantfile*

```
...
  config.vm.define "test" do |test|
    test.vm.hostname = "test"
    test.vm.network :private_network, ip: '192.168.33.102'

    test.vm.provider 'virtualbox' do |vb|
      vb.memory = 1024
    end

    test.vm.provision :chef_zero, install: true do |chef|
      chef.verbose_logging
      chef.nodes_path = 'cookbooks'
      chef.file_cache_path = '/var/chef/cache'
      chef.add_recipe 'sample-app::default'
      chef.json = {
        "databases": ["db_notes"]
      }
    end
  end
...
```
We are telling the cookbook to use the new attribute `databases` with only a database. Cool!

run `vagrant up test` in order to boot the test machine.

## We'd like to add a reverse proxy in production

*recipes/nginx.rb*

```
package 'nginx'

cookbook_file '/etc/nginx/sites-available/default' do
  source 'nginx-default-site'
  notifies :restart, 'service[nginx]', :delayed
end

service 'nginx'
```

*files/nginx-default-site*

```
upstream backend {
    server localhost:8080;
}

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;

        server_name localhost;

        location / {
                proxy_pass http://backend;
        }
}
```

Now we can add the recipe in the root Vagrantfile

*Vagrantfile*

```
...
    chef.add_recipe 'sample-app::default'
    chef.add_recipe 'sample-app::nginx'
...
```

run `vagrant provision test`

# Automates the application deployment

In order to demostrate how is possible to automates an application deployment we are going to build from scratch a deployment workflow for our application. To do this we use [`fabric`](http://docs.fabfile.org/en/1.10/)

**requirements**

* Python 2.7
* virtualenv

install fabric by `pip install -r requirements.txt`

then create our first and very simple deploy workflow:

*fabfile.py*

```
from fabric.api import *

env.warn_only = True

def deploy():
    stop()
    copy_artefact()
    copy_configuration()
    migrate()
    start()

def copy_artefact():
    put("target/sample-app-1.0-SNAPSHOT.jar", "/home/vagrant/")

def copy_configuration():
    put("configuration.yml", "/home/vagrant/")

def start():
    run("screen -S sample-app -d -m java -jar /home/vagrant/sample-app-1.0-SNAPSHOT.jar server configuration.yml", pty=False)

def stop():
    run("screen -S sample-app -X quit", pty=False)

def migrate():
    run("java -jar /home/vagrant/sample-app-1.0-SNAPSHOT.jar db migrate configuration.yml")
```

Let's try to deploy the application on the `test` machine

`fab -u vagrant -H 192.168.33.102 deploy`

# Deployment automation in Jenkins

We want to extend our basic pipeline with a specific task for the deploy. There are some changes we have to introduce in the cookbook `sample-app`:

```
cd cookbooks/sample-app
```

## An home and a user used for application deployment

*recipes/application_deployment.rb*

```
user 'deployer' do
  shell '/bin/bash'
  home '/home/deployer'
  manage_home true
  action :create
end

directory '/home/deployer/.ssh' do
  owner 'deployer'
  group 'deployer'
  mode '0700'
end

remote_file '/home/deployer/.ssh/authorized_keys' do
  source 'https://gist.githubusercontent.com/joebew42/cfb85d25199b94461c27/raw/ebf41312424286b302d4b7b8f645931d12e0c4b8/deployer.pub'
  owner 'deployer'
  group 'deployer'
  mode '0600'
  action :create
end
```

Update the root Vagrantfile in order to execute this recipe:

*Vagrantfile*

```
...
    test.vm.provision :chef_zero, install: true do |chef|
      chef.verbose_logging
      chef.nodes_path = 'cookbooks'
      chef.file_cache_path = '/var/chef/cache'
      chef.add_recipe 'sample-app::default'
      chef.add_recipe 'sample-app::nginx'
      chef.add_recipe 'sample-app::application_deployment'
      chef.json = {
        "databases": ["db_notes"]
      }
    end
...
```
Run the provision of the test machine: `vagrant provision test`

## Authorize Jenkins to perform deploy on Test machine

In order to authorize jenkins to perform deploy on test machine we have to change the default recipe:

* Install the virtualenv
* Install the *deployer* private key
* Add a jenkins plugin to run python code in a virtualenv

`cd cookbooks/ci`

*recipes/default.rb*

```
include_recipe 'java'
include_recipe 'maven'
include_recipe 'jenkins::master'

jenkins_plugin 'git'
jenkins_plugin 'greenballs'
jenkins_plugin 'junit'
jenkins_plugin 'jobConfigHistory'
jenkins_plugin 'delivery-pipeline-plugin'
jenkins_plugin 'shiningpanda' do
  notifies :restart, 'service[jenkins]', :delayed
end

package 'git'
package 'python-virtualenv'
package 'python-dev'

mysql_service 'test' do
  port '3306'
  version '5.5'
  initial_root_password 'root'
  action [:create, :start]
end

mysql2_chef_gem 'default' do
  action [:install]
end

mysql_database 'db_notes_test' do
  connection(
    :host     => '127.0.0.1',
    :username => 'root',
    :password => 'root'
  )
  action :create
end

directory "#{node['jenkins']['master']['home']}/.ssh" do
  owner node['jenkins']['master']['user']
  group node['jenkins']['master']['group']
  mode '0700'
end

remote_file "#{node['jenkins']['master']['home']}/.ssh/deployer" do
  source 'https://gist.githubusercontent.com/joebew42/440c14b70ee305af31f6/raw/2ccd359966d523a026123a434dba262ca9a90e79/deployer'
  owner node['jenkins']['master']['user']
  group node['jenkins']['master']['group']
  mode '0600'
end
```

Run the provision of the `ci` machine: `vagrant provision ci`

Now we can create the job on jenkins to automates the deploy on test machine. We'll adds a simple acceptance test with a `curl` command.

## Deploy in production ? Simple now !

We have already create the `test` machine that is a production-like machine. We have only to add a new machine in the root Vagrantfile:

*Vagrantfile*

```
...
  ['test', 'production'].each_with_index do |environment, index|
    config.vm.define "#{environment}" do |machine|
      machine.vm.hostname = "#{environment}"
      machine.vm.network :private_network, ip: "192.168.33.#{102 + index}"

      machine.vm.provider 'virtualbox' do |vb|
        vb.memory = 1024
      end

      machine.vm.provision :chef_zero, install: true do |chef|
        chef.verbose_logging
        chef.nodes_path = 'cookbooks'
        chef.file_cache_path = '/var/chef/cache'
        chef.add_recipe 'sample-app::default'
        chef.add_recipe 'sample-app::nginx'
        chef.add_recipe 'sample-app::application_deployment'
        chef.json = {
          "databases": ["db_notes"]
        }
      end
    end
  end
...
```

Run `vagrant up production`

# Logging

We are going to provision an ELK stack (elasticsearch, logstash and kibana)

For practicality (time!) reason a simple logging cookbook can be found [**here**](https://github.com/joebew42/dropwizard-sample-app/tree/devops-workshop/cookbooks)

The complete guide for the logging cookbook can be found [**here**](https://github.com/xpeppers/devops-jumpstart/wiki/4.-Centralized-logging)

## Add a new machine for logging purpose

Adds the cookbook `logging` as berks dependency

*Berksfile*

```
source 'https://supermarket.chef.io'

cookbook 'sample-app', path: './cookbooks/sample-app'
cookbook 'ci',         path: './cookbooks/ci'
cookbook 'logging',    path: './cookbooks/logging'
```

Run `berks update`

*Vagrantfile*

```
...
  config.vm.define "management" do |management|
    management.vm.hostname = "management"
    management.vm.network :private_network, ip: '192.168.33.110'

    management.vm.provider 'virtualbox' do |vb|
      vb.memory = 1024
    end

    management.vm.provision :chef_zero, install: true do |chef|
      chef.verbose_logging
      chef.nodes_path = 'cookbooks'
      chef.file_cache_path = '/var/chef/cache'
      chef.add_recipe 'logging::default'
      chef.json = {}
    end
  end
...
```

Run `vagrant up management`

Visits `http://192.168.33.110:5601` for Kibana Dashboard

**Exercise**

Take a look at *unit* and *integration* tests!

Can you add the same "code coverage" for the **ci** cookbook ?

### Send logs from test/production environment

root Vagrantfile

```
...
  ['test', 'production'].each_with_index do |environment, index|
    config.vm.define "#{environment}" do |machine|
      machine.vm.hostname = "#{environment}"
      machine.vm.network :private_network, ip: "192.168.33.#{102 + index}"

      machine.vm.provider 'virtualbox' do |vb|
        vb.memory = 1024
      end

      machine.vm.provision :chef_zero, install: true do |chef|
        chef.verbose_logging
        chef.nodes_path = 'cookbooks'
        chef.file_cache_path = '/var/chef/cache'
        chef.add_recipe 'sample-app::default'
        chef.add_recipe 'sample-app::nginx'
        chef.add_recipe 'sample-app::application_deployment'
        chef.add_recipe 'logging::client'
        chef.json = {
          "databases": ["db_notes"],
          "logging": {
              "host": "192.168.33.110"
          }
        }
      end
    end
  end
...
```

Run `vagrant provision test`

# Monitoring

You'll find the *monitoring* cookbook [**here**](https://github.com/joebew42/dropwizard-sample-app/tree/devops-workshop/cookbooks)

**Exercise**

Try to integrate the cookbook in your infrastructure. See the previous section.

# Cloud infrastructure

## Create AWS AMI with Packer

Go back to project root and create `packer` directory

```
mkdir packer
```

Define packer template in `packer/sample-app.json`

```
{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": "",
    "name": "default",
    "region": "eu-central-1",
    "source_ami": "ami-7e9b7c11",
    "vpc_id": "",
    "subnet_id": ""
  },

  "builders": [
    {
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "type": "amazon-ebs",
      "region": "{{user `region`}}",
      "source_ami": "{{user `source_ami`}}",
      "ami_virtualization_type": "hvm",
      "vpc_id": "{{user `vpc_id`}}",
      "subnet_id": "{{user `subnet_id`}}",
      "instance_type": "t2.small",
      "ssh_username": "ubuntu",
      "ami_name": "{{user `name`}}-sample-app-{{isotime \"20060102-150405\"}}",
      "tags":  {
        "Name": "{{user `name`}}-sample-app-{{isotime \"20060102-150405\"}}"
      }
    }
  ],

  "provisioners": [
    {
      "type": "chef-solo",
      "cookbook_paths": ["berks-cookbooks"],
      "run_list": [
        "sample-app::default",
        "sample-app::nginx",
        "sample-app::application_deployment"
      ]
    }
  ]
}
```

Create a Packer a `packer/private.json` file to customize Packer variables:

```
{
  "aws_access_key": "{your access key}",
  "aws_secret_key": "{your secret key}",
  "name": "{your name}"
}
```

Vendorize all needed cookbooks making them availabe to Packer:

`berks vendor`

Run Packer passing your custom configuration file and the template as arguments:

`packer build -var-file=packer/private.json packer/sample-app.json`

Take note of generated **AMI id**.

## Create AWS stack with Terraform

Create terraform directory:

```
mkdir terraform
cd terraform
```

Define terraform template in `terraform/sample-app.tf`

```
provider "aws" {
    access_key = "${var.access_key}"
    secret_key = "${var.secret_key}"
    region = "eu-central-1"
}

resource "aws_security_group" "sample-app" {
    name = "${var.name}-devops-jumpstart-sample-app"
    description = "Security group for web that allows web traffic from internet"

    ingress {
        from_port = 80
        to_port   = 80
        protocol  = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }
    ingress {
        from_port = 22
        to_port   = 22
        protocol  = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }
    egress {
        from_port = 0
        to_port = 0
        protocol = "-1"
        cidr_blocks = ["0.0.0.0/0"]
    }
}

resource "aws_instance" "sample-app" {
    instance_type = "t2.small"
    ami = "${var.ami}"
    key_name = "devops-jumpstart"
    security_groups = ["${aws_security_group.sample-app.name}"]

    tags {
      Name = "${var.name} devops-jumpstart sample-app"
    }
}

output "ip" {
    value = "${aws_instance.sample-app.public_ip}"
}
```

Define terraform variables in `terraform/variables.tf`

```
variable "access_key" {}
variable "secret_key" {}
variable "name" {
    default = "user"
}
variable "ami" {}
```

Define terraform variables values in `terraform/terraform.tfvars`

```
access_key = "{your access key}"
secret_key = "{your secret key}"
ami = "{packer generated AMI id}"
name = "{your name}"
```

Check terraform build plan

```
terraform plan
```

Create stack

```
terraform apply
```

Show stack state

```
terraform show
```

Take note of generated instance IP address.
Visit instance IP address in a browser.

Get AWS instance details using aws client

```
aws configure
aws ec2 describe-instances --filters "Name=tag:Name,Values={user} devops-jumpstart blog"
```