# Installing Prometheus on Ubuntu 16.04

This gist is a compilation of two tutorials. You can find the original tutorials [here](https://www.digitalocean.com/community/tutorials/how-to-install-prometheus-on-ubuntu-16-04) and [here](https://rahulwa.com/post/monitoring-using-prometheus/). What should you know before using this? Everything can be executed from the home folder. For easier cleanup at the end you can make directory where you'll download everything, and then just use `rm -rf .`. Although, you should be careful. If some strange bugs arise unexpectedly somewhere sometimes, just keep in mind that some user names have underscores in them (this is probably nothing to worry about).

# Create Users

```bash
sudo adduser --no-create-home --disabled-login --shell /bin/false --gecos "Prometheus Monitoring User" prometheus
sudo adduser --no-create-home --disabled-login --shell /bin/false --gecos "Node Exporter User" node_exporter
sudo adduser --no-create-home --disabled-login --shell /bin/false --gecos "Alertmanager User" alertmanager
sudo adduser --no-create-home --disabled-login --shell /bin/false --gecos "Blackbox Exporter User" blackbox_exporter
```

# Create Directories

```bash
sudo mkdir /etc/prometheus
sudo mkdir /etc/alertmanager
sudo mkdir /etc/alertmanager/data
sudo mkdir /etc/alertmanager/template
sudo mkdir /etc/blackbox
sudo mkdir /var/lib/prometheus
```

# Create Dummy Config Files

```bash
sudo touch /etc/prometheus/prometheus.yml
sudo touch /etc/alertmanager/alertmanager.yml
sudo touch /etc/blackbox/blackbox.yml
```

# Assign Ownership

```bash
sudo chown -R prometheus:prometheus /etc/prometheus
sudo chown -R alertmanager:alertmanager /etc/alertmanager
sudo chown -R blackbox_exporter:blackbox_exporter /etc/blackbox
sudo chown prometheus:prometheus /var/lib/prometheus
```

# Download Binaries

```bash
wget https://github.com/prometheus/prometheus/releases/download/v2.0.0/prometheus-2.0.0.linux-amd64.tar.gz
wget https://github.com/prometheus/alertmanager/releases/download/v0.12.0/alertmanager-0.12.0.linux-amd64.tar.gz
wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.11.0/blackbox_exporter-0.11.0.linux-amd64.tar.gz
wget https://github.com/prometheus/node_exporter/releases/download/v0.15.2/node_exporter-0.15.2.linux-amd64.tar.gz
wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_4.6.3_amd64.deb
```

# Untar

```bash
tar xvzf prometheus-2.0.0.linux-amd64.tar.gz
tar xvzf alertmanager-0.12.0.linux-amd64.tar.gz
tar xvzf blackbox_exporter-0.11.0.linux-amd64.tar.gz
tar xvzf node_exporter-0.15.2.linux-amd64.tar.gz
```

# Copy Binaries to /usr/local/bin (and for main app copy libs...)

```bash
sudo cp prometheus-2.0.0.linux-amd64/prometheus /usr/local/bin/
sudo cp prometheus-2.0.0.linux-amd64/promtool /usr/local/bin/
sudo cp -r prometheus-2.0.0.linux-amd64/consoles /etc/prometheus
sudo cp -r prometheus-2.0.0.linux-amd64/console_libraries /etc/prometheus
sudo cp alertmanager-0.12.0.linux-amd64/alertmanager /usr/local/bin/
sudo cp alertmanager-0.12.0.linux-amd64/amtool /usr/local/bin/
sudo cp blackbox_exporter-0.11.0.linux-amd64/blackbox_exporter /usr/local/bin/
sudo cp node_exporter-0.15.2.linux-amd64/node_exporter /usr/local/bin/
```

# Assign Ownership Again

```bash
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
sudo chown alertmanager:alertmanager /usr/local/bin/alertmanager
sudo chown alertmanager:alertmanager /usr/local/bin/amtool
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
sudo chown blackbox_exporter:blackbox_exporter /usr/local/bin/blackbox_exporter
```

# Sample Files

## Prometheus

```bash
sudo nano /etc/prometheus/prometheus.yml
```

```yaml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'node_exporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9100']
```

## Blackbox

```bash
sudo nano /etc/blackbox/blackbox.yml
```

```yaml
modules:
  http_2xx_example:
    prober: http
    timeout: 5s
    http:
      valid_http_versions: ["HTTP/1.1", "HTTP/2"]
      valid_status_codes: []  # Defaults to 2xx
      method: GET
```

## Alertmanager

```bash
sudo nano /etc/alertmanager/alertmanager.yml
```

```yaml
global:
  smtp_smarthost: 'localhost:25'
  smtp_from: 'alertmanager@example.org'
  smtp_auth_username: 'alertmanager'
  smtp_auth_password: 'password'

# templates:
# - '/etc/alertmanager/template/*.tmpl'

route:
  repeat_interval: 3h 
  receiver: team-X-mails

receivers:
- name: 'team-X-mails'
  email_configs:
  - to: 'team-X+alerts@example.org'
```

# systemd Configuration

## Prometheus Unit

```bash
sudo nano /etc/systemd/system/prometheus.service
```

```
[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus
sudo systemctl status prometheus
```

## Node Exporter Unit

```bash
sudo nano /etc/systemd/system/node_exporter.service
```

```
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable node_exporter
sudo systemctl start node_exporter
sudo systemctl status node_exporter
```

## Blackbox Exporter Unit

```bash
sudo nano /etc/systemd/system/blackbox_exporter.service
```

```
[Unit]
Description=Prometheus blackbox exporter
After=network.target auditd.service

[Service]
User=blackbox_exporter
Group=blackbox_exporter
Type=simple
ExecStart=/usr/local/bin/blackbox_exporter --config.file=/etc/blackbox/blackbox.yml
Restart=on-failure

[Install]
WantedBy=default.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable blackbox_exporter
sudo systemctl start blackbox_exporter
sudo systemctl status blackbox_exporter
```

## Alertmanager Unit

```bash
sudo nano /etc/systemd/system/alertmanager.service
```

```
[Unit]
Description=Prometheus Alert Manager service
Wants=network-online.target
After=network.target

[Service]
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager --config.file /etc/alertmanager/alertmanager.yml --storage.path /etc/alertmanager/data
Restart=always

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable alertmanager
sudo systemctl start alertmanager
sudo systemctl status alertmanager
```

# Configuring Firewall

Think about this step. What is your current server firewall setup? What ports do you really need?

```bash
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
sudo ufw allow 9090
sudo ufw allow 9100
sudo ufw allow 9093
sudo ufw allow 9115
sudo ufw allow 3000

sudo ufw enable
```

# Installing Grafana

```shell
sudo apt-get install -y adduser libfontconfig
sudo dpkg -i grafana_4.6.3_amd64.deb
```

**or**

```shell
sudo apt-get update
sudo apt-get install -y gdebi
sudo gdebi grafana_4.6.3_amd64.deb
```

**and then!**

```bash
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server
sudo /bin/systemctl start grafana-server
```

# Cleanup

```bash
sudo rm prometheus-2.0.0.linux-amd64.tar.gz
sudo rm alertmanager-0.12.0.linux-amd64.tar.gz
sudo rm blackbox_exporter-0.11.0.linux-amd64.tar.gz
sudo rm node_exporter-0.15.2.linux-amd64.tar.gz
sudo rm grafana_4.6.3_amd64.deb

sudo rm -rf prometheus-2.0.0.linux-amd64
sudo rm -rf alertmanager-0.12.0.linux-amd64
sudo rm -rf blackbox_exporter-0.11.0.linux-amd64
sudo rm -rf node_exporter-0.15.2.linux-amd64
```

**or**

(do the following only if you're sure what you're doing! don't do stupid stuff)
`sudo rm -rf .`