# Adding Rack-Based Health Checks Without curl or wget
![Lake View Cemetary, Cleveland OH - Wendy Bayer](https://lh3.googleusercontent.com/pw/AMWts8Ao2WC0DCyYmbmG9hOfVYibLanikamZevYDNalvpG-9jtzDUXCeIP86-8jCB_NdA5ozpiB9CQKYa9PClRIs0Lm214YN13tFrbEHloXsGKD0RG55S92KM6AEJVCntGAA6LQZsfs5MHoba88eitj4V6YOrw=w892-h669-s-no)

> Image: Lake View Cemetary, Cleveland OH by Wendy Bayer

---

You can not address a problem without first knowing that there is a
problem.  Having *health checks* in your application allows you to
easily determine and monitor the status of your critical application
or service and in many cases have it automatically *self heal*.

Kubernetes, docker compose, and many other container orchestration
services rely on health checks to know when application services are
alive and ready to serve traffic.

Additionally you can connect your website monitoring services to ensure
that you know when your end users are not able to connect to your
web-based services.

Here you will learn how to create, use, and test both a liveness and
readiness health check in your Ruby Rack-based application without the
need for `curl` or `wget`.  While this post uses Rails and docker
compose, this approach can translate to other Rack-based (Ruby)
frameworks and orchestration services such as Kubernetes.

## Overview of Health Checks
In general (and
[per Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/))
there are three basic types of health checks (or probes):

* ***Liveness*** check which simply determines that the application is
  running (i.e. *alive*)

* ***Readiness*** check which determines that the application is *ready*
  and capable of serving traffic/end users which often means that all of
  the application's dependencies are up and successfully connected and
  initialized (e.g. database, redis)

* ***Startup*** check which determines that a long initializing
  application has successfully started

This post will focus on the two most common ones which are liveness and
readiness as many applications can use their readiness check as a startup
check.

### /livez and /readyz
There is a convention of naming the health check endpoints `/livez` and
`/readyz` which came from Google and its practices of standard "z-pages".
The 'z' is added to avoid any existing name clashes.  When engineers
left Google, they took these conventions with them so it has somewhat
spread throughout the IT industry.

> :eyes: I learned this etymological trivia from
> [Stack Overflow](https://stackoverflow.com/questions/43380939/where-does-the-convention-of-using-healthz-for-application-health-checks-come-f)

### Liveness and Readiness in Rails
Although this is somewhat specific to Rails, this section should also
help to illustrate the practical difference between liveness and readiness.

#### Liveness
Generally for a Rails application, liveness should **not** depend on the
state of dependent systems such as the application's database, redis,
external APIs, etc.  This is true even though a Rails application will
not return non-error responses if it can not connect to the database or
if there are pending migrations.  Consider that in Kubernetes or docker
compose/Swarm orchestration, the general mechanism of "self-healing"
an application/service is to kill and restart the container.  If the
issue is with the dependency like the database, repeatedly killing and
restarting the Rails application container is not going to help and just
adds to the churn.  Thus a liveness endpoint in Rails should simply
return 200 (`OK`) if the application (more specifically the application
server e.g. Puma) is running.

#### Readiness
Since readiness means that the application can provide its primary
function i.e. serve traffic, it should depend on the state of its
critical dependencies.  Thus a Rails readiness endpoint often checks
the database connection, redis connection, pending migrations, etc.
and only returns 200 (`OK`) if all of those dependencies are available.

## Implementing the Health Checks in Rack
At the time of this post (Rails 7.0.4.3), Rails is actually including a
basic liveness health check `/up` in the next version (7.1?).  Being
a liveness check, this does not include the state of the database or
other dependencies.  Here is the
[Pull Request](https://github.com/rails/rails/pull/46972).

While this seems helpful, it is probably best if the health checks,
especially liveness, are at a lower layer of the Rails middleware
and in Rack itself.  This excellent
[post](https://nick.malcolm.net.nz/blog/160831-rails-healthcheck)
by Nick Malcolm does a great job in explaining this.

### Implementing the Liveness Check
Here is an example of a Rack-based liveness check.  For Rails,
you can put this in file `lib/rack/live_check.rb`...
```ruby
# frozen_string_literal: true

module Rack
  # Most basic health check that simply indicates that the application is up
  # but not necessarily ready for traffic
  class LiveCheck
    def call(_env)
      # Must be alive to reach this
      [
        200,
        { 'Content-type' => 'application/json; charset=utf-8' },
        ['{ "status": 200, "message": "alive" }']
      ]
    end
  end
end
```

This example returns a basic informational JSON body, but you
can customize it to suit your needs.

### Configuring the Liveness Check at `/livez`
You will need to configure the liveness check in Rack, mapping it to
the endpoint `get /livez`.

In Rails, you can add the following lines to the Rack configuration file
`config.ru`...
```ruby
require_relative 'lib/rack/live_check'
...

map '/livez' do
  run Rack::LiveCheck.new
end
```

### Implementing the Readiness Check
Here is an example of a Rack-based readiness check.  For Rails,
you can put this in file `lib/rack/ready_check.rb`...
```ruby
# frozen_string_literal: true

module Rack
  # Health check to ensure the database is ready for traffic
  class ReadyCheck
    def initialize
      @body_details = {}
    end

    def call(_env)
      response = ready? ? ready_response : error_response
      response.finish
    end

    def ready?
      database_connected?
      database_migrations?
    end

    def database_connected?
      ActiveRecord::Base.connection.execute('SELECT 1')
      @body_details[:database_connection] = 'ok'
    rescue StandardError
      @body_details[:database_connection] = 'error'
      false
    end

    def database_migrations?
      ActiveRecord::Migration.check_pending!
      @body_details[:database_migrations] = 'ok'
    rescue StandardError
      @body_details[:database_migrations] = 'pending'
      false
    end

    def ready_response
      status = 200
      response_body = { status:, message: 'ready' }.merge(@body_details)
      rack_response(response_body, status)
    end

    def error_response
      status = 503
      response_body = { status:, message: 'error' }.merge(@body_details)
      rack_response(response_body, status)
    end

    def rack_response(body, status)
      Rack::Response.new(
        body.to_json,
        status,
        response_header
      )
    end

    def response_header
      { 'Content-type' => 'application/json; charset=utf-8' }
    end
  end
end
```

This example checks the database connection and if there are pending
migrations, but you can modify it to suit your needs, for instance
by adding checks for redis and/or other dependencies or removing the
pending migrations check.  It also returns an informational JSON
body which can be modified.

> :eyes: The documentation of Rails health check gems like the
> [shlima health-bit](https://github.com/shlima/health_bit)
> gem are a good source for how to check the status of common
> Rails dependencies

### Configuring the Liveness Check at `/readyz`
Again, you will need to configure the readiness check in Rack, mapping it to
the endpoint `get /readyz`.

In Rails, you can add the following lines to the Rack configuration file
`config.ru`...
```ruby
require_relative 'lib/rack/ready_check'
...

map '/readyz' do
  run Rack::ReadyCheck.new
end
```

### Running Your Application Server
Now if you run your application server with the added health checks
(e.g. `bundle exec bin/rails server -p 3000 -b 0.0.0.0`) and go to the
`/livez` endpoint (e.g. http://localhost:3000/livez) using your browser,
Postman, or `curl`, you should get a 200 response code and a body
that looks something like this...
```json
{ "status": 200, "message": "alive" }
```

Similarly for the `/readyz` endpoint (e.g. http://localhost:3000/readyz),
you should get the 200 response code and a body that looks something like
this...
```json
{"status":200,"message":"ready","database_connection":"ok","database_migrations":"ok"}
```

### CORS Considerations
If you are enabling Cross Origin Resource Sharing (CORS) in your Rails
or other Rack-based application, be sure to configure CORS to be positioned
above your health checks in the Rack middleware.  For example in Rails,
in the `config/initializers/cors.rb` initializer...
```ruby
Rails.application.config.middleware.insert_before 0, Rack::Cors do
  allow do
     ...
```

Note the `.middleware.insert_before 0` which configures CORS at the
top (`0`) position.

> :eyes: For more information, see
>[Positioning in the Middleware Stack](https://github.com/cyu/rack-cors#positioning-in-the-middleware-stack)
> in the [rack-cors](https://github.com/cyu/rack-cors)
> documentation

### Giving Credit Where Credit Is Due
The implementations for these Rack-based health checks are inspired
and derived from the post
[Simple health-check for Ruby on Rails](https://kinduff.com/2021/10/15/simple-health-check-for-ruby-on-rails/)
by Alejandro Abarca Rodríguez.

## Writing a Simple Ruby Program To Check the Readiness
In a container orchestration system such as Kubernetes or docker compose,
you need a way to know if your application is up and ready.  For instance,
in my Continuous Integration I use a docker compose framework to run E2E
tests in a container against my application container and need to know
that the application is ready before running the tests.

Most of the docker compose health check examples use `curl` to check the
readiness endpoint, but you may not have `curl` available in your
application's deployment/production image for security and/or image
size reasons.  Since you are running a Rack-based web application
(e.g. Rails), you necessarily have Ruby available, so why not use that?

> :thinking: You can always convert your `curl` commands to Ruby using
> this awesome site https://jhawthorn.github.io/curl-to-ruby/

Here is a simple ruby program that checks your "z" endpoints (with `/readyz`
the default) and returns a `0` return code (Unix standard for success) if
the checked endpoint returns a 200 return code...
```ruby
#!/usr/bin/env ruby
# ----------------------------------------------------------------------
# Simple script to query healthchecks of locally running Ruby
#  applications. Generally intended as a healthchecks for
# container orchestration so curl/wget are not needed
# ----------------------------------------------------------------------
require 'net/http'
require 'uri'

def z_endpoint(endpoint)
  puts "[#{endpoint}]"
  host = ENV.fetch('HOST', 'localhost')
  port = ENV.fetch('PORT', 3000)
  "http://#{host}:#{port}/#{endpoint}z"
end

endpoint = ARGV.shift || 'ready'
healthcheck = URI.parse(z_endpoint(endpoint))

begin
  response = Net::HTTP.get_response(healthcheck)
rescue StandardError
  exit(1)
end

puts "Healthcheck [#{healthcheck}] returned code: [#{response.code}] body: [#{response.body}]"
exit_code = (response.code == '200') ? 0 : 1
exit(exit_code)
```

You can put this program in file `app_is` in you application's
root directory.  This name is influenced by the PostgreSQL `pg_isready`
health check.
**Be sure to set execute permissions e.g. `chmod a+x app_is` on this file.**

Note that when running these health checks in a container orchestration
system such as Kubernetes or docker compose, the checks run in the
application container itself so the host is`localhost`.

> :bulb: You can use this same approach for your liveness health check as well

## Configuring the Readiness Health Check in Docker Compose
Although this example is for docker compose, it is translatable to other
container orchestration system such as Kubernetes.

Here is an example of using the Ruby `app_is` program in a
`docker-compose.yml` file for the application...
```
services:
  app:
    healthcheck:
      test: ["CMD", "bash", "-c", "ruby", "app_is"]
      start_period: 10s
      interval: 10s
      timeout: 5s
      retries: 10
```

:point_right: Note the `test:` line, this is very important.  This is
the only way that I could get this to work.  This executes the `bash`
command with the `-c` option to run the `ruby` command with the
`app_is` program/script.  Running just the `app_is`
program even with the "shebang" line at the top did not work.

You should adjust your timing configurations (e.g. `start_period`,
`interval`, etc.) to suit your application's startup characteristics.

## Testing the Health Check Endpoints
Now that you have implemented your health checks, you should ensure that
they continue to work as you make changes to your application by adding
automated tests.

Because the health checks are implemented in the Rack middleware and not
in the Rails applications, you will not be able to test them with
controller or request/integration specs/tests.  Since they are simple
API endpoints, they are easy enough to test in your End-to-End (E2E)
tests.

Here is an example using the
[Faraday](https://lostisland.github.io/faraday/) gem with
[RSpec](https://rspec.info/) and a custom RSpec matcher
for the health check response.

### Implementing the Tests
In file `spec/health_checks_spec.rb`...
```ruby
# frozen_string_literal: true

require 'spec_helper'

RSpec.describe 'Health Checks' do

  describe '/livez' do
    subject(:livez_request) { app_connection.get(endpoint_url('/livez')) }

    it { expect(livez_request).to be_request_response(200, livez_response) }
  end

  describe '/readyz' do
    subject(:readyz_request) { app_connection.get(endpoint_url('/readyz')) }

    it { expect(readyz_request).to be_request_response(200, readyz_response) }
  end

  private

  def livez_response
    {
      'status' => 200,
      'message' => 'alive'
    }
  end

    def readyz_response
    {
      'status' => 200,
      'message' => 'ready',
      'database_connection' => 'ok',
      'database_migrations' => 'ok'
    }
  end

  def app_connection
    Faraday.new(url: app_base_url) do |conn|
      conn.request :json
      conn.response :json
    end
  end

  def endpoint_url(path)
    "#{app_base_url}#{path}"
  end

  def app_base_url
    ENV.fetch('E2E_BASE_URL')
  end
end
```

### Implementing the Custom Response Matcher
Although it is not necessary, implementing a custom RSpec matcher
allows you to test and see all of the response at once which
is helpful for debugging when the tests fail.

In file `spec/support/matchers/be_request_response.rb`...
```ruby
# frozen_string_literal: true

# Custom RSpec Matcher to match an
# expected request response
module BeRequestResponse
  class BeRequestResponse
    def initialize(status, body)
      @status = status
      @body = body
    end

    def matches?(actual)
      @actual_status = actual.status
      @actual_body = actual.body

      @actual_status == @status &&
        @actual_body == @body
    end

    def failure_message
      "expected that actual status code [#{@actual_status}] would match " \
        "expected status code [#{@status}] and that actual body " \
        "[#{pretty(@actual_body)}] would match expected body " \
        "[#{pretty(@body)}]"
    end

    def failure_message_when_negated
      "expected that actual status code [#{@actual_status}] would not match " \
        "expected status code [#{@status}] and that actual body " \
        "[#{pretty(@actual_body)}] would match expected body " \
        "[#{pretty(@body)}]"
    end

    def description
      "have response status code [#{@status}] and body [#{@body}]"
    end

    private

    def pretty(json)
      JSON.pretty_generate(json)
    end
  end

  def be_request_response(status, body)
    BeRequestResponse.new(status, body)
  end
end

# Include the custom matcher in RSpec
RSpec.configure do |config|
  config.include BeRequestResponse
end
```

### Adding Faraday and Your Custom Matcher to `spec_helper.rb`
Finally, add Faraday and your custom RSpec matcher to
`spec/spec_helper.rb`...
```ruby
...
# --- CUSTOM ADDITIONAL CONFIGURATION ---
require 'faraday'
require_relative 'support/matchers/be_request_response'
```

### Running the Tests Against the Server With Healthchecks
Now if you run your application server with the health checks
(e.g. `bundle exec bin/rails server -p 3000 -b 0.0.0.0`)
and then run your E2E tests supplying the `E2E_BASE_URL`
environment variable set to your running server (e.g.
`E2E_BASE_URL=http://localhost:3000 bundle exec rspec --format documentation`),
your output should look something like the following
```
Health Checks
  /livez
    is expected to have response status code [200] and body [{"status"=>200, "message"=>"alive"}]
  /readyz
    is expected to have response status code [200] and body [{"status"=>200, "message"=>"ready", "database_connection"=>"ok", "database_migrations"=>"ok"}]

Finished in 0.08437 seconds (files took 0.43692 seconds to load)
2 examples, 0 failures
```

## Conclusion
And that's it.  Now you have added liveness and readiness health checks
to your application and a custom Ruby program to test if your app
is ready along with the automated E2E tests to ensure your health checks
are working.

---