Skip to content

Instantly share code, notes, and snippets.

@Dev-Dipesh

Dev-Dipesh/ELK with Nginx.md

Last active June 6, 2025 19:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Dev-Dipesh/2ac30a8a01afb7f65b2192928a875aa1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Dev-Dipesh/2ac30a8a01afb7f65b2192928a875aa1 to your computer and use it in GitHub Desktop.
Download ZIP
Setting up Elasticsearch, Logstash and Kibana with Nginx.
Raw
ELK (E - v2.3.0) with Nginx.md

ELK

Our ELK stack setup has four main components: - Logstash: The server component of Logstash that processes incoming logs - Elasticsearch: Stores all of the logs - Kibana: Web interface for searching and visualizing logs, which will be proxied through Nginx - Filebeat: Installed on client servers that will send their logs to Logstash, Filebeat serves as a log shipping agent that utilizes the lumberjack networking protocol to communicate with Logstash

ELK+NGINX

These are the versions we are currently setting up in this installment, please make note accordingly if you have specific version requirements.

JDK Version - 8
Elasticsearch - 2.x
Logstash - 2.2
Kibana - 4.5

Installation Steps

For production go with three separate instances for each, elasticsearch, logstash and kibana. Currently we are setting it up on a single machine. You can opt for similar configuration on any cloud provider.

OS  - Ubuntu 14.04 LTS
RAM - 4Gb
CPU - 2

**1 - Install Java 8**
	- Add Oracle Java PPA to apt:
		`$ sudo add-apt-repository -y ppa:webupd8team/java`
	- Update your apt package database:
		`$ sudo apt-get update`
	- Install the latest version of Oracle Java 8
		`$ sudo apt-get -y install oracle-java8-installer`

**2 - Install Elasticsearch**
	- Import Elasticsearch public GPG key into apt
		`$ wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -`
	- Create the Elasticsearch source list
		`$ echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list`
	- Update your package database
		`$ sudo apt-get update`
	- Install Elasticsearch
		`$ sudo apt-get -y install elasticsearch`
	- Start Elasticsearch service
		`$ sudo service elasticsearch restart`
	- Test
		`$ curl localhost:9200`
	- If the output is similar to this, then you will know that Elasticsearch is running properly:
		```json
		{
			"status" : 200,
			"name" : "Jigsaw",
			"cluster_name" : "elasticsearch",
			"version" : {
				"number" : "1.7.1",
				"build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
				"build_timestamp" : "2015-07-29T09:54:16Z",
				"build_snapshot" : false,
				"lucene_version" : "4.10.4"
			},
			"tagline" : "You Know, for Search"
		}
		```
	- Enable Elasticsearch to start on boot
		`$ sudo update-rc.d elasticsearch defaults 95 10`

**3 - Logstash Installation**
@danpem
Copy link

danpem commented Oct 5, 2017

Cool works

@mvharii
Copy link

mvharii commented Mar 5, 2018

Thanks Sir it works. However in above log stash section : We have yet to configure Logstash, but let leave it for later? what else to be configured and ES/Kibana/LS .. where they have stored in my local box? am unable to find the folders.. (as am new still learning could you please help me)

@duypv98
Copy link

duypv98 commented Dec 17, 2018

Cool sir

@edsonharantes
Copy link

edsonharantes commented May 7, 2019

love the half life reference :D

@ismailyenigul
Copy link

ismailyenigul commented Jul 14, 2020

Hi @Dev-Dipesh
The following lines must be added to nginx conf to make basic auth work

   auth_basic “Kibana”;
   auth_basic_user_file  /etc/nginx/htpasswd.users;

Here is the complete conf

server {
  listen 80;
    server_name _;
    auth_basic “Kibana”;
   auth_basic_user_file  /etc/nginx/htpasswd.users;

  error_log   /var/log/nginx/kibana.error.log;
  access_log  /var/log/nginx/kibana.access.log;



  location / {
    rewrite ^/(.*) /$1 break;
    proxy_ignore_client_abort on;
    proxy_pass http://127.0.0.1:5601;
    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header  Host $http_host;
   
  }
}

@petsoukos
Copy link

petsoukos commented Jul 27, 2021

How about putting Logstash behind a proxy? It wont work using the same logic with Kibana. Something about being a TCP protocol or something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment