---
title: "Geo location with Filebeat on Elasticsearch 7, HAProxy and NGINX"
url: https://michalasobczak.pl/posts/geo-location-with-filebeat-on-elasticsearch-7-haproxy-and-nginx.html
author: "Michał Sobczak"
description: "Display geo location map for NGINX traffic logs in Kibana Summary There are 3 things to remember and configure in order to have geo location map working: Use “forwardfor” option on pfSense HAProxy TLS frontend Enable filebeat NGINX module and point particular log files Define custom NGINX log format…"
published: 2022-09-23T08:56:27.000Z
updated: 2022-09-23T15:10:10.000Z
language: en
categories: ["Technology"]
tags: ["ElasticSearch", "HAProxy", "Kibana", "NGINX", "pfSense"]
license: CC BY-NC-ND 4.0 (https://creativecommons.org/licenses/by-nc-nd/4.0/)
---
# Geo location with Filebeat on Elasticsearch 7, HAProxy and NGINX

Display geo location map for NGINX traffic logs in Kibana

## Table of Contents

-   [Summary](#summary)
-   [Elasticsearch 7](#elasticsearch-7)
-   [Kibana](#kibana)
-   [Configuration](#configuration)
-   [HAProxy on pfSense](#haproxy-on-pfsense)
-   [Filebeat on client host](#filebeat-on-client-host)
-   [NGINX](#nginx)
-   [Network geo location map](#network-geo-location-map)
-   [Summary](#summary-2)

## Summary

There are 3 things to remember and configure in order to have **geo location map working**:

-   Use “forwardfor” option on pfSense HAProxy TLS frontend
-   Enable filebeat NGINX module and point particular log files
-   Define custom NGINX log format

This guide relates to Ubuntu Linux setup.

## Elasticsearch 7

First **install Elasticsearch 7** as follows.

```bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install elasticsearch
```

> *Note: for more resilent setup install more than one Elasticsearch server node and enable basic security. For sake of clarity I will skip these two aspects which will be covered by another article.*

## Kibana

Then **install Kibana** to have UI for Elasticsearch:

```bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install kibana
```

## Configuration

Now be sure to **configure both Elastic** and Kibana. For Elastic it is /etc/elasticsearch/elasticsearch.yml configuration file. Be sure to set the following (change 0.0.0.0 with your local IP address):

```yaml
network.host:   0.0.0.0
http.port:      9200
discovery.type: single-node
```

To **enable and start Elasticsearch** server:

```bash
sudo systemctl enable elasticsearch
sudo service elasticsearch start
```

Now **configure Kibana**. Configuration file location is /etc/kibana/kibana.yml. Replace “0.0.0.0” with your local IP address:

```yaml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://0.0.0.0:9200"]
```

**Enable and start Kibana** server:

```bash
sudo systemctl enable kibana
sudo service kibana start
```

Be sure to **check if both Elasticsearch and Kibana are up and running** with service or systemctl. If everything is fine proceed to setup Filebeat, Packetbeat on client hosts. There is separate guide how to do this. In beats configuration file point to Elasticsearch server you just installed.

## HAProxy on pfSense

Example setup includes gateway, which is pfSense with HAProxy package. **Configure two frontends, one for HTTP at 80 and one for HTTPS at 443**. On HTTP frontend configure

```text
Action http-request redirect
rule: scheme https
```

On HTTPS **configure certificate and check SSL Offloading**. Of course you need to load this certificate in System, Cert. Manager. Configure your backends and select it on HTTPS frontend. Now, go to HTTPS frontend and check the following:

```text
Use "forwardfor" option
```

**It is required to be able to read client IP in backend NGINX**.

## Filebeat on client host

On client hosts **install filebeat package**. There is separate guide for this one. Edit configuration file, which is /etc/filebeat/filebeat.yml:

```yaml
filebeat.inputs:
  enabled: false
```

And then **setup and enable NGINX module**:

```bash
filebeat setup
filebeat modules enable nginx
```

Now, you are good to go with delivering log files, but first you need to **point them** in the configuration at /etc/filebeat/modules.d/nginx.yml:

```yaml
- module: nginx
  access:
    enabled: true
    var.paths: ["/var/log/nginx/app.log"]
```

You can now enable and start filebeat:

```bash
sudo systemctl enable filebeat
sudo service filebeat start
```

## NGINX

In the http section in configuration file /etc/nginx/nginx.conf add **new log format**:

```nginx
log_format mydefault '$http_x_forwarded_for - $remote_user [$time_local] '
                     '"$request" $status $body_bytes_sent '
                     '"$http_referer" "$http_user_agent"';
```

You can use it in application configuration file at /etc/nginx/conf.d/app.conf in the server stanza:

```text
access_log  /var/log/nginx/app.log mydefault;
```

Restart your NGINX server and go to Kibana to explore your data. You need to add this log format, in order to handle client IP which is present in $http\_x\_forwarded\_for variable. **This format as close as possible to the default one.**

## Network geo location map

If everything went fine which is installting Elasticsearch, Kibana, beats on your client hosts and configuration of HAProxy, NGINX, then you can open Security, Explore, Network section and hit refresh buton to load data into map. But first you need to **select Data sources** (link above the map, on the right side), include filebeat-\* index pattern.

![Fig. Filebeat data source selection](https://michalasobczak.pl/wp-content/uploads/2022/09/obraz-22.png)
*Fig. Filebeat data source selection*

With such configuration you should be able to see **geo points representing client locations**.

![Fig. Location points based on filebeat data](https://michalasobczak.pl/wp-content/uploads/2022/09/obraz-23.png)
*Fig. Location points based on filebeat data*

In case you also enable **Packetbeat**, you will see also **networking information data** below the map.

![Fig. Networking information data](https://michalasobczak.pl/wp-content/uploads/2022/09/obraz-24.png)
*Fig. Networking information data*

Please remember that **enabling Packetbeat generates tons of data** in complex environments, so be sure to allocate enough space for these indices.

## Summary

This guide covers the basic path for installting Elasticsearch, Kibana, Beats and configuring HAProxy and NGINX to deliver traffic, logs to Elasticsearch and be able to visualize traffic as geo location points.