Secure Prometheus with OIDC or LDAP

When hosting applications for internal use, it is common to only allow connections from private networks. However, in addition to using a VPN, being able to authenticate and authorize users with a centralized identity system is often required for security reasons. I found two good alternatives that I will describe next, along with a monitoring use case.

Prometheus and Grafana

Prometheus is one of the most widely used systems for monitoring and alerting, while Grafana is very popular to create observability dashboards like this one:

Although some applications like Grafana already have support for OAuth or LDAP authentication, others like Prometheus delegate this responsibility to other systems.

Pomerium

Pomerium is an identity-aware proxy developed in Go. It can be placed in front of internal services and integrate with identity providers using OpenID Connect. Out of the box, it supports identity providers like Google, Github and standard compliant OIDC providers like node-oidc-provider or IdentityServer. Here’s a comprehensive guide to setup a provider.

Authelia

Authelia is an authentication and authorization server written in Go. It requires integration with a reverse proxy, such as Nginx. Instead of OIDC, LDAP is supported so it can be integrated with Active Directory (AD). On my tests I’ve used a local users file and Nginx, as it was the simplest approach. I also had success with AD and HAProxy (with LUA plugins), following the official documentation.

Architecture

Both auth technologies work quite well. It boils down to what you need to integrate with. Some additional features can be added on the reverse proxy or load balancer, like rate limiting or modifying request headers.

I’ve setup a demo project to demonstrate their capabilities that uses the following architecture:

Besides proxy mode, which passes all traffic through, Pomerium also supports Forward authentication, which causes the reverse proxy to call an endpoint to verify if the user is authorized. User information is then placed on request headers. Both technologies make use of cookie sessions and redirect the user to the login page when not authenticated. They also support Redis, which is important to preserve sessions during restarts or if using multiple instances, as you don’t usually want to depend on sticky sessions at the load balancer level.

If you are interested in authenticating requests from public internet Github actions runners with internal services inside a private subnet, check out this OIDC proxy approach.