Zero-Port Exposure: Routing On-Prem Traffic Through a Cloud VM Without Opening a Single Firewall Hole
A practical guide for small startups running Kubernetes on-premises who want public HTTPS endpoints — without exposing cluster IPs, paying for enterprise load balancers, or sacrificing security.
Your cluster is beautiful. But the internet can’t see it.
You’ve done the hard work. You’ve got a Kubernetes cluster humming on bare metal or on-premises hardware. Your microservices are wired up. ArgoCD is syncing. The team is shipping features. It’s working — inside your office, inside your VPN, inside a bubble.
Then someone asks the obvious question: how do users actually get to this?
For a well-funded enterprise, the answer is simple: cloud load balancers, dedicated IPs, a BGP setup, maybe some SD-WAN. Cost? $1,000–$5,000/month before you’ve served a single request.
For a startup with a $20/month DigitalOcean budget? You need to think differently.
The trick is to never let the internet knock on your cluster’s door — instead, have your cluster knock on the internet’s.
This technique is called a reverse proxy tunnel, and it’s the same principle behind tools like ngrok, Cloudflare Tunnel, and Tailscale — except you control every component, it costs almost nothing, and it works with real production Kubernetes.
The three-layer stack
The architecture has three collaborating layers:
[ Cloudflare / DNS ]
│ HTTPS (public)
▼
[ Cloud VM — $6/mo ]
├── Nginx Proxy Manager (ports 80/443, terminates TLS)
└── frps server (port 7000, receives tunnel registrations)
│ outbound TCP tunnels (clusters dial OUT — no inbound ports)
▼
[ On-Prem Kubernetes Clusters ]
├── frpc client → ingress-nginx (staging, port 8080)
├── frpc client → ingress-nginx (prod, port 9080)
└── frpc client → Kong Gateway (prod, port 9180)
The key insight is the direction of the arrows. Your on-premises clusters never accept inbound connections. Instead, the frpc clients inside each cluster make outbound connections to the frps server on your cloud VM — the same way a person calls a switchboard rather than having the switchboard call back every person's private number.
Your cluster’s real IP is never exposed to the internet, ever.
FRP: The secret ingredient most people don’t know about
Fast Reverse Proxy (FRP) is an open-source tool that does one thing exceptionally well: it creates persistent, authenticated tunnels between a server (the cloud VM) and one or more clients (your on-premises machines). It’s the engine that makes this whole architecture possible.
Here’s the request lifecycle:
1. frpc dials out
Your Kubernetes cluster runs an frpc pod (or Deployment). On startup, it makes an outbound TCP connection to frps on the cloud VM — port 7000. Your cluster firewall only needs to allow outbound internet access, which it already does.
2. frps registers the tunnel
The frps server authenticates the client via a shared token, then maps a local port on the VM (e.g. 8080) to a service inside the cluster (e.g. ingress-nginx:80). This mapping persists as long as the frpc pod is running.
3. NPM receives HTTPS, terminates TLS, forwards plaintext
Nginx Proxy Manager listens on ports 80 and 443, handles Let’s Encrypt certificates automatically, and proxies traffic to localhost:8080 — which goes through the FRP tunnel directly into your cluster's ingress controller.
4. Kubernetes routes the request
Ingress-nginx (or Kong Gateway) reads the Host header and routes traffic to the correct backend Service. Your application receives the request as if it came directly from the internet.
DigitalOcean, AWS, or Azure — which one?
This architecture is entirely cloud-agnostic. The VM is a dumb relay: it runs two Docker containers and forwards TCP. The choice of provider is mostly about cost and team familiarity.
ProviderVM TypeEst. Monthly CostBest ForDigitalOceanDroplet (1 vCPU / 1 GB)~$6/moStartups, simplicityAWSEC2 t3.micro / t4g.micro~$8–15/moAWS-native teamsAzureB1s Standard VM~$8–12/moMicrosoft / enterprise shopsLinode / AkamaiNanode (1 vCPU / 1 GB)~$5/moLowest cost
Whichever provider you choose, the setup is identical: provision a small Linux VM, install Docker and Docker Compose, and deploy the frps + npm stack.
> On AWS or Azure? Make sure your VM’s security group / NSG allows inbound on ports 80 and 443 (public traffic) and 7000 (frpc connections). Ports 81 (NPM admin) and 7500 (frps dashboard) should be restricted to your IP or accessed via SSH tunnelling only.
Setting it up from scratch
This takes about 30 minutes on a fresh VM.
1. The Docker Compose stack on the cloud VM
# /opt/docker-compose.yml — runs on your cloud VM
version: '3.8'
services:
npm:
image: jc21/nginx-proxy-manager:latest
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "81:81" # Admin UI — restrict with your firewall
volumes:
- /opt/npm/data:/data
- /opt/npm/letsencrypt:/etc/letsencrypt
frps:
image: snowdreamtech/frps:latest
restart: unless-stopped
network_mode: host # Required: frpc connects to host port 7000
volumes:
- /opt/frp/frps.toml:/etc/frp/frps.toml:ro
2. frps configuration
# /opt/frp/frps.toml
[common]
bindPort = 7000
token = "your-long-random-secret-here" # Generate: openssl rand -hex 32
[dashboard]
dashboardPort = 7500
dashboardUser = "admin"
dashboardPwd = "another-strong-password"
# Optional: restrict allowed port range for client registrations
# allowPorts = [{ start = 8000, end = 9999 }]
3. frpc inside Kubernetes
# frpc-deployment.yaml — deploy this in each cluster
apiVersion: apps/v1
kind: Deployment
metadata:
name: frpc
namespace: frpc
spec:
replicas: 1
selector:
matchLabels: { app: frpc }
template:
metadata:
labels: { app: frpc }
spec:
containers:
- name: frpc
image: snowdreamtech/frpc:latest
env:
- name: FRP_TOKEN
valueFrom:
secretKeyRef:
name: frp-secret
key: token
volumeMounts:
- name: config
mountPath: /etc/frp
volumes:
- name: config
configMap:
name: frpc-config
---
apiVersion: v1
kind: ConfigMap
metadata:
name: frpc-config
namespace: frpc
data:
frpc.toml: |
serverAddr = "YOUR_VM_IP"
serverPort = 7000
token = "{{ .Env.FRP_TOKEN }}"
[[proxies]]
name = "cluster-http"
type = "tcp"
localIP = "ingress-nginx-controller.ingress-nginx.svc.cluster.local"
localPort = 80
remotePort = 8080 # Use a unique port per cluster
> ⚠️ Never hardcode the token. Always inject it via a Kubernetes Secret, not the ConfigMap. The ConfigMap is readable by anyone with access to that namespace.
kubectl create secret generic frp-secret \
from-literal=token=YOUR_TOKEN \
-n frpc
4. Kubernetes Ingress resource
For each service you want to expose, create a standard Ingress pointing at its Service:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
namespace: myapp
spec:
ingressClassName: nginx
rules:
- host: myapp.yourdomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-web
port:
number: 3000
5. NPM proxy host (3 fields that matter)
In the NPM admin UI at http://your-vm-ip:81, add a Proxy Host:
FieldValueDomain Namesmyapp.yourdomain.comSchemehttp — always; NPM terminates TLSForward Hostname / IP172.18.0.1 (Docker bridge gateway)Forward Port8080 (or whichever port your frpc registered)Force SSL / HTTP/2Tick bothRequest a new SSL certificateTick
Hit Save. NPM handles the Let’s Encrypt HTTP-01 challenge automatically. DNS must already resolve to your VM before this step.
6. Verify end-to-end
# Should return 200 or your app's expected redirect
curl -I https://myapp.yourdomain.com
# If 404 from ingress-nginx, bypass NPM to narrow it down:
curl -H "Host: myapp.yourdomain.com" http://your-vm-ip:8080
# Check Kubernetes side:
kubectl get ing -A | grep myapp
Using Kong Gateway as a second routing layer
For services that need rate limiting, JWT authentication, request transformation, or API versioning, you can add Kong in front of specific routes without changing the tunnel setup.
Register a second frpc proxy pointing at Kong’s proxy port:
[[proxies]]
name = "kong-http"
type = "tcp"
localIP = "kong-kong-proxy.kong.svc.cluster.local"
localPort = 80
remotePort = 9180
Then use an HTTPRoute instead of an Ingress:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: myapp-route
namespace: myapp
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: kong-gateway
namespace: kong
sectionName: http
hostnames: ["api.yourdomain.com"]
rules:
- matches: [{ path: { type: PathPrefix, value: "/" } }]
backendRefs:
- kind: Service
name: myapp-api
port: 4000
weight: 1
Set NPM Forward Port to 9180 for that domain and you're done.
Security hardening — don’t stop at “it works”
The architecture is inherently more secure than exposing cluster IPs: your cluster has zero listening ports on the internet. But there’s more you should do.
🔒 Rotate the FRP token regularly
The shared token is your only authentication layer between frpc clients and frps. Use a 32+ byte random secret (openssl rand -hex 32). Rotate quarterly or after any team member departure. Update frps.toml on the VM and every cluster's Kubernetes Secret simultaneously.
🧱 Firewall the admin ports
Ports 81 (NPM admin) and 7500 (frps dashboard) must never be publicly accessible. Use your cloud provider’s firewall rules to restrict them to your office IP, or access them exclusively via SSH port forwarding:
ssh -L 81:localhost:81 -L 7500:localhost:7500 root@your-vm-ip
🛡️ Put Cloudflare in front
Proxying traffic through Cloudflare hides your VM’s real IP, provides DDoS protection, and lets you apply WAF rules for free. Enable “Under Attack” mode during incidents. Your cluster IP stays hidden behind both Cloudflare and the VM.
🔑 SSH key-only access
Disable password SSH on the VM. Only authorised engineers’ public keys belong in ~/.ssh/authorized_keys. Establish an offboarding process: when someone leaves, remove their key immediately.
📋 Namespace isolation in Kubernetes
Run the frpc Deployment in its own dedicated namespace. Apply a NetworkPolicy that restricts it to only reaching the ingress controller — it should have no path to your application pods directly.
🔍 Monitor tunnel health
Alert on frpc pod restarts (a restart = momentary downtime). The frps dashboard at :7500 shows all connected clients in real time. Pipe this data into your observability stack. Gaps in tunnel connectivity show up as 502 errors in NPM logs.
When you outgrow the single VM
This architecture scales further than you’d expect. Here’s the natural progression:
Level 1 — Single VM, multiple clusters (what we’ve built)
One VM, one frps, many frpc clients. Each cluster registers different remote ports. Works comfortably up to ~50 services and moderate traffic volumes.
Level 2 — Add Kong Gateway for advanced routing
Once you need rate limiting, JWT auth, or API versioning, add Kong as a second routing layer for specific services. Traffic flows: NPM → frps → frpc → Kong → your service. Your simpler services continue through ingress-nginx untouched.
Level 3 — Replace frps with Cloudflare Tunnel
Cloudflare’s own tunnelling product (cloudflared) is conceptually identical to frps but adds the Cloudflare global network in front for free. You trade self-hosted control for a globally distributed edge and zero VM maintenance. Migration is straightforward when you're ready.
Level 4 — Graduate to cloud-native load balancers
When traffic volume justifies the cost, replace the VM relay with a proper cloud load balancer (AWS ALB, Azure Application Gateway, GCP Cloud Load Balancing). Your Kubernetes Ingress manifests don’t change — only the network path changes.
> Each level is an upgrade path, not a migration. Your Kubernetes Ingress resources, domain names, and TLS certificates remain untouched as you move from FRP tunnels to Cloudflare Tunnel to a managed load balancer. The application code is completely insulated from the network topology.
When it doesn’t work — a systematic approach
Most failures fall into one of four buckets. Work through them in order.
1. DNS hasn’t propagated
dig +short yourdomain.com
If this doesn’t return your VM’s IP, the request never reaches NPM. Wait for propagation, or use Cloudflare’s near-instant DNS.
2. The frpc tunnel is down
kubectl -n frpc logs deploy/frpc
Connection refused or authentication errors mean the token is wrong or port 7000 is blocked by the VM’s firewall. Verify from inside the cluster:
nc -zv your-vm-ip 7000
3. NPM has no rule for the domain
curl -I -H "Host: yourdomain.com" http://your-vm-ip
A 404 from NPM means no proxy host rule exists for that domain. Add it in the UI.
4. The Kubernetes Ingress host header doesn’t match
# Bypasses NPM — hits cluster ingress directly
curl -H "Host: yourdomain.com" http://your-vm-ip:8080
If steps 1–3 pass but this returns a 404, your Ingress host: field doesn't match the domain exactly.
kubectl get ing -A | grep yourdomain
The takeaway for small teams
The pattern described here — a cloud VM acting as a public relay for outbound tunnels from on-premises Kubernetes — is one of those infrastructure solutions that punches well above its weight. A $6/month VM, two Docker containers, and a handful of Kubernetes manifests gives you:
- Automatic HTTPS with Let’s Encrypt for every domain
- Zero cluster IP exposure to the internet
- Multi-cluster routing from a single VM
- A clear upgrade path to Cloudflare Tunnel or cloud load balancers
- Observability at every layer (NPM logs, frps dashboard, kubectl pod logs)
It’s not the solution for a company serving millions of requests per second. But for a startup moving fast, shipping product, and watching the cloud bill — it’s hard to beat the simplicity-to-capability ratio.
Build it once. Document it well. Then get back to shipping.
Tags: DevOps · Kubernetes · Infrastructure · Startups · Networking · Cloud
Zero-Port Exposure: Routing On-Prem Traffic Through a Cloud VM Without Opening a Single Firewall… was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.