vm-cloudflare/TUNNEL-HARDENING.md

Cloudflare Tunnel Hardening Guide

Purpose

Security hardening guide for cloudflared deployments across VaultMesh and OffSec infrastructure. Ensures tunnels are isolated, credentials are protected, and monitoring is in place.

---

1. Secure Installation

Binary Verification

# Download official binary
curl -L https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 -o cloudflared

# Verify checksum (get from GitHub releases)
sha256sum cloudflared

# Make executable and move to secure location
chmod +x cloudflared
sudo mv cloudflared /usr/local/bin/

Package Installation (Preferred)

# Debian/Ubuntu
curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared jammy main' | sudo tee /etc/apt/sources.list.d/cloudflared.list
sudo apt update && sudo apt install cloudflared

---

2. Credential Governance

Credential Storage

# Create secure directory
sudo mkdir -p /etc/cloudflared
sudo chmod 700 /etc/cloudflared

# Store credentials with root-only access
sudo mv cert.pem /etc/cloudflared/
sudo mv <tunnel-id>.json /etc/cloudflared/
sudo chmod 600 /etc/cloudflared/*
sudo chown root:root /etc/cloudflared/*

Credential Rotation

• Rotate tunnel credentials every 90 days
• Delete old tunnel, create new one
• Update systemd service with new credential path
• Emit VaultMesh receipt for rotation event

Never Do

• Store credentials in world-readable locations
• Embed credentials in container images
• Commit credentials to git
• Use long-lived tokens without rotation policy

---

3. Systemd Service Isolation

Hardened Service File

# /etc/systemd/system/cloudflared.service
[Unit]
Description=Cloudflare Tunnel
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
ExecStart=/usr/local/bin/cloudflared tunnel --config /etc/cloudflared/config.yml run
Restart=on-failure
RestartSec=5

# Security Hardening
User=cloudflared
Group=cloudflared
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadOnlyPaths=/
ReadWritePaths=/var/log/cloudflared
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
RestrictNamespaces=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
RestrictRealtime=yes
SystemCallFilter=@system-service
SystemCallArchitectures=native

[Install]
WantedBy=multi-user.target

Create Service User

sudo useradd -r -s /usr/sbin/nologin cloudflared
sudo chown -R cloudflared:cloudflared /etc/cloudflared
sudo mkdir -p /var/log/cloudflared
sudo chown cloudflared:cloudflared /var/log/cloudflared

---

4. Configuration Hardening

Minimal Config (/etc/cloudflared/config.yml)

tunnel: <TUNNEL_ID>
credentials-file: /etc/cloudflared/<TUNNEL_ID>.json

# Metrics for monitoring
metrics: 127.0.0.1:9090

# Ingress rules - explicit deny-by-default
ingress:
  - hostname: app.vaultmesh.org
    service: http://127.0.0.1:8080
    originRequest:
      noTLSVerify: false
      connectTimeout: 10s

  - hostname: api.vaultmesh.org
    service: http://127.0.0.1:8081
    originRequest:
      httpHostHeader: api.internal

  # Catch-all: deny everything else
  - service: http_status:404

Security Settings

• Always set catch-all to 404 - no accidental exposure
• Use localhost bindings - origins never exposed publicly
• Enable TLS verification - don't disable unless absolutely necessary
• Set connection timeouts - prevent resource exhaustion

---

5. Origin Server Lockdown

Firewall Rules

# Allow only localhost connections to origin services
sudo iptables -A INPUT -p tcp --dport 8080 -s 127.0.0.1 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 8080 -j DROP

# Or with UFW
sudo ufw allow from 127.0.0.1 to any port 8080
sudo ufw deny 8080

No Public Ports

• Origin servers should have zero public ports
• All traffic flows through Cloudflare Tunnel
• If SSH needed, use Cloudflare Access for SSH

---

6. Monitoring & Alerting

Metrics Endpoint

# Enable in config.yml
metrics: 127.0.0.1:9090

# Scrape with Prometheus
curl http://127.0.0.1:9090/metrics

Key Metrics to Monitor

• cloudflared_tunnel_total_requests - request volume
• cloudflared_tunnel_request_errors - error rate
• cloudflared_tunnel_concurrent_requests_per_tunnel - load
• cloudflared_tunnel_response_by_code - HTTP status distribution
• cloudflared_tunnel_server_locations - edge connectivity

Alert Conditions

• Tunnel disconnected > 1 minute
• Error rate > 5%
• Connection to 0 edge servers
• Credential expiry approaching (30 days)

Log Forwarding

# Send logs to syslog/SIEM
cloudflared tunnel --loglevel info --logfile /var/log/cloudflared/tunnel.log run

---

7. VaultMesh Integration

Receipt Hooks

Every tunnel operation should emit a VaultMesh receipt:

# Tunnel events to capture
- tunnel_created
- tunnel_deleted
- credential_rotated
- config_updated
- service_added
- service_removed

Snapshot Anchoring

# Weekly config snapshot
cloudflared tunnel info <TUNNEL_ID> > /var/lib/vaultmesh/snapshots/tunnel-$(date +%Y%m%d).json

# Hash and anchor
blake3sum /var/lib/vaultmesh/snapshots/tunnel-*.json >> /var/lib/vaultmesh/anchors/tunnel-hashes.log

Audit Trail

• All tunnel changes logged with timestamp + actor
• Changes require dual approval for production tunnels
• Emergency access via break-glass procedure (logged separately)

---

8. Multi-Tunnel Architecture

Per-Service Tunnels

For OffSec cluster, use dedicated tunnels:

• tunnel-vaultmesh-core → Core API
• tunnel-vaultmesh-guardian → Guardian services
• tunnel-offsec-web → Public OffSec sites
• tunnel-offsec-internal → Internal tools

Benefits

• Blast radius containment
• Independent credential rotation
• Granular Access policies per tunnel

---

9. Security Checklist

Installation

• Binary verified via checksum
• Installed from official package repo
• Running as non-root user

Credentials

• Stored in /etc/cloudflared with 600 permissions
• Owned by root or service user only
• Rotation schedule documented (90 days)
• No credentials in git/images

Service

• Systemd hardening directives applied
• NoNewPrivileges=yes
• PrivateTmp=yes
• ProtectSystem=strict

Configuration

• Catch-all ingress returns 404
• All services bound to localhost
• TLS verification enabled
• Metrics endpoint enabled

Monitoring

• Prometheus scraping metrics
• Alerts for disconnection/errors
• Logs forwarded to SIEM
• VaultMesh receipts emitted

Network

• Origin has no public ports
• Firewall blocks non-localhost to origin ports
• Only Cloudflare Tunnel provides ingress

---

10. Emergency Procedures

Tunnel Compromise Response

1. Immediately delete compromised tunnel in CF dashboard
2. Revoke associated credentials
3. Create new tunnel with fresh credentials
4. Update config and restart service
5. Emit incident receipt in VaultMesh
6. Review Access logs for unauthorized access

Credential Leak Response

1. Rotate credentials immediately
2. Review Cloudflare audit logs
3. Check for unauthorized tunnel connections
4. Update all systems with new credentials
5. Document in incident report