Initial commit: Cloudflare infrastructure with WAF Intelligence

- Complete Cloudflare Terraform configuration (DNS, WAF, tunnels, access) - WAF Intelligence MCP server with threat analysis and ML classification - GitOps automation with PR workflows and drift detection - Observatory monitoring stack with Prometheus/Grafana - IDE operator rules for governed development - Security playbooks and compliance frameworks - Autonomous remediation and state reconciliation
2025-12-16 18:31:53 +00:00
commit 37a867c485
123 changed files with 25407 additions and 0 deletions
--- a/DEPLOYMENT_GUIDE.md
+++ b/DEPLOYMENT_GUIDE.md
@@ -0,0 +1,532 @@
+# DEPLOYMENT_GUIDE.md
+
+## OpenCode Cloudflare Infrastructure Deployment Guide
+
+**Status:** 🟢 Production Ready  
+**Version:** 1.0  
+**Updated:** December 9, 2025  
+**Governed by:** [RED-BOOK.md](RED-BOOK.md)
+
+---
+
+## Table of Contents
+
+1. [Quick Start](#quick-start)
+2. [Architecture Overview](#architecture-overview)
+3. [Environment Setup](#environment-setup)
+4. [Component Verification](#component-verification)
+5. [Compliance Oracle Usage](#compliance-oracle-usage)
+6. [Workflow Examples](#workflow-examples)
+7. [Troubleshooting](#troubleshooting)
+8. [Appendix](#appendix)
+
+---
+
+## Quick Start
+
+### 1. Prerequisites
+
+- macOS/Linux with bash >= 4.0
+- Python 3.9+
+- Node.js 18+ (for MCP servers)
+- Git 2.30+
+- OpenCode CLI installed
+
+### 2. Environment Variables (5 min)
+
+```bash
+# Essential (required for GitLab + Cloudflare)
+export GITHUB_TOKEN="ghp_..."           # GitHub PAT (already set)
+export GITLAB_TOKEN="glpat_..."         # GitLab PAT
+export GITLAB_URL="https://gitlab.com"  # or your self-hosted GitLab
+export CLOUDFLARE_API_TOKEN="..."       # Cloudflare API token
+export CLOUDFLARE_ACCOUNT_ID="..."      # Cloudflare account ID
+
+# Save to .env (source before running opencode)
+source /Users/sovereign/Desktop/CLOUDFLARE/.env
+```
+
+**How to Get Tokens:**
+
+- **GitLab PAT:** https://gitlab.com/-/user_settings/personal_access_tokens
+  - Scopes: `api`, `read_user`, `read_repository`, `write_repository`
+  - Expiry: 30 days
+
+- **Cloudflare API Token:** https://dash.cloudflare.com/profile/api-tokens
+  - Create custom token with: DNS:Read, Settings:Read, Firewall Rules:Read, Tunnels:Read
+  - Expiry: 1 year
+
+- **Cloudflare Account ID:** https://dash.cloudflare.com/ (right sidebar under Account)
+
+### 3. Verify Setup (3 min)
+
+```bash
+cd /Users/sovereign/Desktop/CLOUDFLARE
+
+# Run quick test
+bash TEST_WORKFLOW.sh quick
+
+# Expected output:
+# ✓ All environment variables set
+# ✓ Terraform files valid
+# ✓ All checks passed!
+```
+
+### 4. Launch OpenCode (1 min)
+
+```bash
+opencode
+/init
+
+# In OpenCode:
+/mcp list          # Verify MCPs load
+/agent cloudflare-ops
+# Now you can query your infrastructure
+```
+
+---
+
+## Architecture Overview
+
+### MCP Stack (16 MCPs)
+
+```
+┌─────────────────────────────────────────────────┐
+│         OpenCode Platform                       │
+│  (Claude API + MCP Router)                      │
+└─────────────────────────────────────────────────┘
+                        ↓
+┌─────────────────────────────────────────────────┐
+│  Enabled by Default (4 MCPs)                    │
+├─────────────────────────────────────────────────┤
+│ • filesystem      - Local file operations       │
+│ • git             - Git repository management   │
+│ • github          - GitHub API queries          │
+│ • gh_grep         - GitHub code search          │
+└─────────────────────────────────────────────────┘
+                        ↓
+┌─────────────────────────────────────────────────┐
+│  Per-Agent Optional (12 MCPs)                   │
+├─────────────────────────────────────────────────┤
+│ Core Infrastructure:                            │
+│ • gitlab          - GitLab API (CI/CD, repos)   │
+│ • cloudflare      - Cloudflare API (DNS, WAF)   │
+│ • postgres        - Query audit logs            │
+│ • sqlite          - Local analytics             │
+│                                                  │
+│ Advanced:                                       │
+│ • docker          - Container testing          │
+│ • aws             - AWS infrastructure          │
+│ • slack           - Notifications               │
+│ • linear          - Issue tracking              │
+│ • memory          - Knowledge base              │
+│ • context7        - Doc search                  │
+│ • web-scraper     - Web automation              │
+│ • googlemaps      - Location services           │
+└─────────────────────────────────────────────────┘
+```
+
+### Agent Ecosystem (3 Agents)
+
+| Agent | Purpose | Tools | Use Case |
+|-------|---------|-------|----------|
+| **cloudflare-ops** | Infrastructure & GitOps | filesystem, git, github, gitlab, cloudflare, gh_grep | Add DNS, update WAF, manage tunnels, validate infrastructure |
+| **security-audit** | Compliance & Security | filesystem, git, github, gitlab, cloudflare, gh_grep | Check PCI-DSS, review WAF rules, audit access controls |
+| **data-engineer** | Database Operations | filesystem, git, gitlab, postgres, sqlite | Query logs, analyze metrics, troubleshoot data pipelines |
+
+### Compliance Oracle Architecture
+
+```
+Question
+   ↓
+[oracle_runner.py]
+   ├─ Search Documents (framework-aware)
+   ├─ Extract Snippets (relevance scoring)
+   ├─ Build Context (citations)
+   ├─ Validate Answer (typing)
+   ├─ Hash Answer (SHA256)
+   └─ Emit Receipt (ledger.jsonl)
+   ↓
+Receipt (json)
+   ├─ timestamp
+   ├─ oracle_answer (full answer JSON)
+   ├─ answer_hash (SHA256)
+   └─ version (v0.4.0)
+```
+
+---
+
+## Environment Setup
+
+### 1. Configure opencode.jsonc
+
+The configuration is **already set up**. Key sections:
+
+```jsonc
+{
+  "mcp": {
+    // Enabled globally
+    "filesystem": { "enabled": true },
+    "git": { "enabled": true },
+    "github": { "enabled": true },
+    "gh_grep": { "enabled": true },
+    
+    // Per-agent (disabled globally, enabled per agent)
+    "gitlab": { "enabled": false },    // Enabled in cloudflare-ops, security-audit
+    "cloudflare": { "enabled": false } // Enabled in cloudflare-ops, security-audit
+  },
+  
+  "agents": {
+    "cloudflare-ops": {
+      "tools": {
+        "gitlab": true,
+        "cloudflare": true,
+        // + filesystem, git, github, gh_grep
+      }
+    }
+    // ... other agents
+  }
+}
+```
+
+### 2. Environment Variables
+
+Create or update `.env`:
+
+```bash
+# Copy from example
+cp .env.example .env
+
+# Edit and add your tokens
+export GITLAB_TOKEN="glpat_..."
+export CLOUDFLARE_API_TOKEN="..."
+export CLOUDFLARE_ACCOUNT_ID="..."
+
+# Verify
+source .env
+echo $GITLAB_TOKEN  # Should not be empty
+```
+
+### 3. Verify MCP Installation
+
+```bash
+# Inside opencode
+/mcp list
+
+# Expected:
+# ✓ filesystem (enabled globally)
+# ✓ git (enabled globally)
+# ✓ github (enabled globally, requires GITHUB_TOKEN)
+# ✓ gh_grep (enabled globally)
+# ⚠ gitlab (disabled globally, enabled per-agent, requires GITLAB_TOKEN)
+# ⚠ cloudflare (disabled globally, enabled per-agent, requires CLOUDFLARE_API_TOKEN)
+# ⚠ postgres (disabled, requires DATABASE_URL)
+# ... (other optional MCPs)
+```
+
+---
+
+## Component Verification
+
+### Test Suite
+
+```bash
+# Quick test (environment check)
+bash TEST_WORKFLOW.sh quick
+
+# Full test (integration tests)
+bash TEST_WORKFLOW.sh full
+```
+
+### Manual Verification
+
+**1. Git Integration**
+```bash
+cd /Users/sovereign/Desktop/CLOUDFLARE
+git log --oneline -n 3
+git status
+```
+
+**2. Terraform Validation**
+```bash
+cd terraform/
+terraform validate
+terraform fmt -check .
+```
+
+**3. Cloudflare API Test**
+```bash
+curl -X GET "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID" \
+  -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" | jq '.success'
+# Should return: true
+```
+
+**4. GitLab API Test**
+```bash
+curl -H "PRIVATE-TOKEN: $GITLAB_TOKEN" \
+  "$GITLAB_URL/api/v4/user" | jq '.name'
+# Should return your GitLab username
+```
+
+---
+
+## Compliance Oracle Usage
+
+### Quick Usage
+
+```bash
+# Run oracle for GDPR compliance
+python3 oracle_runner.py "Are we GDPR compliant?" --frameworks gdpr
+
+# Run oracle for NIS2 obligations
+python3 oracle_runner.py "What are NIS2 requirements?" --frameworks nis2
+
+# Run oracle for AI Act with verbose output
+python3 oracle_runner.py "What does AI Act Annex IV require?" --frameworks ai-act -v
+```
+
+### Oracle Output
+
+The oracle returns:
+
+1. **Answer** - Context-aware response with citations
+2. **Citations** - Linked documents with relevance scores
+3. **Gaps** - Identified compliance gaps with remediations
+4. **Receipt** - SHA256-hashed proof stored in COMPLIANCE_LEDGER.jsonl
+
+### Example: Golden Answer
+
+See: `examples/oracle_answer_ai_act.json` and `examples/oracle_receipt_ai_act.json`
+
+These demonstrate the complete oracle pipeline for a real compliance question.
+
+---
+
+## Workflow Examples
+
+### Example 1: Add HTTPS Enforcement
+
+**Task:** Add HTTPS enforcement to all zones
+
+```bash
+opencode
+/agent cloudflare-ops
+
+# Query: Add HTTPS enforcement to all zones, then show me the plan
+```
+
+**Behind the scenes:**
+1. Agent uses `cloudflare` MCP to query current zones
+2. Agent uses `filesystem` to read `terraform/zones.tf`
+3. Agent uses `git` to track changes
+4. Agent generates terraform plan
+5. You review and approve
+
+### Example 2: Audit WAF Rules for PCI-DSS
+
+**Task:** Check if WAF rules meet PCI-DSS requirements
+
+```bash
+opencode
+/agent security-audit
+
+# Query: Review our WAF rules in terraform/waf.tf and check PCI-DSS compliance
+```
+
+**Behind the scenes:**
+1. Agent uses `filesystem` to read WAF configuration
+2. Agent uses `gh_grep` to find similar PCI-DSS patterns
+3. Agent searches documentation for compliance mappings
+4. Agent generates audit report with gaps
+
+### Example 3: Incident Response
+
+**Task:** DNS compromise detection and remediation
+
+```bash
+opencode
+/agent cloudflare-ops
+
+# Query: A domain is showing unauthorized DNS records. Query Cloudflare to see current records, check playbooks/, and generate a remediation plan.
+```
+
+**Behind the scenes:**
+1. Agent uses `cloudflare` MCP to query live DNS records
+2. Agent uses `filesystem` to read `playbooks/DNS-COMPROMISE-PLAYBOOK.md`
+3. Agent uses `git` to prepare rollback commits
+4. Agent generates step-by-step remediation
+
+### Example 4: Compliance Report
+
+**Task:** Generate PCI-DSS compliance report
+
+```bash
+# Use oracle directly
+python3 oracle_runner.py "What are our PCI-DSS compliance gaps?" --frameworks pci-dss
+
+# Then use agent to generate remediation plan
+opencode
+/agent security-audit
+
+# Query: Based on the gaps, create a 30-day remediation plan
+```
+
+---
+
+## Troubleshooting
+
+### MCP Won't Load
+
+**Symptom:** `/mcp list` shows error for `gitlab` or `cloudflare`
+
+**Solution:**
+1. Verify tokens are exported: `echo $GITLAB_TOKEN`
+2. Check token format: `glpat_` for GitLab, bearer token for Cloudflare
+3. Verify network connectivity: `curl https://api.cloudflare.com/client/v4/zones`
+
+### Terraform Validate Fails
+
+**Symptom:** `terraform validate` returns errors
+
+**Solution:**
+1. Run `terraform init` first
+2. Check `terraform.tfvars` exists and is valid
+3. Verify Cloudflare provider version in `.terraform.lock.hcl`
+
+### Oracle Returns "Insufficient Context"
+
+**Symptom:** Oracle answer shows `insufficient_context: true`
+
+**Solution:**
+1. Ensure documentation files exist in project root
+2. Check file names match in `oracle_runner.py` line 97
+3. Add more detailed documentation files
+4. Test with verbose mode: `python3 oracle_runner.py ... -v`
+
+### Token Expired
+
+**Symptom:** API calls return 401 Unauthorized
+
+**Solution:**
+1. **GitLab:** Renew PAT at https://gitlab.com/-/user_settings/personal_access_tokens
+2. **Cloudflare:** Renew token at https://dash.cloudflare.com/profile/api-tokens
+3. Update `.env` and re-source: `source .env`
+
+---
+
+## Appendix
+
+### A. File Structure
+
+```
+/Users/sovereign/Desktop/CLOUDFLARE/
+├── opencode.jsonc           # 16 MCPs configured (DO NOT edit unless expert)
+├── .env                      # Your environment variables (DO NOT commit)
+├── .env.example              # Template for .env (safe to commit)
+├── TEST_WORKFLOW.sh          # Integration test suite
+├── oracle_runner.py          # Compliance oracle v0.4.0
+├── AGENTS.md                 # Agent documentation
+├── MCP_GUIDE.md              # Complete MCP reference
+├── GITLAB_CLOUDFLARE_AUTH.md # Token setup guide
+├── DEPLOYMENT_GUIDE.md       # This file
+│
+├── terraform/                # Infrastructure code
+│   ├── main.tf
+│   ├── zones.tf
+│   ├── dns.tf
+│   ├── waf.tf
+│   ├── tunnels.tf
+│   ├── access.tf
+│   └── ...
+│
+├── gitops/                   # CI/CD agents
+│   ├── plan_summarizer.py
+│   ├── ci_plan_comment.py
+│   ├── drift_pr_bot.py
+│   └── webhook_receiver.py
+│
+├── playbooks/                # Incident response
+│   ├── DNS-COMPROMISE-PLAYBOOK.md
+│   ├── TUNNEL-ROTATION-PROTOCOL.md
+│   └── waf_incident_playbook.md
+│
+├── scripts/                  # Automation utilities
+│   ├── state-reconciler.py
+│   ├── drift_guardian_py.py
+│   ├── autonomous_remediator_py.py
+│   └── invariant_checker_py.py
+│
+├── observatory/              # Monitoring & observability
+│   ├── metrics-exporter.py
+│   ├── prometheus.yml
+│   ├── alertmanager/
+│   └── dashboards/
+│
+├── examples/                 # Golden examples
+│   ├── oracle_answer_ai_act.json
+│   └── oracle_receipt_ai_act.json
+│
+└── COMPLIANCE_LEDGER.jsonl   # Created by oracle_runner.py
+```
+
+### B. Supported Frameworks
+
+| Framework | Key Doc | Focus |
+|-----------|---------|-------|
+| **PCI-DSS** | cloudflare_waf_baseline.md | Network security, access controls |
+| **GDPR** | zero_trust_architecture.md | Data protection, access logging |
+| **NIS2** | TUNNEL-HARDENING.md | Network resilience, monitoring |
+| **AI Act** | WEB-INFRA-SECURITY-PATTERNS.md | Governance, explainability |
+| **SOC2** | WEB-INFRA-SECURITY-PATTERNS.md | Security controls, audit logs |
+| **ISO27001** | zero_trust_architecture.md | Information security management |
+
+### C. Quick Reference
+
+| Task | Command |
+|------|---------|
+| Start OpenCode | `opencode` |
+| Initialize | `/init` |
+| List MCPs | `/mcp list` |
+| Start agent | `/agent cloudflare-ops` |
+| Run oracle | `python3 oracle_runner.py "question"` |
+| Validate terraform | `cd terraform && terraform validate` |
+| Test setup | `bash TEST_WORKFLOW.sh quick` |
+| View git log | `git log --oneline -n 10` |
+| Query Cloudflare | OpenCode (with cloudflare-ops agent) |
+| Query GitLab | OpenCode (with cloudflare-ops agent) |
+
+### D. Common Queries
+
+**For cloudflare-ops agent:**
+- "What DNS records do we have for example.com?"
+- "Show me our WAF rules and check if they block SQL injection"
+- "List all tunnel configurations"
+- "Create a terraform plan to add HTTPS enforcement"
+- "Show recent changes in GitLab that affect infrastructure"
+
+**For security-audit agent:**
+- "Are we compliant with PCI-DSS?" 
+- "Review WAF rules for OWASP compliance"
+- "Check if access controls meet GDPR standards"
+- "Audit DNS configurations for security risks"
+
+**For oracle_runner.py:**
+- "python3 oracle_runner.py 'What are NIS2 incident reporting requirements?'"
+- "python3 oracle_runner.py 'Summarize our AI Act obligations' --frameworks ai-act"
+- "python3 oracle_runner.py 'Check GDPR data retention requirements' -v"
+
+---
+
+## Support & Feedback
+
+**OpenCode Issues:** https://github.com/sst/opencode/issues
+
+**Project Issues:** Create issue in your project repo
+
+**Documentation:** See AGENTS.md, MCP_GUIDE.md, GITLAB_CLOUDFLARE_AUTH.md
+
+---
+
+**Last Updated:** December 8, 2025  
+**Status:** 🟢 Production Ready  
+**Next Review:** December 15, 2025