vm-cloudflare/STRUCTURE.md

# VaultMesh — Project Structure

**Master Index** | Governed by the [Red Book](RED-BOOK.md)

---

## Documentation by Purpose

### Getting Started
| Document | Purpose |
|----------|---------|
| [README.md](README.md) | Project overview & quick start |
| [FIRST_RUN.md](FIRST_RUN.md) | First-time walkthrough |
| [DEPLOYMENT_GUIDE.md](DEPLOYMENT_GUIDE.md) | Production deployment |

### Core Operations
| Document | Purpose |
|----------|---------|
| [AGENTS.md](AGENTS.md) | OpenCode agent definitions |
| [MCP_GUIDE.md](MCP_GUIDE.md) | MCP server reference |
| [MULTI_ACCOUNT_AUTH.md](MULTI_ACCOUNT_AUTH.md) | Multi-account setup (canonical) |
| [GITLAB_CLOUDFLARE_AUTH.md](GITLAB_CLOUDFLARE_AUTH.md) | GitLab & Cloudflare tokens |
| [NVIDIA_INTEGRATION.md](NVIDIA_INTEGRATION.md) | NVIDIA AI integration |

### Security & Architecture
| Document | Purpose |
|----------|---------|
| [WEB-INFRA-SECURITY-PATTERNS.md](WEB-INFRA-SECURITY-PATTERNS.md) | Security patterns checklist |
| [zero_trust_architecture.md](zero_trust_architecture.md) | Zero-Trust architecture |
| [cloudflare_waf_baseline.md](cloudflare_waf_baseline.md) | WAF baseline rules |
| [cloudflare_dns_manifest.md](cloudflare_dns_manifest.md) | DNS baseline |
| [TUNNEL-HARDENING.md](TUNNEL-HARDENING.md) | Tunnel hardening |
| [SECURITY_WAF_INTEL.md](SECURITY_WAF_INTEL.md) | WAF intelligence guardrails |

### AI Agent Governance
| Document | Purpose |
|----------|---------|
| [AGENT_GUARDRAILS.md](AGENT_GUARDRAILS.md) | AI coding guardrails |
| [IDE_OPERATOR_RULES.md](IDE_OPERATOR_RULES.md) | Operator doctrine |
| [RED-BOOK.md](RED-BOOK.md) | **Governing doctrine** |

### Cognitive Layer
| Document | Purpose |
|----------|---------|
| [COGNITION_FLOW.md](COGNITION_FLOW.md) | Query processing flow |
| [DEMO_COGNITION.md](DEMO_COGNITION.md) | Demo transcripts |
| [DECISION_MATRIX.md](DECISION_MATRIX.md) | Decision framework |

### Incident Response
| Document | Purpose |
|----------|---------|
| [playbooks/DNS-COMPROMISE-PLAYBOOK.md](playbooks/DNS-COMPROMISE-PLAYBOOK.md) | DNS incident response |
| [playbooks/TUNNEL-ROTATION-PROTOCOL.md](playbooks/TUNNEL-ROTATION-PROTOCOL.md) | Tunnel rotation |
| [playbooks/waf_incident_playbook.md](playbooks/waf_incident_playbook.md) | WAF incident response |

### Subsystem Documentation
| Document | Purpose |
|----------|---------|
| [terraform/README.md](terraform/README.md) | Terraform usage |
| [gitops/README.md](gitops/README.md) | GitOps workflows |
| [observatory/README.md](observatory/README.md) | Monitoring stack |
| [systemd/README.md](systemd/README.md) | Systemd units |

### Archive (Historical)
| Document | Purpose |
|----------|---------|
| [archive_docs/](archive_docs/) | Deprecated & historical docs |

---

## Directory Structure

```
CLOUDFLARE/
├── mcp/                          # MCP Tool Implementations
│   ├── oracle_answer/            # Oracle compliance tool
│   │   ├── __init__.py
│   │   └── tool.py              # OracleAnswerTool class (single responsibility)
│   │
│   └── tools/                    # (Future MCP tools)
│       ├── tunnel_admin/
│       ├── security_check/
│       └── threat_intel/
│
├── scripts/                      # Standalone utilities
│   ├── threat-intel-collector.py
│   ├── state-reconciler.py
│   └── autonomous_remediator_py.py
│
├── observatory/                  # Monitoring & analytics
│   ├── metrics-exporter.py
│   ├── waf-intel.py             # (Phase 7)
│   └── dashboards/
│
├── gitops/                       # CI/CD & GitOps agents
│   ├── plan_summarizer.py
│   ├── ci_plan_comment.py
│   ├── drift_pr_bot.py
│   ├── waf-rule-generator.py    # (Phase 7)
│   └── config.yml
│
├── terraform/                    # Infrastructure code
│   ├── main.tf
│   ├── zones.tf
│   ├── dns.tf
│   ├── waf.tf
│   └── ...
│
├── playbooks/                    # Incident response
│   ├── DNS-COMPROMISE-PLAYBOOK.md
│   ├── TUNNEL-ROTATION-PROTOCOL.md
│   └── waf_incident_playbook.md
│
├── examples/                     # Golden examples
│   ├── oracle_answer_ai_act.json
│   └── oracle_receipt_ai_act.json
│
├── opencode.jsonc               # MCP configuration (16 MCPs)
├── AGENTS.md                    # Agent documentation
├── DEPLOYMENT_GUIDE.md          # Production deployment
├── STRUCTURE.md                 # This file
├── PRODUCTION_READY_SUMMARY.md  # Build summary
└── .env                         # Environment variables (DO NOT commit)
```

---

## Coding Standards

### 1. Single Responsibility Rule (SRP)

**Each file should do ONE thing:**

| File | Responsibility |
|------|-----------------|
| `oracle_runner.py` | Document search, citation linking, gap identification |
| `mcp/oracle_answer/tool.py` | Wrap oracle_runner for consistent API |
| CLI script | Parse args, format output, call tool |

**Bad Example:**
```python
# ❌ DON'T DO THIS
def main():
    # Parse args
    parser = argparse.ArgumentParser()
    parser.add_argument("--question", ...)
    
    # Initialize tool
    tool = OracleAnswerTool()
    
    # Search docs (200 lines)
    # Build answer (300 lines)
    # Format output (150 lines)
    # ... all in one function
```

**Good Example:**
```python
# ✅ DO THIS
# tool.py - wraps oracle_runner
class OracleAnswerTool:
    def answer(self, question, frameworks):
        result = self.runner.run(question, frameworks)
        return ToolResponse(...)

# cli.py - only handles CLI
def main():
    tool = OracleAnswerTool()
    response = tool.answer(args.question, args.frameworks)
    print(format_pretty(response))
```

### 2. Argument Parser Rule

**RULE: Define args in ONE place only.**

When modifying `argparse`:
1. ✅ Modify the `build_parser()` function
2. ✅ Add new arguments there
3. ❌ DO NOT append args in other functions
4. ❌ DO NOT duplicate argument definitions

**Correct Pattern:**
```python
def build_parser() -> argparse.ArgumentParser:
    """Single source of truth for CLI args."""
    parser = argparse.ArgumentParser(description="...")
    
    # Add ALL args here
    parser.add_argument("--question", ...)
    parser.add_argument("--frameworks", ...)
    parser.add_argument("--verbose", ...)
    
    return parser

def main():
    parser = build_parser()  # Use it, don't modify it
    args = parser.parse_args()
    # ... rest of logic
```

### 3. File Editing Rule for Agents

**When an agent (Cline, Claude) needs to modify a file:**

1. ✅ Read the ENTIRE file first (don't patch blind)
2. ✅ Rewrite whole functions/blocks (not line-by-line patches)
3. ✅ Check for duplicates before editing
4. ❌ Never append to a section without reviewing it
5. ❌ Never add "quick fixes" that create tech debt

**For Cline/Claude instructions:**
```
Before editing any Python file:
1. Read the entire file
2. Check if your change already exists
3. Rewrite the WHOLE function, not just insert lines
4. Never create duplicate argument definitions
5. If in doubt, rewrite the whole file cleanly
```

### 4. Module Imports Rule

**Import at the top of the file, organized:**

```python
# Standard library
import sys
import json
from typing import Optional, List, Dict
from dataclasses import dataclass

# Third-party
import requests

# Local
from .tool import OracleAnswerTool
from ..oracle_runner import OracleRunner
```

**Bad:**
```python
# ❌ Scattered imports
def main():
    import argparse  # Don't do this
    ...
    from .tool import OracleAnswerTool  # And this
```

### 5. Type Hints Rule

**All functions must have type hints:**

```python
# ✅ CORRECT
def answer(
    self,
    question: str,
    frameworks: Optional[List[str]] = None,
    verbose: bool = False
) -> ToolResponse:
    """Answer a compliance question."""
    ...

# ❌ WRONG
def answer(self, question, frameworks=None, verbose=False):
    ...
```

### 6. Docstring Rule

**Every module, class, and function needs a docstring:**

```python
"""
ORACLE_ANSWER MCP TOOL
Version: 0.2.0
Responsibility: Wrap oracle_runner for consistent API.
"""

class OracleAnswerTool:
    """Tool for answering compliance questions."""
    
    def answer(self, question: str, ...) -> ToolResponse:
        """Answer a compliance question.
        
        Args:
            question: Compliance question
            frameworks: Optional frameworks to search
        
        Returns:
            ToolResponse with answer and receipt
        """
```

---

## File Modification Process

### When You Need to Change a File

**Step 1: Understand the change**
- What is the problem?
- What part of the code needs to change?
- Will it affect other parts?

**Step 2: Read the whole file**
- Don't just edit the first matching line
- Look for duplicates
- Check the overall structure

**Step 3: Make the change**
- Rewrite the whole function if needed
- Don't patch or append
- Maintain consistency

**Step 4: Verify**
- Test the change
- Check for new errors
- Run tests

### When an AI Agent Changes a File

**Before asking an agent to edit:**
```
DO NOT patch lines in [filename].
If you need to change [function_name]:
1. First, read the entire file
2. Check if the change already exists
3. Rewrite the WHOLE function cleanly
4. Never create duplicates

For argparse specifically:
- Never append argument definitions
- Always rewrite the entire build_parser() function
- Check for conflicting option strings before writing
```

---

## Testing Standards

### Run Tests Before Committing

```bash
# Test suite
bash TEST_WORKFLOW.sh quick    # Quick environment check
bash TEST_WORKFLOW.sh full     # Full integration test

# Specific tests
python3 oracle_runner.py "test question" --frameworks gdpr
python3 oracle_answer_mcp.py --tool-info
python3 oracle_answer_mcp.py --question "test?" --frameworks gdpr
```

### Error Categories

| Category | Example | Fix |
|----------|---------|-----|
| **Import Error** | `ModuleNotFoundError` | Check sys.path, imports at top |
| **Type Error** | `TypeError: x cannot be subscripted` | Add proper type hints, check nulls |
| **Duplicate Arg** | `argparse.ArgumentError` | Review full argparse block, remove duplicates |
| **File Not Found** | `FileNotFoundError` | Verify base_path, check absolute paths |

---

## Future Phases Structure

### Phase 7: WAF Intelligence Engine

```
observatory/
  ├── waf-intel.py            # WAF log analyzer
  ├── threat-feeds/           # Threat intelligence data
  │   ├── feed_collector.py
  │   └── threat_scores.json
  └── dashboards/
      └── waf-intelligence.json

gitops/
  ├── waf-rule-generator.py   # ML-based rule proposer
  ├── waf_incident_playbook.md

scripts/
  └── threat-intel-collector.py
```

### Phase 8: Multi-Tenant Isolation

```
terraform/
  ├── tenants/
  │   ├── customer_a/
  │   ├── customer_b/
  │   └── customer_c/
  └── shared/

mcp/
  ├── tenant_admin/
  └── isolation_checker/
```

---

## Code Review Checklist

Before committing code, verify:

- [ ] No duplicate definitions
- [ ] Type hints on all functions
- [ ] Docstrings present
- [ ] Error handling implemented
- [ ] Tests passing
- [ ] No secrets in code
- [ ] Single responsibility per file
- [ ] Imports organized
- [ ] Code follows style guide

---

## Common Mistakes & Fixes

### Mistake 1: Duplicate Arguments

**Symptom:**
```
argparse.ArgumentError: argument --question: conflicting option string
```

**Cause:** Argument defined twice in argparse

**Fix:** Read full `build_parser()`, remove duplicates

---

### Mistake 2: Type Error on None

**Symptom:**
```
TypeError: 'NoneType' object is not subscriptable
```

**Cause:** Accessing dict/list key without checking if None

**Fix:** Add null checks
```python
answer = response.answer
if answer:
    value = answer.get('key', 'default')
```

---

### Mistake 3: Patchy Edits

**Symptom:** File has multiple similar functions, unclear which is used

**Cause:** Agent appended "fixes" instead of rewriting

**Fix:** Rewrite whole file cleanly, remove duplicates

---

## Guardrails for Agents

### When Using Cline/Claude/Agents:

**DO:**
- ✅ Read entire file before editing
- ✅ Rewrite whole functions
- ✅ Check for duplicates
- ✅ Maintain single responsibility
- ✅ Test after changes

**DON'T:**
- ❌ Append code blindly
- ❌ Create duplicate definitions
- ❌ Mix concerns in one function
- ❌ Ignore type errors
- ❌ Skip testing

---

## Coding Standards Summary

| Rule | Importance |
|------|-----------|
| Single responsibility per file | Critical |
| Type hints on all functions | Critical |
| Docstrings present | High |
| No duplicate definitions | Critical |
| No secrets in code | Critical |

For detailed coding standards, see the full coding standards section above.

---

## Documentation Invariants

**Enforced by:** `scripts/doc-invariants.sh`

### Checklist (for any doc change)

- [ ] If I added/removed/moved a doc → I updated STRUCTURE.md
- [ ] Multi-account config lives only in MULTI_ACCOUNT_AUTH.md
- [ ] Incident procedures live only in `playbooks/`
- [ ] Cognition/agent docs reference RED-BOOK.md
- [ ] No dead links outside `archive_docs/`
- [ ] Playbook paths use correct casing (UPPERCASE)

### Run the checker

```bash
bash scripts/doc-invariants.sh
```

---

**Version:** 2.0  
**Last Updated:** December 9, 2025  
**Governed by:** [RED-BOOK.md](RED-BOOK.md)