vm-core/docs/observability/README.md

Observability - VaultMesh

This directory contains a Prometheus exporter for VaultMesh and a Grafana dashboard.

Metrics Exposed

Metric	Type	Labels	Description
vaultmesh_receipts_total	Counter	module	Number of receipts emitted
vaultmesh_receipts_failed_total	Counter	module, reason	Failed receipt emissions
vaultmesh_anchor_age_seconds	Gauge	-	Seconds since last guardian anchor
vaultmesh_emit_seconds	Histogram	module	Receipt emit latency

Quick Start (Local)

Option 1: Run exporter directly

cd vaultmesh-observability
cargo run --release

Exposes metrics at http://0.0.0.0:9108/metrics

Option 2: Using Docker Compose

cd docs/observability
docker-compose up --build

Services:

• Exporter: http://localhost:9108/metrics
• Prometheus: http://localhost:9090
• Grafana: http://localhost:3000 (admin/admin)

Importing the Dashboard

1. Open Grafana at http://localhost:3000
2. Go to Dashboards → Import
3. Upload dashboards/receipts.json
4. Select the Prometheus data source
5. Click Import

CI Smoke Test

The smoke test verifies the exporter responds on /metrics:

cargo test -p vaultmesh-observability --tests

Add to .gitlab-ci.yml:

observability-smoke:
  stage: test
  image: rust:1.75
  script:
    - cargo test -p vaultmesh-observability --tests -- --nocapture

Environment Variables

Variable	Default	Description
VAULTMESH_METRICS_ADDR	0.0.0.0:9108	Listen address for metrics server

Guardian Metrics Integration Test

The Guardian engine has an integration test that verifies metrics are emitted after anchors:

cargo test -p vaultmesh-guardian --features metrics --test metrics_integration

This test:

• Starts ObservabilityEngine on a test port
• Creates Guardian with observability enabled
• Performs an anchor
• Verifies /metrics contains vaultmesh_anchor_age_seconds 0 (fresh anchor)

Integration with Other Engines

Other VaultMesh engines can record metrics by calling:

use vaultmesh_observability::ObservabilityEngine;
use std::sync::Arc;

let engine = Arc::new(ObservabilityEngine::new());

// Record successful receipt emission
engine.observe_emitted("guardian", latency_seconds);

// Record failure
engine.observe_failed("treasury", "io_error");

// Update anchor age (0 = just anchored)
engine.set_anchor_age(0.0);