> For the complete documentation index, see [llms.txt](https://docs.zetrix.com/zetrix-l2-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zetrix.com/zetrix-l2-documentation/reliability-and-security/monitoring.md).

# Monitoring

## Architecture

```mermaid
flowchart LR
    subgraph Exporters
        N[Nodes] & S[Sequencer] & V[Validators] & P[Proposer] & R[Relayer/Refueler] & E[Explorer]
    end
    Exporters -->|/metrics| PROM[Prometheus]
    Exporters -->|logs| LOGS[Log aggregation]
    PROM --> GRAF[Grafana]
    LOGS --> GRAF
    PROM --> AM[Alertmanager]
    AM --> CH[Email / Slack / PagerDuty]
```

## Metrics

| Domain    | Key metrics                                                        |
| --------- | ------------------------------------------------------------------ |
| Node      | Block height, peers, QBFT rounds, mempool size, RPC latency/errors |
| Sequencer | Batch rate/size, submission latency, EOA balance                   |
| Validator | Signature participation, validation latency, quorum health         |
| Proposer  | Assertion rate/success, challenge status, EOA balance              |
| Relayer   | Deposit lag, events processed, failures                            |
| Explorer  | Indexer lag, query latency                                         |
| Host      | CPU, memory, disk, network                                         |

## Logs

* Centralized aggregation, structured (JSON) logs, retention tiers, searchable.

## Health Checks

* Liveness/readiness endpoints per service; used by NLB and orchestration.

## Prometheus & Grafana

* Prometheus scrapes `/metrics`; Grafana dashboards per component and platform-wide.
* Alertmanager routes alerts to on-call.

## Suggested Alerts

| Alert               | Condition (example)        | Severity |
| ------------------- | -------------------------- | -------- |
| High CPU            | `cpu > 85%` for 5m         | Warning  |
| High Memory         | `mem > 90%` for 5m         | Warning  |
| Low Disk            | `disk_free < 15%`          | Critical |
| Disk trend          | Projected full < 7 days    | Warning  |
| RPC failures        | `error_rate > 5%` for 5m   | Critical |
| Node down           | Peer/quorum loss           | Critical |
| Batch failures      | Batch submission errors    | Critical |
| Validator failures  | Quorum/participation drop  | Critical |
| Proposer stale      | No assertion in N min      | Critical |
| Deposit lag         | Relayer lag > threshold    | Warning  |
| EOA low balance     | Below Refueler threshold   | Warning  |
| Refueler source low | Treasury near depletion    | Critical |
| Dead-man's-switch   | Monitoring pipeline silent | Critical |

> \[!NOTE] Always alert on the monitoring pipeline itself (dead-man's-switch) so that a monitoring outage is not mistaken for a healthy system.

***

[← Security](/zetrix-l2-documentation/reliability-and-security/security.md) · [Index](/zetrix-l2-documentation/readme.md) · [Scaling Strategy →](/zetrix-l2-documentation/reliability-and-security/scaling-strategy.md)