shigoto_metrics

View Source

Prometheus metrics collectors and Grafana dashboard for shigoto.

Subscribes to the shigoto telemetry surface and exposes it as canonical Prometheus metrics. A ready-to-import Grafana dashboard lives in priv/grafana/shigoto.json. shigoto itself takes no dependency on this package, and this package has no web dependency — it only needs telemetry and prometheus.

Quickstart

Start the application (it declares the metrics and attaches the telemetry handlers at boot):

{ok, _} = application:ensure_all_started(shigoto_metrics).

Then serve shigoto_metrics:scrape/0 from your app's web layer and point Prometheus at it. On Nova — shigoto's web framework — that is a one-line controller:

-module(metrics_controller).
-export([index/1]).

index(_Req) ->
    {status, 200, #{~"content-type" => ~"text/plain; version=0.0.4"},
        shigoto_metrics:scrape()}.

wired into your routes:

{"/metrics", fun metrics_controller:index/1, #{methods => [get]}}

Reusing your existing Nova listener means the endpoint sits behind whatever auth and network policy your app already has. Import priv/grafana/shigoto.json into Grafana.

Not on Nova? Return scrape/0 from any handler you already run, or mount prometheus_cowboy2_handler on your own Cowboy listener (add prometheus_cowboy to your app — it is not a dependency here).

What gets exported

MetricTypeLabels
shigoto_jobs_inserted_totalcounterworker, queue
shigoto_jobs_claimed_totalcounterworker, queue
shigoto_jobs_completed_totalcounterworker, queue
shigoto_jobs_failed_totalcounterworker, queue
shigoto_jobs_snoozed_totalcounterworker, queue, reason
shigoto_jobs_discarded_totalcounterworker, queue
shigoto_jobs_cancelled_totalcounterworker, queue
shigoto_queue_polls_totalcounterqueue
shigoto_queue_paused_totalcounterqueue
shigoto_queue_resumed_totalcounterqueue
shigoto_resilience_rate_limited_totalcounterworker
shigoto_resilience_circuit_open_totalcounterworker
shigoto_resilience_bulkhead_full_totalcounterworker
shigoto_resilience_load_shed_totalcounterpriority
shigoto_batches_completed_totalcounter—
shigoto_cron_scheduled_totalcountername, worker
shigoto_testing_mode_armed_totalcountermode
shigoto_job_duration_secondshistogramworker, queue
shigoto_job_queue_wait_secondshistogramworker, queue
shigoto_resilience_retry_after_secondshistogramworker

High-cardinality identifiers (job_id, batch_id) are kept out of labels; they stay on the raw telemetry events for tracing sinks that need them. The per-job [shigoto, job, progress] event is intentionally not exported.

shigoto_testing_mode_armed_total being non-zero in production is a red flag: a testing insert mode (inline/manual) is armed and inserts are not persisting. Alert on it.

Cardinality: label values must be bounded, developer-controlled dimensions. Never route untrusted input into a queue name, a priority, or a snooze reason — each distinct value mints a new time series.

Configuration

KeyDefaultMeaning
auto_setuptrueDeclare metrics and attach handlers at application start. Set false to call shigoto_metrics:setup/0 yourself.

License

MIT.