Fix Vector config and OpenObserve auth for live E2E verification
Ticket 10 verification surfaced runtime issues that static validation
missed; all verified against running Docker stack (Vector 0.43 + OO 0.92.2):
- Rewrite VRL for Vector 0.43: no coalesce/default funcs, merge!/replace!
for guarded fallible calls, no two-value parse_json destructuring
- Quote env-interpolated sink credentials (empty env left a bare YAML null)
- Fix logs sink URI: OO needs /api/{org}/{stream}/_json (stream segment
was missing, causing 404); logs now land in contract_check stream
- Replace OPENOBSERVE_AUTH_TOKEN with root email/password Basic auth:
prometheus_remote_write ignores request.headers, so remote-write got 401
- Update .env.example, DEPLOY.md, ARCHITECTURE.md, ticket 05 accordingly
Verified live: logs with service + correlation_id searchable in
OpenObserve; contract_check_* metrics queryable via its Prometheus API;
no outbound port 4318 connections from app containers.
This commit is contained in:
febux
parent
b279d6c61a
commit
85db70887d
5 changed files with 46 additions and 48 deletions
|
|
@ -48,10 +48,6 @@ REFUND_FULL_USAGE_THRESHOLD=0.20 # ≤20% consumed + within window ⇒ full ref
|
|||
|
||||
# --- Observability (leave empty to disable) ---
|
||||
SENTRY_DSN=
|
||||
# Auth header for OpenObserve. Used by Vector in the `observer` profile to forward
|
||||
# logs and metrics. Generate with: echo -n 'user:pass' | base64
|
||||
# Default value is for root@example.com:Complexpass#123.
|
||||
OPENOBSERVE_AUTH_TOKEN=cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT is no longer used. The application does not push
|
||||
# OTLP; Vector/OpenObserve collects logs/metrics passively from stdout and
|
||||
# /metrics endpoints. Remove this line from existing .env files.
|
||||
|
|
@ -59,6 +55,8 @@ OPENOBSERVE_AUTH_TOKEN=cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=
|
|||
# OTEL_EXPORTER_OTLP_HEADERS=
|
||||
|
||||
# --- OpenObserve (profile: observer) ---
|
||||
# Vector (profile `observer`) authenticates to OpenObserve with these
|
||||
# credentials, so keep them in sync with the OpenObserve root user below.
|
||||
OPENOBSERVE_PORT=5080
|
||||
OPENOBSERVE_ROOT_USER_EMAIL=root@example.com
|
||||
OPENOBSERVE_ROOT_USER_PASSWORD=Complexpass#123
|
||||
|
|
|
|||
|
|
@ -26,7 +26,10 @@ sources:
|
|||
- http://worker-billing:9105/metrics
|
||||
- http://worker-notify:9103/metrics
|
||||
scrape_interval_secs: 15
|
||||
# Auth header is injected via a remap transform when METRICS_BEARER_TOKEN is set.
|
||||
# The token is optional: an empty METRICS_BEARER_TOKEN leaves /metrics open.
|
||||
authorization:
|
||||
strategy: bearer
|
||||
token: "${METRICS_BEARER_TOKEN-}"
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Transforms
|
||||
|
|
@ -38,43 +41,32 @@ transforms:
|
|||
inputs:
|
||||
- docker_logs
|
||||
source: |
|
||||
# Keep the raw message and container name; derive a short service label.
|
||||
.container = .container_name ?? .container_id ?? "unknown"
|
||||
.service = .container_name ?? "unknown"
|
||||
# Derive a short service label from the container metadata
|
||||
# (docker_logs always provides container_name).
|
||||
.container = .container_name
|
||||
.service = .container_name
|
||||
|
||||
# If the log line is JSON from our structured logger, merge its fields
|
||||
# without overwriting Vector/container metadata.
|
||||
raw = .message ?? ""
|
||||
raw = .message
|
||||
if is_string(raw) {
|
||||
parsed, err = parse_json(raw)
|
||||
if err == null && is_object(parsed) {
|
||||
for_each(parsed) -> |key, value| {
|
||||
if !exists(.) || !exists(.[key]) {
|
||||
.[key] = value
|
||||
parsed = parse_json(raw) ?? null
|
||||
if is_object(parsed) {
|
||||
. = merge!(parsed, .)
|
||||
service = parsed.service
|
||||
if service != null {
|
||||
.service = service
|
||||
}
|
||||
}
|
||||
# Ensure OpenObserve can filter by the canonical service label even when
|
||||
# the JSON payload carries its own "service" field.
|
||||
if exists(parsed.service) {
|
||||
.service = parsed.service
|
||||
}
|
||||
if exists(parsed.correlation_id) {
|
||||
.correlation_id = parsed.correlation_id
|
||||
correlation_id = parsed.correlation_id
|
||||
if correlation_id != null {
|
||||
.correlation_id = correlation_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Trim container runtime prefixes for a cleaner service label.
|
||||
.service = replace(.service, "^contract_check-", "")
|
||||
|
||||
add_metrics_auth:
|
||||
type: remap
|
||||
inputs:
|
||||
- service_metrics
|
||||
source: |
|
||||
token = get_env_var("METRICS_BEARER_TOKEN") ?? ""
|
||||
if token != "" {
|
||||
.headers = {"Authorization": "Bearer " + token}
|
||||
# Trim the compose project prefix for a cleaner service label.
|
||||
if is_string(.service) {
|
||||
.service = replace!(.service, r'^contract_check-', "")
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
|
|
@ -86,12 +78,12 @@ sinks:
|
|||
type: http
|
||||
inputs:
|
||||
- enrich_logs
|
||||
uri: http://openobserve:5080/openobserve/api/default/_json
|
||||
uri: http://openobserve:5080/openobserve/api/default/contract_check/_json
|
||||
method: post
|
||||
auth:
|
||||
strategy: basic
|
||||
user: ""
|
||||
password: ${OPENOBSERVE_AUTH_TOKEN}
|
||||
user: "${OPENOBSERVE_ROOT_USER_EMAIL-}"
|
||||
password: "${OPENOBSERVE_ROOT_USER_PASSWORD-}"
|
||||
encoding:
|
||||
codec: json
|
||||
batch:
|
||||
|
|
@ -104,12 +96,14 @@ sinks:
|
|||
openobserve_metrics:
|
||||
type: prometheus_remote_write
|
||||
inputs:
|
||||
- add_metrics_auth
|
||||
- service_metrics
|
||||
# OpenObserve answers Vector's healthcheck probe with 405; this is
|
||||
# harmless — actual remote-write POSTs succeed (200).
|
||||
endpoint: http://openobserve:5080/openobserve/api/default/prometheus/api/v1/write
|
||||
auth:
|
||||
strategy: basic
|
||||
user: ""
|
||||
password: ${OPENOBSERVE_AUTH_TOKEN}
|
||||
user: "${OPENOBSERVE_ROOT_USER_EMAIL-}"
|
||||
password: "${OPENOBSERVE_ROOT_USER_PASSWORD-}"
|
||||
batch:
|
||||
max_events: 100
|
||||
timeout_secs: 1
|
||||
|
|
|
|||
|
|
@ -390,8 +390,9 @@ services:
|
|||
# profile: obs → Grafana + Prometheus + Loki (mature, heavier).
|
||||
# Use for production-grade visibility.
|
||||
#
|
||||
# For the `observer` profile, set OPENOBSERVE_AUTH_TOKEN so Vector can write
|
||||
# to OpenObserve. No OTLP endpoint configuration is needed in the app.
|
||||
# Vector (profile `observer`) authenticates to OpenObserve with
|
||||
# OPENOBSERVE_ROOT_USER_EMAIL / OPENOBSERVE_ROOT_USER_PASSWORD. No OTLP
|
||||
# endpoint configuration is needed in the app.
|
||||
|
||||
# ── LIGHTWEIGHT OBSERVABILITY (profile: observer) ───────────────────────────
|
||||
openobserve:
|
||||
|
|
@ -428,7 +429,8 @@ services:
|
|||
container_name: contract_check-vector
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
OPENOBSERVE_AUTH_TOKEN: ${OPENOBSERVE_AUTH_TOKEN}
|
||||
OPENOBSERVE_ROOT_USER_EMAIL: ${OPENOBSERVE_ROOT_USER_EMAIL:-root@example.com}
|
||||
OPENOBSERVE_ROOT_USER_PASSWORD: ${OPENOBSERVE_ROOT_USER_PASSWORD:-Complexpass#123}
|
||||
METRICS_BEARER_TOKEN: ${METRICS_BEARER_TOKEN:-}
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
|
|
|
|||
|
|
@ -1087,7 +1087,8 @@ None of 1–5 requires touching `core/` application code — only compose/infra.
|
|||
| `REFUND_WINDOW_DAYS` | `14` | full-refund window (§8a) |
|
||||
| `REFUND_FULL_USAGE_THRESHOLD` | `0.20` | usage ratio for full refund |
|
||||
| `SENTRY_DSN` | (empty) | if set, init sentry |
|
||||
| `OPENOBSERVE_AUTH_TOKEN` | default creds base64 | Basic auth token Vector uses to forward logs/metrics to OpenObserve |
|
||||
| `OPENOBSERVE_ROOT_USER_EMAIL` | `root@example.com` | OpenObserve root user; Vector uses it for Basic auth |
|
||||
| `OPENOBSERVE_ROOT_USER_PASSWORD` | `Complexpass#123` | OpenObserve root password; Vector uses it for Basic auth |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | (removed) | no longer used; the application does not push OTLP |
|
||||
| `OTEL_EXPORTER_OTLP_HEADERS` | (removed) | no longer used |
|
||||
| `OTEL_SERVICE_NAME` | (removed) | service name is set by `configure_logging()` and compose labels |
|
||||
|
|
@ -1400,9 +1401,11 @@ enforcing the adapter boundary even in dependency ordering.
|
|||
compose network (`api:8000/metrics`, workers on `:9101`/`:9102`/`:9104`/`:9105`/`:9103`).
|
||||
- Vector enriches logs with `service` (container name) and preserves
|
||||
`correlation_id` from JSON log lines.
|
||||
- Logs are sent to `http://openobserve:5080/openobserve/api/default/_json` and
|
||||
- Logs are sent to
|
||||
`http://openobserve:5080/openobserve/api/default/contract_check/_json` and
|
||||
metrics to `http://openobserve:5080/openobserve/api/default/prometheus/api/v1/write`,
|
||||
both authenticated with `OPENOBSERVE_AUTH_TOKEN`.
|
||||
both authenticated with the OpenObserve root credentials
|
||||
(`OPENOBSERVE_ROOT_USER_EMAIL` / `OPENOBSERVE_ROOT_USER_PASSWORD`).
|
||||
- OpenObserve UI is exposed under `/openobserve/` via nginx.
|
||||
- Worker metrics ports are no longer published on the Docker host; they are only
|
||||
reachable inside the compose network for scraping.
|
||||
|
|
|
|||
|
|
@ -144,9 +144,10 @@ PRICE_PER_DOC_KOPECKS=19900 # 199 ₽ за документ без подпис
|
|||
|
||||
# --- Observability (опционально) ---
|
||||
SENTRY_DSN=https://...@sentry.io/...
|
||||
# OpenObserve auth token used by Vector in the `observer` profile.
|
||||
# Generate with: echo -n 'user:pass' | base64
|
||||
OPENOBSERVE_AUTH_TOKEN=cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=
|
||||
# Vector (профиль `observer`) авторизуется в OpenObserve с этими кредами
|
||||
# (совпадают с root-пользователем OpenObserve).
|
||||
OPENOBSERVE_ROOT_USER_EMAIL=root@example.com
|
||||
OPENOBSERVE_ROOT_USER_PASSWORD=Complexpass#123
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT больше не используется: приложение не шлёт OTLP.
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue