diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ad955ce..8b59866 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -15,6 +15,7 @@ repos: exclude: ^deploy/vps/docker-compose\.override\.example\.yml$ - id: check-toml - id: check-added-large-files + exclude: ^docs/ - id: check-merge-conflict - id: mixed-line-ending args: [--fix=lf] diff --git a/docs/ARCHITECTURE_AS_BUILT.md b/docs/ARCHITECTURE_AS_BUILT.md new file mode 100644 index 0000000..a176e25 --- /dev/null +++ b/docs/ARCHITECTURE_AS_BUILT.md @@ -0,0 +1,363 @@ +# ARCHITECTURE (AS BUILT) — «Контракт-чек» + +> This document describes the system **as it exists in the code today**, verified +> against `src/`, `migrations/`, `tests/`, and compose files. The design-time +> rationale, decision history and future steps remain in +> [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) (design doc); this file is the +> as-built map. Domain language is defined in the root [`CONTEXT.md`](../CONTEXT.md). + +--- + +## 1. What the system is + +LLM-powered screening of contract (deal) documents under Russian / Belarusian +civil law. A user (Telegram bot, web JWT auth, or B2B API key) uploads a +document; an event-driven pipeline extracts text, prescreens it, optionally +runs deep LLM analysis, and produces a markdown report with findings, quotes +and clause references. Document consumption is metered: subscription Quota +first, then prepaid Credits, with ЮKassa payments (topups, subscriptions, +renewals, refunds with clawback). + +**Stack:** Python 3.14, uv + hatchling, FastAPI, SQLAlchemy 2 (async) + +Alembic, aio-pika (RabbitMQ), MinIO (S3), Redis, aiogram 3, httpx, +Prometheus + structlog + Sentry, OpenObserve/Vector (or Grafana/Loki) for +observability, Nginx + certbot at the edge. Docker Compose deployment +(one Dockerfile per service in `srv/`), k8s-ready but not k8s-running. + +--- + +## 2. Hexagonal shape and import rules + +``` +src/contract_check/ +├── core/ # domain — owns ALL state and side effects +│ ├── db/ # models, session, repositories/ +│ ├── mq/ # topology, publisher, consumer, messages, management +│ ├── s3/ # Storage port + MinIO adapter +│ ├── llm/ # LLMProvider port + ollama_cloud / yandex_gpt adapters +│ ├── billing/ # PaymentProvider port + yookassa, quota, fulfillment, refunds +│ ├── extraction/ # DocumentExtractor port + format adapters +│ ├── analysis/ # chunker, checklist, analyzer, report_schema +│ ├── notifications/ # SMTP transport + notify publisher +│ ├── security/ # argon2 passwords +│ └── config.py logging.py metrics.py telemetry.py sentry.py errors.py +│ credits.py tokens.py api_keys.py rate_limit.py auth*.py passkeys.py +│ redis_client.py review/ +├── api/ # FastAPI image (routes/, admin/, billing/ pay pages, schemas/) +├── worker_extract/ # CPU image: pymupdf + mammoth + striprtf + chardet + tesseract +├── worker_prescreen/# hybrid metadata extraction + routing +├── worker_analyze/ # I/O image: LLM provider +├── worker_notify/ # email (aiosmtplib) +├── worker_billing/ # timer scheduler (no MQ) +└── bot/ # aiogram adapter — HTTP-only +``` + +**Boundary rules (statically enforced for the bot by +`tests/unit/test_bot_boundary.py`):** + +- `core/*` may import anything. +- `api/*`, `worker_*/*` import only `core/*` (+ their own modules). +- `bot/*` imports only `httpx`/`aiohttp`/aiogram and its own modules — never + `core.db`, `core.s3`, `core.llm`, `core.mq`, `core.credits`. It talks to the + api over HTTP exclusively, and may run on a separate host + (`docker-compose.bot.yml`). + +**Control plane:** `api` is the only writer for user-initiated mutations +(upload → reserve Document Slot → MinIO → publish). Workers write their own +stage rows / reports; credits move only through the idempotent compensation +helpers (`core/billing/quota.compensate_document_slot`, `core/credits.refund_credit`). + +--- + +## 3. Services and the pipeline + +``` +Telegram ──► bot (aiogram, HTTP-only) ──HTTP──► api (FastAPI) ──publish──► RabbitMQ + owns: PG, MinIO, Redis, │ + quota/credits, tokens ▼ + extract.q ──► worker-extract + (CPU: pymupdf/mammoth/tesseract) + │ publish + ▼ + prescreen.q ──► worker-prescreen + (hybrid heuristic + optional LLM) + │ deep_analysis │ manual_review (terminal) + ▼ ▼ + analyze.q ──► worker-analyze ──► Postgres (Report, done) +notify.x ──► notify.q ──► worker-notify (SMTP: reset, magic link, …) +worker-billing (60 s Postgres-advisory-lock tick; no MQ) +``` + +### api (`api/`, `srv/api/`) + +- `create_app` (`api/app.py:115`): logging/Sentry, MinIO `ensure_bucket`, + MQ publisher (publisher confirms), Redis rate-limiter with in-memory + fallback, admin seed. Middleware: CORS, correlation-id propagation, + `http_request_duration`, access logs, redacted debug payloads, generic 500. +- Routes under `/api/v1`: `auth/` (Telegram bot/web/miniapp, email+password + JWT pair, passkeys, magic links, support), `documents` (upload → 202), + `reports` (polling + SSE `/events`), `me` (profile, overview, documents, + telegram bind, password), `billing` (plans, checkout, invoices, + subscription, autorenew, refund), `b2b` (API-key analyze/status/usage/ + profile/keys CRUD), `webhooks/yookassa`. Plus `/healthz`, `/readyz`, + `/metrics` (bearer-gated), server-rendered `/pay/{invoice_id}` (short-lived + JWT pay token), and `/admin/*` (Jinja2 + HTMX panel: users, credits, + invoices/refunds, hold clearing, gift subscriptions, manual-review queue, + MQ queues/DLQ requeue). +- **Upload flow** (`api/services.py:56` `upload_and_enqueue`): validate + suffix/format → reject on `billing_hold` (402) → stream body with size cap + (413) → MinIO put (DB failure triggers best-effort S3 cleanup) → insert + `documents` (queued) + `jobs` (extract/pending) → reserve Document Slot + (Quota-then-Credit when `PLANS_ENABLED`, else legacy `reserve_credit`; 402 + on `NoCredits`) → publish `DocumentUploaded` (publisher confirms; on MQ + failure: release slot / refund credit, status `publish_failed`) → + `202 {document_id, correlation_id, credits_left}`. + +### worker-extract (`worker_extract/`, metrics :9101, prefetch 1) + +Consumes `extract.q` / `DocumentUploaded`. Idempotency check against terminal +statuses → status `extracting` → download blob → `extract_document` +(`core/extraction/factory.py`: python-magic sniff → uploader MIME → suffix; +adapters: pdf/pymupdf, docx/mammoth, rtf/striprtf, txt+csv/chardet, +images/tesseract OCR; unsupported formats are terminal + refunded) → upload +extracted markdown to MinIO → publish `PrescreenRequested` (or +`DocumentExtracted` straight to analyze.q when `PRESCREEN_ENABLED=false`). +Failure classes: `extraction_failed` (non-refundable under `infra_only`), +`ocr_failed`, `infra`. Terminal failure → DLQ + slot compensation. + +### worker-prescreen (`worker_prescreen/`, metrics :9104, prefetch 1) + +Consumes `prescreen.q` / `PrescreenRequested`. Handler +(`worker_prescreen/handler.py:116`): transition `prescreening` → download +text → **hybrid extractor** (`extractor_hybrid.py`): Stage 1 deterministic +heuristic (`heuristic-v2`, regex-free, keyword/positional); Stage 2 LLM +fallback only if enabled AND confidence < `PRESCREEN_LLM_FALLBACK_THRESHOLD` +(LLM failure never fatal; LLM fills heuristic `None`s, booleans OR-merged, +confidence rescored) → **router** (`router.py:30`): missing +type/parties/low-confidence → `manual_review` (terminal status, admin queue); +amount ≥ `PRESCREEN_HIGH_VALUE_THRESHOLD` or penalty/arbitration clause → +`deep_analysis` (publish `AnalyzeRequested` with `prescreen_meta`); +`auto_approve` disabled by default (remapped to manual_review; when enabled +writes a lightweight Report, `status=done`). Persists `prescreen_results`. + +### worker-analyze (`worker_analyze/`, metrics :9102, prefetch 3) + +Consumes `analyze.q` / `AnalyzeRequested`. Status `analyzing/llm` → download +text → `provider.analyze(text, checklist)` (chunking, fan-out under +`asyncio.Semaphore(LLM_MAX_CONCURRENCY)`, json-schema + repair loop, 429 → +fallback model — all inside the LLM adapter) → `render_markdown` (findings, +quotes, clause refs, «не заменяет юриста» disclaimer) → `reports` upsert, +`status=done`. Failure classes: `llm_quota`, `llm_timeout`, +`llm_invalid_output`, `infra`; DLQ → slot compensation. + +### worker-notify (`worker_notify/`, metrics :9103, prefetch 5) + +Consumes `notify.q` / `NotificationMessage` (kinds: password_reset, welcome, +email_verification, magic_link). SMTP via aiosmtplib; dev logger when +`SMTP_HOST` empty. + +### worker-billing (`worker_billing/`, metrics :9105, no MQ) + +60-second tick guarded by a **Postgres advisory lock** (single replica): +renewal invoices for auto-renew subscriptions expiring < 3 days; period roll / +`past_due` (grace) / `expired` transitions; reconciliation of pending +invoices older than 15 minutes via the same `apply_payment_status` used by +the webhook. No-op safe when ЮKassa disabled. + +### bot (`bot/`, `srv/bot/`) + +aiogram 3, polling or webhook mode (aiohttp, secret-token auth). Commands: +`/start /help /profile /balance /plans /reports /status`; checkout callbacks +(`plan:*`, `topup:*`); document upload → API → poll report with stage labels; +per-user rate limit (Redis, memory fallback); service-token login cached per +`telegram_id`; `ensure_disclaimer` guarantees the legal disclaimer. + +--- + +## 4. RabbitMQ topology (declared idempotently in `core/mq/topology.py`) + +| Object | Type / args | Purpose | +|---|---|---| +| `contracts.x` | direct | main exchange; RK `extract` / `prescreen` / `analyze` | +| `extract.q` / `prescreen.q` / `analyze.q` | **quorum**, DLX → `contracts.retry.x` | main work queues | +| `contracts.retry.x` + `*.retry.q` | direct + classic (per-message TTL, `lazy` policy via mgmt API) | delay slots; TTL expiry dead-letters the message back to `contracts.x` | +| `extract.dlq` / `prescreen.dlq` / `analyze.dlq` | quorum | poison; manual requeue via admin panel / `core/mq/management.py` | +| `notify.x` / `notify.q` / `notify.retry.q` / `notify.dlq` | mirror of the above | notification pipeline | + +**Retry mechanics** (`core/mq/consumer.py`): generic `Consumer[MsgT]` with +`handle`/`classify`/`on_failure`/`on_dlq` hooks. On failure the consumer +re-publishes to the retry exchange with `expiration = MQ_RETRY_BASE_MS · +2^(attempt−1)` (default 2 s base → 2/4/8/16/32 s) and acks the original. +`x-attempt >= MQ_MAX_ATTEMPTS` (5) → DLQ with `x-failure-class` headers, +`jobs.dlq=true`, `documents.status=failed`, slot compensation. Pydantic +validation failure → straight to DLQ (`infra` poison). `core.errors.TerminalError` +bypasses retries. If the failure hooks themselves raise, the message is +retried (hook-failure fallback). Publisher uses **publisher confirms** — +unconfirmed publish fails the request, so a paid slot never silently vanishes. + +Message schemas (`core/mq/messages.py`, pydantic v2, base `PipelineMessage` +with `next_attempt()`): `DocumentUploaded`, `DocumentExtracted`, +`PrescreenRequested`, `AnalyzeRequested` (carries `prescreen_meta`), +`PrescreenCompleted`, `NotificationMessage`. Headers: `x-correlation-id`, +`x-attempt`, `x-origin`; `delivery_mode=2`. + +**Idempotency:** every handler re-reads `documents.status` first; terminal or +in-flight → ack and exit. No double-LLM, no double-refund. + +--- + +## 5. Data model (Postgres, 16 tables, `core/db/models.py`; migrations 0001–0011) + +| Table | Role | +|---|---| +| `users` | identity (telegram_id and/or email), argon2 `password_hash`, reset/magic-link token hashes, `role` (user/admin), `credits_left` (CHECK ≥ 0), `billing_hold` | +| `passkey_credentials` | WebAuthn credentials (credential_id, public_key, sign_count) | +| `documents` | one upload: s3 keys, status `queued→extracting→prescreening→ocr→analyzing→done\|failed\|manual_review`, `refunded` | +| `reports` | 1:1 with document: `content_json` JSONB + `markdown`, model, tokens, latency, prescreen link | +| `prescreen_results` | extracted metadata (type, parties, amount, dates, clause flags), `confidence_score`, `routing_decision`, `extractor_version` | +| `jobs` | per-stage correlation (UNIQUE document+queue), attempts, `last_failure_class`, `dlq` | +| `service_tokens` | per-adapter bearer auth (bot/web/cli), revocable | +| `api_keys` / `api_key_requests` | B2B keys (SHA-256 hash, per-key rps + monthly quota) and per-call usage ledger | +| `invoices` | money in **integer kopecks**; kinds `topup`/`subscription`/`renewal`; statuses draft→pending→succeeded/cancelled/refunded | +| `plans` / `subscriptions` / `quota_usage` | seeded plan catalog; one active-or-past_due subscription per user (partial unique); quota ledger with UNIQUE(document_id) idempotency | +| `credit_events` | append-only credit ledger (`delta ≠ 0`, `balance_after ≥ 0` self-verifying) | +| `user_profiles` | 1:1 passive settings (language, TZ, notif/dashboard prefs) | + +**Data access:** all SQL lives in `core/db/repositories/` (14 repositories); +a repository receives an `AsyncSession` and never commits — callers own +transactions. Raw `text()` only inside repositories/migrations. Status/enum +columns are `TEXT + CHECK` (not PG enums) so migrations stay additive. Alembic +is async; CI exercises `upgrade head` → `downgrade base`. + +--- + +## 6. Credits, Quota, refunds (billing invariants, `core/billing/`) + +- **Reserve-on-enqueue:** the API synchronously reserves a Document Slot + before publishing (`core/billing/quota.py:22` + `reserve_document_slot`): `SELECT … FOR UPDATE` on the active subscription → + quota rows this period < `plan.monthly_quota` → insert `quota_usage` + (`ON CONFLICT DO NOTHING`) → source `quota`; else atomic credit reserve → + `credits`; else 402. `PLANS_ENABLED=false` → legacy credits-only path. +- **Compensation (exactly-once):** on terminal processing failure, + `compensate_document_slot` (`quota.py:100`) releases the quota row or + refund the credit (guarded by `documents.refunded`), honouring + `REFUND_POLICY=all|infra_only` with the failure-class taxonomy + (`extraction_failed` non-refundable under `infra_only`). +- **Payments:** `PaymentProvider` port (`port.py`) + `YookassaProvider` + (httpx, Basic auth, Idempotence-Key = invoice UUID, kopecks↔RUB only at + the HTTP boundary). `YOOKASSA_ENABLED=false` → `PaymentsDisabled` → + billing mutations 503, catalog readable. +- **Webhook trust** (`api/routes/webhooks.py`): Basic auth constant-time + check, then the payment is **re-fetched via REST** — the push payload's + amount is never trusted; fulfillment is idempotent + (`core/billing/fulfillment.apply_payment_status`), also used by the + reconciliation sweep. +- **Refunds (D10)** (`core/billing/refunds.py`): within `REFUND_WINDOW_DAYS` + (14) and usage ≤ 20 % of purchased → full; else proportional + `max(0, amount − used × PRICE_PER_DOC)`; otherwise 409. Execution: + provider refund → invoice `refunded` → **clawback** (unspent credits / + subscription + period quota rows) → `users.billing_hold = TRUE` when the + balance would go negative; uploads 402 until an admin clears the hold. + +--- + +## 7. LLM provider port (`core/llm/`) + +`LLMProvider` Protocol (`port.py:31`): `analyze(text, *, checklist)` and +`extract_prescreen(text)`. Registered providers via `LLM_PROVIDER`: +`ollama_cloud` (default; `format: json-schema` chat, model +`qwen2.5:14b`, fallback `qwen2.5:7b`) and `yandex_gpt` +(`yandexgpt-lite`, `Api-Key`, JSON_OBJECT). Shared behaviour in each adapter: +chunk fan-out under a semaphore, repair loop (one REPAIR re-send on invalid +JSON, then `LLMError`), 429 → fallback model → `LLMQuotaError`, exponential +HTTP backoff for timeouts/5xx, terminal `LLMConfigError` for connect/404. +Checklist: 10 frozen items (`core/analysis/checklist.py`); `Finding` schema +(`report_schema.py`) doubles as the Ollama json-schema and tolerates alias +field names. + +--- + +## 8. AuthN/AuthZ surface + +| Caller | Mechanism | +|---|---| +| bot adapter | `Authorization: Bearer ` (only `/api/v1/auth/telegram/bot`) → user JWT exchange | +| Telegram web / Mini App | Login Widget hash / `initData` verification → JWT | +| web user | email+password → JWT pair (access 24 h + refresh 30 d, Redis-backed revocation); passkeys (WebAuthn, Redis challenges) and magic links → single access JWT | +| B2B | `X-API-Key` (SHA-256, token-bucket rate limit, monthly quota) | +| admin panel | HttpOnly cookie + `users.role = admin` | +| ЮKassa webhook | Basic (shopId:secret) | +| health/metrics | none / bearer | + +Redis is used only for rate limiting, refresh-token store, passkey +challenges and bot limiter — never as a job queue. + +--- + +## 9. Storage (MinIO, `core/s3/`) + +Single bucket `contract-check-docs`; keys `users/{uid}/docs/{did}.{ext}` +(original) and `users/{uid}/docs/{did}.txt` (extracted markdown), built only +via key builders. Uploads proxy through the API (multipart, 25 MiB cap, +streamed with limit). `minio-init` one-shot creates the bucket and the ILM +expiry (`DOC_RETENTION_DAYS` 7 / `TEXT_RETENTION_DAYS` 30) — 152-ФЗ lever: +raw text leaves on schedule; reports live in Postgres and survive the purge. + +--- + +## 10. Observability & deploy + +- **Logs/metrics/traces:** structlog JSON + correlation-id contextvar + (propagated through HTTP headers and MQ headers), Prometheus per-service + (`/metrics` on api :9100; workers :9101–:9105 scraped in-network by + Vector), Sentry (DSN-gated). Passive collection: `observer` profile = + OpenObserve + Vector; `obs` profile = Prometheus + Loki + Promtail + + Grafana. OTLP push was removed. +- **Compose** (`docker-compose.yml`): default = infra only (postgres:18 with + `wal_level=replica` + WAL archive, redis:8, rabbitmq:4, minio + minio-init); + profiles `services` (api + 5 workers), `bot` (standalone-able), + `edge` (nginx + certbot + exporter), `observer`/`obs`. Host ports offset: + PG 15432, Redis 17379, Rabbit 5672/15672, MinIO 9000/9001, api 8000, + OpenObserve 5080, edge 80/443. +- **Images:** one Dockerfile per service in `srv/` (two-stage uv build, + `uv sync --no-default-groups --group ` → python:3.14-slim); dependency + groups (PEP 735) per service keep images minimal; bot is leanest. +- **CI** (`.github/workflows/ci.yml`): ruff check+format, `ty` typecheck, + unit tests with `--cov-fail-under=50`, integration tests against an + isolated compose test stack (`docker-compose.test.yml`, offset ports). + +--- + +## 11. Testing + +- **Unit** (`tests/unit/`, ~40 files / 326 tests): no infra; respx for LLM + HTTP; includes `test_bot_boundary.py` (AST check of the hexagonal bot rule). +- **Integration** (`tests/integration/`, 21 files / 134 tests, `-m + integration`): full docker-compose test stack, alembic upgrade, seeded + service token, ASGI httpx + LifespanManager; covers upload pipeline, each + worker, credits/quota DB, auth flows, passkeys/magic links, B2B, admin + panel, billing checkout/webhook/pay page, subscriptions, refunds. +- `tests/conftest.py` deduplicates `src.contract_check.*` vs + `contract_check.*` module loading (avoids duplicate Prometheus timeseries). + +--- + +## 12. Cross-cutting conventions + +1. Money is integer kopecks everywhere; decimals exist only at the ЮKassa + HTTP boundary. +2. Enum-ish columns are `TEXT + CHECK`; migrations additive and hand-written. +3. Repositories never commit; callers own transactions. +4. Every queue handler is idempotent via `documents.status` re-read; + refunds/slot releases are exactly-once via guards. +5. Publisher confirms on every publish that follows a paid reservation. +6. External providers (LLM, payments, storage) sit behind ports in `core/`; + adapters are swappable and respx-testable. +7. Feature degrade switches: `PLANS_ENABLED`, `YOOKASSA_ENABLED`, + `PRESCREEN_ENABLED`, `PRESCREEN_LLM_FALLBACK_ENABLED`, + `SMTP_HOST` (empty = dev logger), `REFUND_POLICY`. +8. Domain language (Plan/Subscription/Quota/Credits/Topup/Invoice/Renewal/ + Saved Payment Method/Dunning/Signup Bonus/ЕРИП/Billing Hold/Clawback/ + Document Slot) is normative — see root `CONTEXT.md`; ADRs live in + `docs/adr/`. diff --git a/docs/diagrams/contract-check-architecture.html b/docs/diagrams/contract-check-architecture.html new file mode 100644 index 0000000..2a81158 --- /dev/null +++ b/docs/diagrams/contract-check-architecture.html @@ -0,0 +1,15272 @@ + + + + + + + Контракт-чек — Production Architecture Diagram + + + + + + + + +
+ +
+
+
+

Контракт-чек — Production Architecture

+
+
+ + + + + + + +
+ + Контракт-чек — Production Architecture + An architecture diagram generated by Archify. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Telegram User · chat client · Architecture component + + + + Telegram User + chat client + + + + Web User · browser / SPA · Architecture component + + + + Web User + browser / SPA + + + + B2B Client · X-API-Key · Architecture component + + + + B2B Client + X-API-Key + + + + Admin · /admin panel · Architecture component + + + + Admin + /admin panel + + + + Telegram Bot · aiogram · HTTP-only · Single VPS · Docker Compose · adapter + + + + Telegram Bot + aiogram · HTTP-only + adapter + + + + Web UI · future SPA · SSE · Single VPS · Docker Compose · adapter (planned) + + + + Web UI + future SPA · SSE + adapter (planned) + + + + Nginx + certbot · TLS edge · reverse proxy · Single VPS · Docker Compose · edge layer + + + + Nginx + certbot + TLS edge · reverse proxy + edge layer + + + + API · FastAPI · /api/v1 · /admin · Single VPS · Docker Compose › core domain · contract_check.core · core · sole writer + + + + API + FastAPI · /api/v1 · /admin + core · sole writer + + + + Redis · rate limit · sessions · Single VPS · Docker Compose › infra · stateful · future + + + + Redis + rate limit · sessions + future + + + + RabbitMQ · contracts.x · quorum · DLQ · Single VPS · Docker Compose › infra · stateful · direct exchange + + + + RabbitMQ + contracts.x · quorum · DLQ + direct exchange + + + + worker-extract · pymupdf · mammoth · tesseract · Single VPS · Docker Compose › core domain · contract_check.core · CPU image + + + + worker-extract + pymupdf · mammoth · tesseract + CPU image + + + + worker-prescreen · heuristic → LLM fallback · router · Single VPS · Docker Compose › core domain · contract_check.core · CPU image + + + + worker-prescreen + heuristic → LLM fallback · router + CPU image + + + + worker-analyze · LLM provider · repair loop · Single VPS · Docker Compose › core domain · contract_check.core · I/O image + + + + worker-analyze + LLM provider · repair loop + I/O image + + + + worker-billing · renewals · expiry · reconcile · Single VPS · Docker Compose › core domain · contract_check.core · scheduler · 60s tick + + + + worker-billing + renewals · expiry · reconcile + scheduler · 60s tick + + + + worker-notify · aiosmtplib · password reset · Single VPS · Docker Compose › core domain · contract_check.core · email + + + + worker-notify + aiosmtplib · password reset + email + + + + LLM Provider · Ollama Cloud · YandexGPT · Architecture component · provider port + + + + LLM Provider + Ollama Cloud · YandexGPT + provider port + + + + PostgreSQL · source of truth · WAL archive · Single VPS · Docker Compose › infra · stateful · replication-ready + + + + PostgreSQL + source of truth · WAL archive + replication-ready + + + + MinIO · raw docs + extracted text · TTL · Single VPS · Docker Compose › infra · stateful · S3 API + + + + MinIO + raw docs + extracted text · TTL + S3 API + + + + ЮKassa · payments · webhooks · Architecture component · billing port + + + + ЮKassa + payments · webhooks + billing port + + + + SMTP · email delivery · Architecture component · notify transport + + + + SMTP + email delivery + notify transport + + + + + + Telegram updates + + + + HTTPS + + + + X-API-Key + + + + admin cookie + + + + HTTP → :443 + + + + HTTPS → :443 + + + + proxy :443 → :8000 + + + + publish DocumentUploaded + + + + rate limit · sessions + + + + users · documents · invoices + + + + upload raw doc + + + + extract.q · prefetch=1 + + + + prescreen.q · prefetch=1 + + + + analyze.q · prefetch=3 + + + + notify.q + + + + PrescreenRequested + + + + AnalyzeRequested + + + + prescreen_results + + + + save Report · status=done + + + + read extracted text + + + + analyze(text, checklist) + + + + LLM fallback + + + + renewals · quota · ledger + + + + payment · refund · reconcile + + + + raw + .txt blobs + + + + send email + + + + webhook payment.succeeded + + + + create_payment · refund + + + + + + Single VPS · Docker Compose + + + + + core domain · contract_check.core + + + + + infra · stateful + + + + + Legend + + + Frontend + + + + Backend + + + + Database + + + + Security + + + + Message bus + + + + External + + + +

+ + + + + + + + + +
+ + +
+
+
+
+

Hexagonal boundary

+
+
    +
  • • Telegram bot and web UI are thin HTTP adapters — they never import core.db / s3 / llm / mq
    • +
  • • Nginx is a separate TLS edge layer: all adapter traffic flows through it to the API
    • +
  • • API is the sole writer for user mutations; workers own their stage rows only
    • +
  • • Enforced by tests/unit/test_bot_boundary.py (static AST)
    • +
+
+ +
+
+
+

Screening pipeline

+
+
    +
  • • PDF/DOCX/RTF/CSV/images → text → prescreen (hybrid heuristic + LLM fallback) → LLM analysis → markdown report
    • +
  • • Quorum queues with TTL retry, exponential backoff, and DLQ; refund_credit on terminal failure
    • +
  • • Fine-grained status: queued → extracting → prescreening → ocr → analyzing → done | failed
    • +
+
+ +
+
+
+

Billing & credits

+
+
    +
  • • Reserve-on-enqueue invariant: credit moves on POST /documents, never in a worker
    • +
  • • ЮKassa webhook-driven invoice state machine; worker-billing handles renewals, expiry, reconciliation
    • +
  • • Quota (subscription) consumed before Credits; refund policy D10 with clawback and billing_hold
    • +
+
+ +
+
+
+

Auth surfaces

+
+
    +
  • • User JWT for bot/web/Mini App; service token for adapter-level; X-API-Key for B2B
    • +
  • • Passwordless: passkeys (WebAuthn) + magic links; email/password JWT pair with refresh in Redis
    • +
  • • /admin panel uses HttpOnly cookie + users.role = admin
    • +
+
+
+ +
+ + + + diff --git a/docs/diagrams/contract-check-architecture.visual-check.json b/docs/diagrams/contract-check-architecture.visual-check.json new file mode 100644 index 0000000..a010c63 --- /dev/null +++ b/docs/diagrams/contract-check-architecture.visual-check.json @@ -0,0 +1,60 @@ +{ + "schemaVersion": 1, + "ok": false, + "command": "visual-check", + "evidenceKind": "automated-browser", + "status": "skipped", + "visualReview": "pending", + "artifact": { + "path": "/home/san/PythonProjects/DealDocumentScreening/docs/diagrams/contract-check-architecture.html", + "sha256": "14b219ba1da1f4525e59338b04dadf208ade2b6bdc30eb779875c9842364f0c7", + "bytes": 846404 + }, + "state": { + "detail": "read", + "motion": "still" + }, + "chrome": { + "status": "unavailable", + "executable": null + }, + "diagnostics": [ + { + "code": "viewer/chrome-unavailable", + "severity": "warning", + "message": "Chrome or Chromium is unavailable. Set ARCHIFY_CHROME to its executable path.", + "subject": { + "artifact": "/home/san/PythonProjects/DealDocumentScreening/docs/diagrams/contract-check-architecture.html" + }, + "evidence": { + "executable": null + }, + "supportedFixes": [ + "set ARCHIFY_CHROME to a Chrome or Chromium executable and rerun visual-check" + ] + } + ], + "containment": { + "status": "skipped", + "viewports": [] + }, + "readability": { + "status": "skipped", + "minimumProjectedNodeTextPx": 6, + "viewports": [] + }, + "viewerChrome": { + "status": "skipped", + "viewports": [] + }, + "captures": { + "status": "skipped", + "screenshots": [], + "contactSheet": null + }, + "sidecars": { + "receipt": "contract-check-architecture.visual-check.json", + "contactSheet": "contract-check-architecture.visual-check.html" + }, + "error": "Chrome or Chromium is unavailable. Set ARCHIFY_CHROME to its executable path." +}