Commit Graph

6 Commits

Author SHA1 Message Date
3052f7b799 Xendit webhook: metadata.app routing + survival audit log + rolling fallback file
Every Xendit invoice now carries metadata: { app: 'halobestie_v2' } so an
external webhook router (no DB access) can fan out v1/v2 traffic purely off
the echoed payload.

Every inbound webhook lands in a new webhook_logs table BEFORE auth or
business logic, so a forensic row survives 401/409/unknown/exception paths.
Primary fields are parsed as columns; raw_body keeps the full payload
verbatim. The handler captures outcome in closure-scoped vars and stamps
http_status/processing_result/processing_error in a single update before
the lone reply.send() — Fastify flushes reply.send() immediately, which
defeated the original finally-block stamp.

A non-UUID external_id no longer crashes the Postgres cast; it ACKs with
ignored_non_uuid_external_id so Xendit stops retrying legacy old-app IDs.

When the DB log itself fails, an optional rolling JSONL file sink absorbs
the event. Disabled by default — opt in via XENDIT_WEBHOOK_FALLBACK_ENABLED.
Naming: <NAME>-YYYY-MM-DD.jsonl in XENDIT_WEBHOOK_FALLBACK_DIR (default
./logs), basename XENDIT_WEBHOOK_FALLBACK_NAME (default
xendit-webhook-fallback). No stdout fallback by design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:09:14 +08:00
553dbac52f Phase 6: Valkey availability mirror — move read path off Postgres
Mitra-availability state (online flag, deactivated flag, per-mitra session
count, heartbeat liveness) mirrored into Valkey so the customer beacon
+ pairing blast + dashboard counts no longer hit Postgres on the hot path.
Postgres remains the durable source of truth; Valkey state is fully
derivable via seedFromPostgres on startup + reconnect.

Schema
- mitras:online           SET    — mirror of is_online
- mitras:deactivated      SET    — mirror of is_active=false
- mitra:capacity:<id>     STRING — active+pending_payment session count
- mitra💓<id>    STRING — ISO timestamp of last ping
- availability:snapshot   JSON   — beacon cache, TTL 10s, cluster-shared

Write paths (Postgres first, best-effort Valkey)
- setOnline/setOffline mirror SADD/SREM + heartbeat SET/DEL
- updateMitraStatus mirrors mitras:deactivated AND revokes auth_sessions
  on deactivate (bounds the "ghost online" window to access-token TTL)
- heartbeat is Valkey-only on the hot path; the per-ping Postgres UPDATE
  on last_heartbeat_at is eliminated (was 1,200 ops/min at prod scale)
- chat_session lifecycle (accept/end/reroute/extension/expiry) calls
  recomputeCapacityForMitra after each UPDATE — derive-from-truth avoids
  the bookkeeping risk of per-transition INCR/DECR

Read paths (Valkey-first, Postgres fallback on Valkey error)
- isMitraReachable: SISMEMBER mitras:online + heartbeat freshness
- findAvailableMitras: SDIFF + pipelined GETs, filter by capacity + heartbeat
- countAvailableMitrasFromCache: Valkey-driven, cached cluster-wide 10s TTL
- dashboard online count: SCARD
- Each reader wraps Valkey ops in try/catch → Postgres fallback on outage

Heartbeat path on /api/mitra/status/heartbeat
- resolveMitra preHandler replaced with heartbeatGuard: SISMEMBER on
  mitras:deactivated (~0 DB hits per ping). Falls back to full DB
  resolveMitra if Valkey is unreachable so a Valkey outage doesn't
  silently accept heartbeats from deactivated mitras.

Three sweeps, env-configurable cadences
- MITRA_AUTO_OFFLINE_SWEEP_SECONDS (30) — Valkey-driven stale detection
- HEARTBEAT_MIRROR_INTERVAL_SECONDS (60) — batched UPSERT writes
  Valkey timestamps to Postgres last_heartbeat_at via UNNEST (1 statement
  per cycle, idempotent across instances)
- VALKEY_ONLINE_MIRROR_SWEEP_SECONDS (300) — periodic reseed heals drift

Startup
- restoreActiveTimers → seedFromPostgres → bind listeners
- onValkeyReady re-runs the seed on every reconnect (cold start + reseed
  on Valkey restart, no manual intervention)

Failure semantics
- Read fallback: every Valkey read wrapped, falls back to existing
  Postgres JOIN query — system stays correct during Valkey outage,
  performance degrades not breaks
- Write best-effort: Postgres write commits before Valkey is touched;
  Valkey errors log + continue; reconciliation sweep heals drift
- Auto-offline sweep aborts entirely on Valkey error (does NOT mass-
  offline via Postgres scan during Valkey hiccup)

Tests
- New: 32 integration tests in mitra-status.valkey-mirror.test.js
  covering seed, write-through, fallbacks, capacity lifecycle,
  auto-offline sweep, heartbeat mirror, deactivation flow, beacon cache
- Updated: fixtures.js seeds Valkey alongside Postgres when isOnline=true
- Updated: helpers/db.js resetDb also flushes test Valkey
- Fixed 2 pre-existing session-timer flakes (string IDs failed uuid
  parse; vi.advanceTimersByTimeAsync raced real Postgres I/O)
- All 124/124 backend tests pass (was 90/92)

Docs
- requirement/valkey-online-mirror-plan.md — canonical plan
- requirement/valkey-online-mirror-testing.md — manual E2E checklist
- requirement/deployment.md — infra + Valkey persistence guidance for
  prod (Memorystore Standard tier recommended; migration from
  self-hosted Valkey is zero-downtime via reseed-from-Postgres)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 18:07:55 +08:00
3fff4b1c6e Phase 5 Xendit: Stages 1-7 (XENDIT_ENABLED=false; Stage 8 pending creds)
Backend
- payment_sessions → payment_requests rename across DB schema + 29 files
- payment.service.js becomes product-agnostic owner: EventEmitter +
  Xendit wrapper + requestPayment / confirmPayment public API; legacy
  aliases retained for existing chat callers
- Webhook handler at POST /api/shared/payment/webhooks/xendit, with
  constant-time token verification (8 vitest cases)
- Server-driven pairing: payment.service emits
  payment_request.confirmed → pairing subscriber starts the blast.
  Legacy POST /chat/request still works during the cutover.
- Reconciliation sweeper extended (re-emits events for confirmed rows
  with no chat session)
- SIGTERM drain + startup reconciliation pass in server.js

Customer app
- waiting_payment_screen opens xendit_invoice_url via
  LaunchMode.inAppBrowserView
- searching / no-bestie / targeted-waiting / pairing-notifier updated
  to consume the new payment_request_id contract
- pending_payments_provider + bestie-unavailable dialog migrated

Dev / testing
- XENDIT_ENABLED=false is the safe default; .env.example documents the
  four new vars
- backend/.dev/xendit-fake-webhook.sh exercises the handler without
  ngrok
- 90/92 backend tests pass (two pre-existing session-timer flakes,
  unrelated); client_app analyzer clean
- requirement/phase5-xendit-plan.md is the canonical reference

Stage 8 (live E2E) blocked on Xendit test-mode keys. The dashboard's
single-webhook-URL constraint will be worked around via a self-poll
script next session.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 12:52:33 +08:00
f860ab6c85 Phase 3.4: backend self-managed auth cutover
All backend auth now goes through our own token service — Firebase Auth
dependency is fully removed from auth paths. FCM (firebase-admin messaging)
is still used for push.

Schema:
- auth_sessions (multi-device refresh tokens, bcrypt-hashed)
- otp_requests (Fazpass reference + rate-limit history)
- customers.email + google_sub + apple_sub (social identity)
- control_center_users.password_hash + failed_login_count + lockout_until
- firebase_uid columns made nullable (drop in later cleanup migration)
- 6 new app_config keys for OTP + CC lockout tuning

Services:
- password.service.js — bcrypt cost 12 + complexity (min 8, digit + upper +
  lower)
- token.service.js — JWT HS256 access (1h) + opaque refresh (30d, bcrypt-
  hashed, rotated on use); session_id claim pre-wires future Valkey-based
  instant revocation; revokeSession + revokeAllSessionsForUser helpers
- social-identity.service.js — Google via google-auth-library, Apple via
  jwks-rsa + jsonwebtoken
- otp.service.js — Fazpass stub (generates locally, logs the code) clearly
  marked for replacement once real API docs arrive; rate-limit + resend
  cooldown + verify-attempts all configurable via app_config
- auth.service.js — orchestrator: signInAnonymous, completeCustomer/Mitra-
  PhoneSignIn, signInWithGoogle, signInWithApple, signInCcUser, refresh,
  logout; reject-on-existing for identity conflicts
- cc-user.service.js — email+password helpers + lockout counters

Routes & middleware:
- authenticate middleware now verifies our JWT and attaches
  request.auth = { userType, userId, sessionId }
- WebSocket handshake verifies our JWT (no more Firebase lookup)
- All existing routes updated to use request.auth.userId instead of
  request.firebaseUser.uid
- New public routes:
    /api/shared/auth/anonymous /refresh /logout
    /api/client/auth/otp/request /otp/verify /google /apple /me /profile
    /api/mitra/auth/otp/request /otp/verify /me
- New internal routes:
    /internal/auth/login /refresh /logout /me (httpOnly cookie refresh)
    /internal/control-center-users (accepts plain password, bcrypt-hashed)
    /internal/control-center-users/me/password (self-service change)
    /internal/control-center-users/:id/password (admin forced reset)
- Deleted legacy customer.routes.js (anonymous + link handled by auth now)
- app.internal.js: @fastify/cookie + CORS credentials for CC httpOnly cookie

Config:
- AUTH_JWT_SECRET + ACCESS_TOKEN_TTL_SECONDS + REFRESH_TOKEN_TTL_DAYS env
- FAZPASS_* env vars (TBD until real API docs)
- GOOGLE_OAUTH_CLIENT_IDS, APPLE_SERVICES_ID/TEAM_ID/KEY_ID/PRIVATE_KEY
- ADMIN_EMAIL + ADMIN_PASSWORD for seed
- CC_ORIGIN for internal-app CORS origin allowlist

Dependencies:
- Added: bcrypt, jsonwebtoken, jwks-rsa, google-auth-library, @fastify/cookie
- Kept: firebase-admin (messaging only)

Still outstanding: Fazpass API integration (stub in place), Apple Developer
prereqs for end-to-end iOS testing, client_app/mitra_app/control_center auth
flow rewrites.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 11:43:25 +08:00
d668112edd Phase 2 scaffold: mitra online status & pairing logic
Add mitra online/offline status with heartbeat-based auto-offline,
customer-mitra pairing via Valkey pub/sub blast, session management,
and control center dashboard with real-time stats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 23:17:49 +08:00
a7a2a32d27 Phase 1 scaffold: auth for all apps
- Backend: Fastify with two listeners (public + internal), routes, services, DB migration + seed
- client_app: Flutter with BLoC, all auth screens (welcome, display name, register, OTP, force-register)
- mitra_app: Flutter with BLoC, OTP-only login
- control_center: React + Vite, email/password login, mitra/user management, anonymity settings
- Docs: phase1 plan, API contract, client app mockup
- CLAUDE.md and shared memory for all subprojects

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 10:08:42 +08:00