Distillation

Date: July 30, 2026

• 🟡 (15:50) User asked: their processor uses the event_accepted signal to forward basic event info to another service, and the event volume has dropped dramatically in the past several weeks. Asked for possible reasons
• 🟡 (15:50) Assistant identified d7fcb07bc42 (Jul 21, 2026) — reprocess_only_events_not_in_nodestore flag — as the most likely culprit. Reasoning: adds new early-return at line 196 that skips event_accepted.send_robust() when the event is already in nodestore. If this flag is enabled on main ingest consumers (not just reprocess tools), it would cause a volume drop matching the rollout timeline
• 🟡 (15:50) Assistant suggested verification command: grep -r "reprocess-only-events-not-in-nodestore" --include="*.py" --include="*.yaml" --include="*.toml" to check consumer config/launch flags
• 🟡 (15:50) Assistant identified 5ae4736ff76 (Jun 15, 2026) — inline_save_event=True in taskbroker passthrough mode (#117580) — as second-likeliest cause. Reasoning: when preprocess_event runs inline, exceptions caught at line 317 and re-raised as Retriable cause retries; if retries exhausted, message goes to DLQ and event_accepted is never emitted. Before this PR, those exceptions were absorbed by Celery
• 🟡 (15:50) Assistant listed less-likely causes worth checking:
  • User's own @receiver(event_accepted) handler — if "another service" is rate-limiting/down, send_robust swallows receiver exception but signal still fires (returned in responses). Suggested instrumenting receiver to log failures
  • Organization.DoesNotExist (line 246) — if source projects deleted, events silently drop. Look for Organization does not exist warnings
  • Quota drops upstream in Relay before message reaches consumer — would affect all consumers, not just this signal
• 🟡 (15:50) Assistant noted relevant metrics to investigate: metrics.timer("ingest_consumer.process_event") and kafka_consumer retry/DLQ metrics in dashboards would show preprocess_event failures
• 🔴 [debugging-investigation] (15:50) Assistant ranked candidates by likelihood: (1) reprocess_only_events_not_in_nodestore flag rollout, (2) inline_save_event taskbroker passthrough failures, (3) receiver-side failures, (4) Organization.DoesNotExist drops, (5) upstream Relay quota. Primary recommendation: confirm reprocess flag rollout first as most surgical match for "drop in past several weeks"