half our events just… vanished

We asked Claude to help us design a RabbitMQ setup for our vehicle event pipeline. The suggestion was clean and sensible: one exchange, one queue, one consumer group. It handled our stated requirements perfectly.

We ran it for a week. Then we noticed our analytics were off. Vehicle view events were missing. About half of them.

The setup

Our event pipeline ingests vehicle-related events: views, searches, price changes, inquiries. These feed into analytics, recommendation models, and dealer dashboards. High volume — tens of thousands of events per hour at peak.

I described this to Claude and asked for a RabbitMQ topology. It suggested:

Exchange: vehicle_events (topic)
Queue: events_processing
Binding: vehicle_events → events_processing (routing key: #)
Consumers: 3 instances

Reasonable. One queue, multiple consumers competing for messages, even load distribution. I implemented it, we deployed, life was good.

What vanished and why

The problem wasn’t the queue. It was that we had two fundamentally different consumers pulling from the same queue:

1. Analytics consumer: needs every event, writes to a data warehouse
2. Real-time consumer: needs recent events, updates live dealer dashboards

With competing consumers on a single queue, each message is delivered to exactly one consumer. The analytics consumer and the real-time consumer were splitting the events between them. Analytics was seeing ~50% of events, real-time was seeing the other 50%.

Neither failed. Both appeared healthy. We just had silent data loss.

The architecture Claude didn’t know to ask about

When I described “multiple consumers” I meant “multiple instances for throughput.” What I should have clarified — and what Claude had no way to infer — was whether different consumers needed independent copies of each message.

The correct RabbitMQ topology for our case:

Exchange: vehicle_events (topic)

Queue: analytics_events     ← bound to exchange, 3 consumer instances
Queue: realtime_events      ← bound to exchange, 3 consumer instances

Each message → copied to BOTH queues by the exchange
Each queue → consumed independently

Two queues, same exchange. Each message fan-out to both. Each consumer group gets every event.

What I’d do differently

Before accepting an AI-suggested architecture, I’d write out the data flow requirements explicitly:

• Who needs to receive each message?
• Does each consumer need an independent copy, or are they competing instances?
• What happens if one consumer is slow — does it affect others?

These questions reveal whether you need competing consumers (scale) or independent queues (fan-out). Claude can design either pattern — but you have to tell it which one you need.