Open Source Tales #1: WebSocket Operator

One of the most interesting problems I have worked on was something that is easy in stateless systems and much less pleasant in stateful systems: using horizontal scaling to add more WebSocket servers.

If your application keeps connection state in memory, adding more replicas is not enough. You also need a way to route a message to the replica that owns the target connection.

That was the real problem.

This post is about why application-level messaging workarounds eventually fail, and what a platform-native alternative looks like.

The constraints

Our original environment came with a long list of restrictions:

• we had to respect an internal framework built around model events
• identity and access management (IAM) filtering had to be preserved
• Socket.IO was already part of the stack
• RabbitMQ-based Socket.IO plugins were considered too costly
• Redis plugins were not a good fit for our case
• multi-tab support was required
• we wanted to avoid adding new infrastructure if possible

In other words, this was not a clean-sheet design.

Our WebSocket servers also ran on older Socket.IO versions and barely used the framework's stronger features. At that point, they could almost have been plain WebSocket servers.

The first solution

To scale horizontally, we used Redis publish/subscribe (Pub/Sub) so WebSocket servers could forward messages to each other.

That solved the immediate routing problem, but it also made me realize something important: scaling a stateful WebSocket layer is itself a costly architectural choice.

Once you go down that path, you inherit a set of distributed-systems problems whether you wanted them or not.

The uncomfortable lessons

The pain points were familiar:

• observability became harder
• Redis Pub/Sub introduced delivery and bandwidth concerns
• connection distribution across replicas was weak
• some replicas ran hot while others stayed cold

At some point I started thinking the real question was not "how do I scale this with Redis?" but "what if the platform should help route and rebalance connections?"

Kubernetes, proxies, sidecars, and service discovery started to look more interesting than another application-level messaging layer.

The project idea

A couple of years later I found this article, which described a Kubernetes-based approach to distributed WebSocket routing.

That article was the spark. I had already solved a similar problem under stricter constraints, and they had solved it in a cleaner infrastructure-aware way. That made me wonder how many teams were independently reinventing roughly the same architecture.

That curiosity became websocket-operator, a Kubernetes operator that watches declared configuration and adjusts the running system.

What the operator does

The project is a blueprint for running horizontally scalable WebSocket workloads on Kubernetes without depending on the older Redis Pub/Sub shape of the solution I had used before.

The central idea is simple: route connections and messages through Kubernetes-aware components instead of pushing the whole problem into a broker plugin.

The operator currently revolves around three components.

Load balancer

This load balancer is the user-facing entry point.

It accepts connections and routes them toward proxy sidecars using the same connection-placement logic shared elsewhere in the system.

Proxy sidecar

The sidecar intercepts WebSocket traffic and can proxy a message to another sidecar that actually owns the destination connection.

This is what lets the system route stateful traffic across replicas.

Controller

The controller injects sidecars into workloads marked with the ws.operator/enabled label and manages the Kubernetes-side control loop around that setup.

The longer-term goal is to help with behaviors such as rebalancing, but that part is still project direction, not a claim that every piece is already production-complete.

Why this interested me

This project let me combine two things I wanted at the same time:

• deeper Go and Kubernetes operator work
• a chance to revisit an old distributed-systems problem without the original constraints

That made it more than a coding exercise. It was a way to pressure-test a better architectural direction.

Roadmap

The roadmap is still intentionally practical:

• interchangeable hashing strategies that consistently choose a replica for each connection
• pluggable message-routing logic
• support for broadcast-style delivery

The first versions assume JavaScript Object Notation (JSON) messages, but there is no good reason to stop there. Real systems may want binary payloads, text frames, or custom protocols.

Should you scale your WebSocket layer?

Before building anything like this, ask yourself:

• Is connection count actually the bottleneck, or is it business logic?
• Can you use vertical scaling to give your current instances more processor or memory capacity instead?
• Is your WebSocket traffic evenly distributed, or bursty?
• Do you have a Kubernetes-native platform to lean on?

If your answers suggest that horizontal scaling is truly necessary, then an operator-based approach is worth exploring. If not, you may be solving the wrong problem.

Takeaway

Stateful systems become expensive in ways stateless systems do not. Before scaling a WebSocket architecture, make sure the product really needs it.

If it does, lean on the platform where possible instead of pushing every routing problem into application code.

Sometimes the right lesson from a past system is not how to rebuild it. Sometimes it is how to avoid rebuilding the same constraints.