Skip to main content

Building a non-blocking service in a blocking world

In past year or so I've been watching dozens of tech talks about reactive architectures. Most of the time I have felt inspired by them but could not quite get a grasp on what a reactive architecture is really like. Sure, the list of gained benefits such like 100x increase in response times is convincing but not having the experienced that myself, it all seems a bit vague. I need something concrete.

The funny thing is that for the last one and a half years I have been working on a message-driven, non-blocking component (service). The technology stack is at least partially modern, with AngularJS (Some might object, I know...), Vaadin with websocket push as the only server-client communication method and Akka. There is no Java EE or any heavy platform used but rather a customized version of Karaf OSGi container where we run the software as small modules. The UI is not locked at any point, the user can navigate at any time since there are no HTTP requests. Every end-user action results either to a command or a query starting from such user actions as a button click or a navigation event -> websocket -> Akka message -> some other service.

Here's the thing though. This "some other service" can be one of many other components which make up the system. Unfortunately the service is always accessed via SOAP or REST. This is where the nice non-blocking flow will terminate. While inside our component, it does not matter who will answer to the actor who has sent the command or query (we don't use ask pattern), the answering party is never the service which has actually executed the action but rather the SOAP or REST client running inside our component. That is then where we block.

There is certainly value that there is blocking only on the "edges" of the component. However here I come back to my original point of me not fully embracing the reactive architecture. It is not enough that there is one part (component) attempting to be reactive. In our case since it is responsible of the UI, it creates the illusion of a non-blocking world.

Recently three components have been added to the system, all of which handle streams of data. The most interesting one is Apache Kafka. The drive seems to be towards modernizing the architecture and there is hope for me to getting some valuable experience on a system-wide reactive design.

Comments

Popular posts from this blog

I'm not a passionate developer

A family friend of mine is an airlane pilot. A dream job for most, right? As a child, I certainly thought so. Now that I can have grown-up talks with him, I have discovered a more accurate description of his profession. He says that the truth about the job is that it is boring. To me, that is not that surprising. Airplanes are cool and all, but when you are in the middle of the Atlantic sitting next to the colleague you have been talking to past five years, how stimulating can that be? When he says the job is boring, it is not a bad kind of boring. It is a very specific boring. The "boring" you would want as a passenger. Uneventful.  Yet, he loves his job. According to him, an experienced pilot is most pleased when each and every tiny thing in the flight plan - goes according to plan. Passengers in the cabin of an expert pilot sit in the comfort of not even noticing who is flying. As someone employed in a field where being boring is not exactly in high demand, this sounds pro...

PydanticAI + evals + LiteLLM pipeline

I gave a tech talk at a Python meetup titled "Overengineering an LLM pipeline". It's based on my experiences of building production-grade stuff with LLMs I'm not sure how overengineered it actually turned out. Experimental would be a better term as it is using PydanticAI graphs library, which is in its very early stages as of writing this, although arguably already better than some of the pipeline libraries. Anyway, here is a link to it. It is a CLI poker app where you play one hand against an LLM. The LLM (theoretically) gets better with a self-correcting mechanism based on the evaluation score from another LLM. It uses the annotated past games as an additional context to potentially improve its decision-making. https://github.com/juho-y/archipylago-poker

Careful with externalTrafficPolicy

On a project I am working on is hosted in an EKS cluster with the NGINX ingress controller (the one maintained by Kubernetes). It is deployed using it's official official Helm chart, which I realized, after a lengthy debugging session, was a mistake. The initial setup I aimed to improve had several flaws. Firstly, we were using the AWS Classic Load Balancer in front of the nginx ingress in the cluster, which has been deprecated for some time (years?). Continuing to use it makes little sense to us. The second issue was that we were only running one(!) nginx pod, which is quite sketchy since the exposed web services had essentially no high availability.  I switched to the Network Load Balancer (NLB), which was straightforward - I just needed to change the ingress-nginx service annotation to specify the load balancer type as NLB: service.beta.kubernetes.io/aws-load-balancer-type: nlb However, increasing the replica count turned out to be tricky. When I bumped it up to two, I began to ...