The Three Pillars of Modern Enterprise Architecture

Welcome to Issue #008 of the /dev/stdout newsletter. This time, I wanted to highlight three foundational pillars of modern enterprise architecture.

Table of Contents

1. Kubernetes API

2. Kafka Protocol

3. Zero Trust Security Model

1. Kubernetes API

Kubernetes API is becoming a standard when it comes to managing software infrastructure. Kubernetes API can be used in various ways. The most common way of using it is of course with Kubernetes itself. Managed Kubernetes services are available from all the major cloud providers and beyond. With Kubernetes, the Kubernetes API can be extended further using operators such as Cert Manager, Nginx Ingress Controller, Apache Druid Operator, or Elasticsearch (Elastic Cloud for Kubernetes) Operator. These operators add new Custom Resource Definitions alongside default resources like Deployment, StatefulSet, Pod, Namespace, etc.

However, Kubernetes API is not only used with Kubernetes or extending Kubernetes’ default functionality with Operators. Kubernetes API can also be used to deploy other cloud infrastructure outside of Kubernetes, like with using Crossplane.

A third way to utilize Kubernetes API emerging in the form of novel cloud providers such as fly.io that natively support Kubernetes API for building your infra on their services. All that without having to have a single Kubernetes cluster running.

Kubernetes is complex by design as it serves as the baseline for building more easily manageable platforms. It is not the developer platform you're looking for; rather, it provides the building blocks to create one.

Azure's new AKS Automatic offering is an interesting initiative aimed at reducing the operational burden of using, even the traditional managed, Kubernetes. They claim it's an immediately production-grade way to run Kubernetes, and it looks quite promising!

2. Kafka Protocol

Apache Kafka has become the standard for streaming data infrastructure these days. And by meaning Apache Kafka, I do not only mean the Apache Kafka software and the cluster, but the protocol.

Why is the protocol interesting on its own? Well, that’s because there are many other implementations of the Kafka protocol besides Apache Kafka. The hot alternative currently is RedPanda, which is speculated to be bought by Snowflake soon. Another relatively new one worth mentioning is WarpStream, which saves a lot on operational costs compared to Kafka by running it on top of Object Storage infra. Saving a ton of operational costs, but with a tradeoff of higher latency compared to traditional Kafka. WarpStream was also recently acquired by Confluent.

Given how fast Apache Kafka evolves and how active the alternative implementations and the business around them are, it seems that investing in using the Kafka Protocol in your organization is a solid decision going forward, regardless of the flavor of implementation you’re about to use.

3. Zero Trust Security Model

Many companies that handle sensitive data or are otherwise active in critical domains have bolstered their security posture for a long time already. What has changed during recent years is that the cybersecurity space is getting more and more risky for mid-sized and small companies too. Even companies that do not work with extra sensitive data or otherwise have felt that they’re not interesting enough for cyber perpetrators are acknowledging its importance.

Given that cybersecurity is a socio-technical domain, a good rule of thumb is that policies that are simple to articulate work best. When the premise is straightforward, everyone in the organization can make decisions, both big and small, that align with the policy.

Although I am not a security professional, I have collaborated with companies in high-security domains and possess a solid foundation in the topic. Based on my experience one solid policy is the Zero Trust Security Model. From a technical point of view, the policy assumes no implicit trust in any entity, whether inside or outside the network. Every access request is thoroughly vetted before granting access to resources. This model includes strict identity verification, continuous monitoring, and least-privilege access principles.

One of the great advantages of the Zero Trust Security Model is its perfect alignment with initiatives such as the NIS2 Directive and its seamless compatibility with the ISO27001 certificate.

I’m currently working or have recently worked with companies that aim for certain security certifications, are planning to utilize Kafka in their modern architecture, and are building developer platforms on top of Kubernetes.

Would you like to hear more about how to apply these in your organization?

Thanks for reading and have a great week ahead!

P.S. It’s still tough to keep the monthly writing habit, but I’ll get there eventually.

Best,
Pyry

P.P.S. I got a significant part of my writing energy by listening to the song below. Go ahead and listen to find out why!

_{*You might wonder what the song recommendation has to do with anything. Well, as I listen to a lot of music while I write, be it newsletter or code, I simply wanted to share some of that with you too. I hope you like them!}