Section650

Writing

TopicArchitecture
Reading8 min

The Architecture You Don’t Build Is Sometimes Your Best Decision

After more than twenty years building and operating software, I’m much more interested in whether a system needs to exist than whether we know how to build it.

By Ka Lun Chan · Software architecture · Architecture / Engineering judgment / Leadership

Just because we can build it

I enjoy the technical part of architecture. Figuring out how systems fit together, where the boundaries belong, and how something will behave under load is interesting work. It’s easy to keep going once a design starts taking shape.

That enthusiasm needs a check. A new service can be well designed, thoroughly tested, and entirely unnecessary. We can do excellent implementation work on a decision we should have questioned earlier.

My background includes carrier networks and VoIP infrastructure, building and scaling SaaS applications, monoliths, microservices, and AWS. Across those environments, the business needed different things from the technology. A call in progress has constraints that a report someone reads tomorrow morning doesn’t. Treating both as the same kind of architecture problem wastes effort and can make the important requirements harder to meet.

Good architecture solves the business problem with the least complexity necessary to meet its requirements. Reliability, security and recovery belong in those requirements. So does the team’s ability to keep the system running.

Before discussing tools, I want to know who needs the result, how quickly they need it, and what happens when it’s late or wrong. Those answers often remove whole parts of a proposed design.

A reporting request becomes a platform

Consider a hypothetical SaaS team. A customer asks for a dashboard showing yesterday’s activity. The engineers sketch an event pipeline, a separate reporting service, a database optimized for analytics, and an API for the dashboard.

Each choice has a reasonable explanation. Events decouple reporting from the application. A separate database protects transactional workloads. The API gives future clients a clean way to access the data. Put those explanations together, and a daily report has become a new platform.

I’d first ask whether a scheduled job in the existing application could produce the report. Could an indexed query meet the load? Could we precompute the totals and serve them through an existing endpoint? If the data only needs to be ready each morning, streaming every change may buy nothing the customer can use.

That smaller design still needs care. The job needs safe retries, a way to detect stale results, and a limit on how much database work it can consume. We should check the query against representative data and watch its effect on the application. Sharing infrastructure only saves work when it remains safe to share.

The answer could change. Customers might need results within seconds. Reporting queries might start affecting transaction latency. Several independent systems might need the same activity stream and the ability to replay it. Those requirements could justify a separate store or Kafka. They also give us something concrete to evaluate before adding either.

A simpler first version gets the customer useful information sooner and lets the team learn what they actually do with it. Building the whole platform first spends that learning time on infrastructure.

The work that stays after launch

The AWS bill is easy to put in a spreadsheet. The rest of the cost needs someone to ask about it.

A new database brings access policies, backups, restore procedures, schema changes and capacity decisions. A backup that nobody has restored is an assumption about recovery. A second database also raises questions about which copy owns a fact and how we repair disagreement between them.

A new API creates a contract. Someone has to handle authentication, authorization, timeouts, compatibility and error behavior. When a request crosses services, debugging may require correlating logs and traces across them. An operation that used to fit inside one database transaction may now need a recovery process when only half of it succeeds.

Kafka brings its own responsibilities: event schemas, consumer lag, retention, duplicate handling and replay. I’ve worked with Kafka and event-driven systems, and there are good reasons to use them. A durable stream that independent consumers can process at their own pace can be very valuable. But publishing an event doesn’t finish the design. Someone must know how to recover when a consumer falls behind or processes bad data. I discuss those choices in the Kafka tradeoffs article.

On AWS, even managed components need configuration, permissions, monitoring and cost ownership. More infrastructure can mean more network paths to secure, more credentials to manage, and more environments to keep consistent. A managed service reduces some operating work; the application’s failure behavior is still our responsibility.

These obligations accumulate. Testing has to cover more failure modes. Deployments need coordination when contracts change. Future engineers need to understand more of the system before they can change it safely. The business pays through infrastructure expenses and through engineering time that is no longer available for customers.

I want a proposal to explain what recurring work it removes as well as what it adds. A service that isolates an unreliable integration might save enough incident time to justify itself. A service that merely moves existing code behind a network call needs a stronger case.

Architecture has to fit the people operating it

I’ve led small teams and helped grow teams from five engineers to thirty. The amount of attention available for infrastructure changes substantially as a team grows. Architecture needs to account for that.

A five-person team adopting the service layout of a company with hundreds of engineers also inherits work that the larger company may distribute across platform, security and reliability teams. Five people still have customer requests, support and holidays. Being capable of building the system doesn’t mean they have room to operate it indefinitely.

For a small team, a modular monolith is often a sensible starting point. Clear internal modules can organize accounts, billing and reporting while keeping one application deployment. Local calls remain local, and operations within the same database can use its transactions. You can enforce boundaries in code review and automated checks without deploying every boundary separately.

Microservices can earn their place when a workload needs independent scaling, a security requirement calls for isolation, or teams need to release independently around stable domains. Those benefits can outweigh the distributed debugging and deployment work. Team size alone doesn’t settle it; capabilities, operational maturity and the consequences of an outage matter too.

I’ve covered this in more detail in why team size should influence architecture. In a design review, I’d also ask who can operate the proposed system when its main author is away. If the answer is nobody, training and backup coverage belong in the estimate.

Leave room without building everything

Restraint can become an excuse for shortcuts. Putting business rules wherever it’s convenient, letting every module change every table, and ignoring data ownership will make a small application difficult to change too.

I’m willing to spend time on boundaries before there is a reason to split deployments. Give a module a clear responsibility. Keep its interface small. Decide where authoritative data lives. Make important constraints explicit in the data model. Test the behavior that another part of the application depends on.

In the reporting example, the calculation can live behind an internal interface. The scheduler calls it, and the dashboard reads a defined result. That is enough separation to let us change how reports are produced later without scattering reporting logic through the product.

It doesn’t make a future extraction free. Moving into another process changes latency, failure handling and data consistency. Data may need migration or backfilling. Good boundaries reduce the work we have to untangle; they don’t eliminate the work of becoming distributed.

I also want to distinguish a deliberate limit from unexamined technical debt. “This job can run overnight, and we’ll revisit it if it misses the agreed completion window” is a useful decision. “We’ll fix reporting later” leaves the next engineer guessing what was acceptable and why.

There are times to build ahead. A confirmed customer requirement, a lengthy migration, or a hard recovery target may make waiting expensive. Future flexibility is worth paying for when we can explain what we’re preserving and how likely we are to need it.

Make the decision explicit

Experienced engineers should be able to challenge complexity without shutting down the person proposing it. Usually there’s a concern underneath the proposed technology. I want to understand that concern before deciding the design is too much.

These are the questions I find useful:

  • What problem are we solving, and what does success look like?
  • What’s the simplest solution that meets the requirements?
  • What happens if we don’t build this?
  • What operational responsibilities are we introducing, and who will own them?
  • Can our current architecture handle this with a smaller change?
  • What evidence would justify a more complex solution later?

Then write down the decision. For our hypothetical report, I’d record the freshness requirement, the expected load, why we’re using the existing database, and the signs that would trigger a review. If reports miss their completion window or harm application performance, we have a reason to return to the design. Whoever inherits the system can see the reasoning instead of assuming we forgot to consider alternatives.

Saying no also needs to account for the cost of doing nothing. If an existing system is causing repeated outages or blocking delivery, keeping it unchanged may be the expensive choice. The questions should expose that too.

This is part of engineering leadership: making the ongoing commitment visible while there’s still a choice. The team needs room to say that an interesting design doesn’t yet solve a problem worth its cost.

What does the technology need to do?

Early in my career, I was interested in what technology could do. Over time, I’ve become more interested in what technology actually needs to do.

I still enjoy building systems. I also want the engineers who inherit them to spend their time improving the product, with an architecture they can understand and operate. Every component we add needs to earn the attention it will keep asking for.

Sometimes the best architecture is the one we decided we didn’t need to build.