How to beat uncertainty when designing systems

In 2022, I got a task to design a new subsystem in the microservice app.

Excited? Hell yeah.

But also scared.

Since I’m not an experienced architect.

And this is the first time I will design something huge like that.

So I started like any normal senior developer would:

• Googling different ways to build it,
• Reading books that mention similar design problems,
• And occasionally biting myself, why I accepted the challenge in the first place.

There were parts that I knew how to design. There were parts I wasn’t sure how to design and implement.

But instead of guessing those unknown parts and hoping for the best, what we did was perform “Spikes.”

What are spikes?

A spike is a time-boxed task where you:

• Try to explore potential solutions.
• Gather additional information about technology, pattern, or framework.
• Reduce uncertainty about a specific implementation by creating a separate PR with a potential solution.

2 important characteristics to mention are:

• Time-boxed - You set aside a fixed duration to spend on investigation and research. This prevents spikes from becoming open-ended research and doesn’t prolong the whole project duration.
• No production-grade code - The code you write during research doesn’t have to be fully optimized, structured in a production-grade way, or have tests.

The purpose is speed. You want to find out new information as soon as possible. So you can make a more informed decision.

Also, the best practice is to later throw away the code you write on the spike PR.

Here are some examples of spikes you might perform:

• Evaluating email providers - Should you use SendGrid, Mailjet, Resend, or a self-hosted SMTP server? Spin up a quick spike project to send a few test emails, measure deliverability, and compare how much each tool is developer-friendly.
• Comparing the performance of Entity Framework vs Dapper for bulk inserts - Use a spike to benchmark how EF Core and Dapper handle inserting thousands of records. Measure execution time, memory usage, and ease of implementation to decide which is better suited for your use case.
• Evaluating background job libraries like Hangfire vs Quartz.NET - Research both libraries to compare their setup complexity, scheduling flexibility, dashboard features, and how well they integrate with your existing .NET stack.

Takeaway:

When you’re unsure how to implement a part of your system, don’t guess.

Spike it.

Spikes are like cheat codes for reducing uncertainty.

And you’ll look a lot more like an architect when you use them.