The vast majority of web applications should start life as a Majestic Monolith: A single codebase that does everything the application needs to do. This is in contrast to a constellation of services, whether micro or macro, that tries to carve up the application into little islands each doing a piece of the overall work.
And the vast majority of web applications will continue to be served well by The Majestic Monolith for their entire lifespan. The limits upon which this pattern is constrained are high. Much higher than most people like to imagine when they fantasize about being capital-a Architects.
But. Even so, there may well come a day when The Majestic Monolith needs a little help. Maybe you’re dealing with very large teams that constantly have people tripping over each other (although, bear in mind that many very large organizations use the monorepo pattern!). Or you end up having performance or availability issues under extreme load that can’t be resolved easily within the confines of The Majestic Monolith’s technology choices. Your first instinct should be to improve the Majestic Monolith until it can cope, but, having done that and failed, you may look to the next step.
That next step is The Citadel, which keeps the Majestic Monolith at the center, but supports it with a set of Outposts, each extracting a small subset of application responsibilities. The Outposts are there to allow the Majestic Monolith to offload a particular slice of divergent behavior, either for organizational or performance or implementation reasons.
One example at Basecamp of this pattern was our old chat application Campfire. It was built back in 2005, when Ajax and other JavaScript techniques were still novel, so it was based on polling rather than the persistent connections modern chat apps use these days. That meant that every client connected to the system would trigger a request every three seconds asking “are there any new messages for me?”. The vast majority of these requests would reply “no, there’s not”, but to give that answer, you still had to authenticate the request, query the database, all that jazz.
This service had vastly different performance characteristics from the rest of the application. At any given time, it would be something like 99% of all requests. It was also a really simple system. In Ruby, it was barely 20 lines long, if I remember correctly. In other words, a perfect candidate for an Outpost!
So an Outpost we made. Over the years, it became a hobby to rewrite this Outpost in every high-performance programming language under the sun, because it could usually be done in a few hundred lines of code, regardless of the language. So we wrote it in C, C++, Go, Erlang, and I’m probably forgetting a few others.
But it was clearly an Outpost! The rest of the application continued as a Majestic Monolith built in Ruby on Rails. We didn’t try to carve the entire app up into little services, each written in a different language. No, we just extracted a single Outpost. That’s a Citadel setup.
As more and more people come to realize that the chase for microservices ended in a blind alley, the pendulum is going to swing back. The Majestic Monolith is here waiting for microservice refugees. And The Citadel is there to give peace of mind that the pattern will stretch, if they ever do hit that jackpot of becoming a mega-scale app.
Do the main App talk to the outpust? If so, how are you modeling it in rails way? Service model?
Michał, for the Campfire example, we did integration through the database. So the poller just fetched directly from the monolith’s database. No additional chatting needed.
But for, say, HEY, we have a postfix intake system that does POST against a monolith controller.
So I’d say both works.
Thanks David. However, this does clarify this completely for me.
Is the intake system also a full Rails app? I’m imaging it is only used to interact with the mailbox, but it is also quite complex. If so: where exactly would you propose the HTTP request should reside?
Mixing database-based model with HTTP can lead to random problems. So, I’m guessing a service… something would be called from a callback. But what would you propose that service something should be? Some Model that would have
“`
def send_to_hey
Net::HTTP::Post.net….
end
“`
?
@DHH do you store emails in a mysql database instead of UNIX folders (like cur, new, tmp)? why does exactly postfix do POST requests?
Michał, for intake, we post a raw email onto a Action Mailbox, which then determines whether to record that or reject it. But all that lives within the monolith. The HTTP request is dead-simple and resides in a shell script called by postfix.
Nick, postfix receives the inbound email, and can buffer it on disk (if the app is down), then posts to the app for processing. We store the emails themselves in encrypted cloud storage.
How can I get to the online members
@DHH
Has your team tried Phoenix/Elixir as a past Campfire rewrite. If so, I’m curious to hear your thoughts in the framework & language.
No, we’ve retired Campfire and ye ol’ faithful poller. Basecamp 3 has Campfire built-in, but it uses Action Cable as part of the Majestic Monolith, and we don’t need any Outposts to make that scalable.
Your video resources in this article are superb.
This is a sensible and pragmatic pattern, I’m just glad somebody gave it a catchy name.
We’ve followed the same pattern for years on our chat-focused product and we’ve managed to scale it well.
Thanks yall
Thanks for the video XD
I’ve never seen someone articulate the reality of software development than you do. Every point, article, thought and tweet you dish out is a goldmine. Just wanted to say thank you for the knowledge and inspiration you share.
I love your analysis and it fits inline with other philosophies like YAGNI and KISS and Getting Real. Instead of trying to predict the future and build things because we may need them (very hard and we’ll likely screw it up), we wait and build things when the problem to solve becomes crystal clear (much easier and chance of success is high).
This post is great. Thank you.
However, the link to the video “microservice refugees” ….. Thank you. That one is awesome!
Super pragmatic. Love it.
Right here is the right blog for anyone who really wants to find out about this topic. You realize so much its almost hard to argue with you (not that I actually will need to…HaHa). You certainly put a new spin on a subject that’s been discussed for decades. Excellent stuff, just great!
https://coralisland.wiki/index.php?title=User:BettinaSutton25