Fear, shark attacks, and “Will it scale?” 02 Oct 2006
19 comments Latest by elnate
People like scary movies. And roller coasters. We like to be afraid. We get off on it. It’s one of those primal emotions that titillates us and gets our blood flowing.
Fear clouds our judgement though. Our fear radars tune in to things that have a slim-to-none chance of ever happening. In fact, what we fear is often quite different than what’s actually threatening us.
Terror, disease, and sharks
Look at the widespread fear over terrorism. Politicians and newspapers know fear gets votes and sells papers. But when you read something like “Don’t Be Terrorized,” you realize the level of fear is way out of proportion to the actual danger. Your risk of dying in a plausible terrorist attack is actually much lower than your risk of dying in a car accident, by walking across the street, by drowning, in a fire, by falling, or by being murdered. But CNN and Fox realize viewers want to hear about suitcase bombs, not seat belts, so that’s what we get.
Or look at the anxiety surrounding unusual diseases that come along: bird flu, anthrax, smallpox, West Nile virus, SARS, etc. Scary! But the fear just doesn’t match the risk (see “Alive and well: The fear epidemic”). We would be far better off focusing on greater threats like AIDS, tuberculosis, and malaria. But those diseases don’t tweak our fear radars the same way. Been there, done that.
And don’t even get me started on shark attacks.
Scaling and mental stiffies
Fear is part of business and technology too. I was reminded of this when reading Chris Petrilli’s views on scaleability. His argument: You aren't Google so don't worry about fantasy scaling issues.
“You are not going to become that popular, so don’t sweat it. You can sweat it if you get that level of success…You have to be able to move fast enough to get to the position of popularity, unless you truly have a one-of-a-kind idea, for that sort of success. Seriously, this argument is built on the mental masturbation of ‘what if I was Google?’ But you aren’t, so just stop. Quit sweating the problems that give you a mental stiffy, and work on the problems that are truly hard: ease-of-use, process, transparency, discoverability and the whole use experience.”
There are certain problems that give us, as Chris puts it, a mental stiffy. “Will it scale?” is one of those problems. It’s a shark attack fear. “It’s possible my site could be the next MySpace. It could happen, right?!”
Well, yeah, it’s possible. But not likely. Very not likely.
We gave a similar view in “Scale Later” (PDF), an essay in Getting Real:
The truth is the overwhelming majority of web apps are never going to reach that stage. And even if you do start to get overloaded it’s usually not an all-or-nothing issue. You’ll have time to adjust and respond to the problem. Plus, you’ll have more real-world data and benchmarks after you launch which you can use to figure out the areas that need to be addressed.
Allocate your fear properly
When it comes to building a web app, some things create more fear than they should…
Fear: It won’t scale
Truth: You’re not going to be Google overnight.
Fear: Too many bugs
Truth: As long as they don’t wipe the database, you can live with most bugs for a while.
Fear: Too few features
Truth: You can always add features later.
Fear: Never go down
Truth: Once-in-a-while downtime won’t scare people away.
Fear: It’s too simple
Truth: Simple solutions are fine if they get the job done.
Fear: They’ll copy us
Truth: It’s about the execution, not the idea.
Fear: We must sound serious
Truth: Trying to sound serious all the time makes you bland and unremarkable. It’s ok to be playful and have personality.
So get a grip. Yes, these things matter. But some people get hysterical over these issues before they even deserve to be on the radar. If you fear one of the above issues, make sure it’s due to a genuine risk and not just something that’s giving you a mental stiffy. Otherwise you may be wasting time chasing phantom problems.
The flip-side is that you need to recognize genuine threats — even if they’re “boring” issues. Things like these are a lot more likely to actually cripple your chances of success:
Taking forever to launch
Running out of money
Not solving a real problem
Designing a confusing UI
Obsessing over the wrong things
Trying to do too much at once
Never ignore real here-and-now threats in order to focus on maybe-in-the-future threats.
Advance!
We all know FDR’s famous line “The only thing we have to fear is fear itself.” But the part that followed those words is often overlooked: “Nameless, unreasoning, unjustified terror which paralyzes needed efforts to convert retreat into advance.”
Don’t let unjustified fears paralyze you. Advance!
19 comments so far (Jump to latest)
Eloy Anzola 02 Oct 06
…back to the sharks… :-)
From a recent ‘rands in response’ post:
“Once I’m good and lost in that fear, the impossible task, I’m no longer thinking abut getting the task done, I’m thinking about the fear.”
“My advice is: START.”
http://www.randsinrepose.com/archives/2006/09/25/trickle_theory.html
jonezy 02 Oct 06
i think this post offers some very reasonable and sound advice. what i see as a problem affecting a lot of companies today is the assumption that they know what there users and customers want.
they assume people will be angry about downtime and leave
they assume users can figure it out if they can
they assume all users are stupid and should be treated in kind.
i wonder do any of these companies take the time to ask the people that use there products these very questions?
Anonymous Coward 02 Oct 06
As for Scaling Issues, I use MT on a few sites and it scales pretty well. I use it on an educational site as a content management system (as opposed to ‘just a blog’) and it probably gets as many hits as this one and no problem (all the while acting dynamic).
Its all in the execution. I know others have pointed out exactly why MT doesn’t scale in this instance, but we’ve heard no word that 37 wants anything to do anything except MT doesn’t scale — when quite a few of us use it every day in ‘enterprise ready’ situations.
Beyond this, it almost reminds me of the new post Gruber just put up over at Daring Fireball where he talks about marketting for the sake of marketting. There is no need to brand the blog as a RoR blog, nor any reason to disparage another application, but it almost sounds like there is a product coming up from 37 that will compete for the mere reason to compete.
http://daringfireball.net/2006/10/brand_new
Is it marketting or is there a real need? Sometimes I can’t tell if this site is doing a better job at provoking developers into making better products or simply scaring us into using their products (and they are mighty fine products).
Dave Chiu 02 Oct 06
I think there are a couple of things going on here beyond simply a blas� attitude.
For one, I think the typically-long time scale inherent to a disease such as AIDS distorts our perception of threat. Because it can take years to manifest, the incubation period and progression of HIV/AIDS makes it appear less urgent than something like terrorism.
This time scale issue also wraps into the “spectacular-ness” of a given situation. Ebola, while still deadly, is less of a long-term concern than AIDS. However, the short-term, spectacular, and lurid nature of its symptoms makes it appear more threatening. My sense is that the human mind’s threat-detection system is wired for the obvious and near-term, not the subtle and long-term.
The real danger lies in the numbing nature of most reporting, which makes it appear as though threats are everywhere, but that addressing those threats is beyond the ability of “normal” people. Similarly, real danger also lies in mistaking short-term symptoms for long-term causes.
IMO we live in an interesting time: While the pace of innovation is extremely high and returns on investment (so to speak) are near-term and obvious, we are also encountering many problems (such as terrorism) which require thinking and approaches on the time-scale of human generations.
--Josh 02 Oct 06
What’s wrong with fear of shark attacks. Sure, the general public doesn’t need to worry about them, but a surfer looking to kit a break known to have sharks is probably justified to act accordingly. Like you said, it matters when it matters. Sometimes, though, it does matter.
ML 02 Oct 06
@ Josh
a surfer looking to kit a break known to have sharks is probably justified to act accordingly.
Yes, .000001% of those watching shark attack stories on CNN do so out of some sort of genuine need. The rest just get excited by thinking about the idea.
Jamie 02 Oct 06
I agree that most companies don’t need to worry about scaling and that they probably won’t be the next Google or MySpace. However, I’m not convinced that these motivations to build something that scales large are based solely on fear. I think people just want to be proud of stuff they build. Maybe my web app will never withstand a huge amount of traffic, but dammit, I want to be proud of the fact that it has and it can. Maybe it’s not fear, but rather bragging rights. Ego? All of the above?
steve o 02 Oct 06
Re: “As long as they don�t wipe the database, you can live with most bugs for a while.”
I would tend to agree with this… but it was interesting listening to an interview on NPR this morning with Steve Wozniak, where he pointed out that with the Apple II (I believe) there were no software OR hardware bugs that shipped—zero. He continued by pointing out how we just accept these bugs today as part of the deal—“oh well, sometimes computers just won’t work.” Like I said, I can live with some bugs. But Woz makes an interesting point.
ML 02 Oct 06
In order to keep this thread on the subject of the orig post, some of the back comments have been collected here (the originals were deleted)…
Yvonne 02 Oct 06
I still see the .php extension in the URL. Does this mean the blog is not Riding Rails yet?
—
JF 02 Oct 06
Yvonne, this blog isn’t powered by PHP anyway. It’s running on Movable Type which is written in perl. We use PHP for quick includes and a few small things MT doesn’t provide.
Anyway, we’ll be running on our own blog system shortly. It’s not the top priority so it’s slipped a bit.
—
Yvonne 02 Oct 06
It’s not the top priority so it’s slipped a bit.
May I ask what 37signals top priority is right now? I’m just curious :)
—
JF 02 Oct 06
May I ask what 37signals top priority is right now? I’m just curious
This thread isn’t about PHP, Rails, the blog engine, or priorities. Let’s stay on topic please.
—
Yvonne 02 Oct 06
This thread isn’t about PHP, Rails, the blog engine, or priorities. Let’s stay on topic please.
@JF
Ouch. That was harsh.
Actually, the topic of this thead is ‘Will it scale’.
I was asking a question about the blogging software you use that ‘.. wait for it ‘. isn’t scalling to your liking, thus the reason for creating your own system.
I could go back in the archives and find where you guys have exactly stated that MT is not scaling to your liking but I have better things to do with my time.
Why so harsh to a customer who was asking a harmless, related to the topic question ’ in a nice tone (I used the smiley if you didn’t notice).
—
JF 02 Oct 06
but it almost sounds like there is a product coming up from 37 that will compete for the mere reason to compete.
Huh? We’re not releasing a blog product. We’re building something custom for ourselves for our own needs. This isn’t a product.
Yvonne, I’m not being harsh, I’m trying to keep things on topic. First you asked about php/rails then asked about 37signals’ priorities. Those don’t relate to this post.
If you have a comment relating to the post topic feel free to share it and we may respond, but I’m just trying to keep things pointed in the appropriate direction.
—
Yvonne 02 Oct 06
@JF,
To make sure we are clear. First I asked if the site had been moved over to the announced Rails blog system (non product for those of you wondering) being developed by your team.
It was replied back that no, the transition away from MT has not happened b/c priorities have changed.
It seems strange to me, actually some what ironic since this post is about Scaling issues, that 37signals scaling problems with MT is no longer a high priority.
—
Our final response: We’ll keep you posted on what happens with the blog setup. We don’t have any more to say about it now. Let’s stick to the topic at hand. Thanks.
Brian 02 Oct 06
With regards to terrorism, I think you’ve confused the fear of dying in an attack (as covered in reason) with the fear of being affected by an attack. The likelihood of me being killed an an auto accident is definitely greater than that of being killed in a terrorist attack. However, the likelihood of me being affected by any auto accident is infinitely less that that of being affected by any terrorist attack, which would likely have widespread political and economic repercussions.
I agree that it is irrational to change travel plans based on news of a hijacking plot, but it is not irrational to have a healthy concern for the safety of United States’ airways. Politicians and newspapers are not only justified, but obligated to cover such topics.
Joe Ruby 02 Oct 06
Your risk of dying in a plausible terrorist attack is actually much lower than your risk of […] being murdered.
Not the same thing? :P
Bird flu’s scary because they’re saying it’s a matter of when, not if, it mutates into a human form that will likely rival the Spanish Flu which killed 50-100 million people.
kangarool 02 Oct 06
Brian’s post about the flow-on effect (economic repercussions, psycological effects) of certain incidents (e.g., terrorism) vs. other more common types (car accident) is perceptive.
I’d also add that an element of control comes into the psychology: whereas i believe that I mostly have *control* over whether I’m in a car accident or not (true or not), I also believe (rightly) that I have no control over whether there’s a bomb on this plane (no control at all, which I would guess equates to more fear. I’m not a psychologist).
And even on something more remote, Shark Attack, it’s not just a numbers issue. That is, you can’t say “Numerically, it’s almost impossible that there is a shark in these waters, therefore, I will not think/fear shark attack.” You saw Jaws. You swum/swam/swimmed (?) afterward. You thought about Sharks! You KNOW there’s not one down there… er, could there be?
The article, while I’m sure numerically accurate, is applying Ration (numbers) to Emotion (Fear). Like Thomas Friedman’s Freedom/Price of Oil thing.
Des Traynor 02 Oct 06
Nice one Matt, I enjoyed your post, and the outbound links.
As a *mild* counterpoint, I’d say that in many cases the scaling issues must be dealt with before your site gets big. Say for example you wanted to create a free online library (i.e. find every free book on the net, host it, and provide a good interface complete with citations/references etc), you’ll be into massive scaling issues with data storage and search algorithms , the complexity of data structures used and the like before you even have your first customer
Maybe its a case of…
1) If your users produce your content scale _later_.
Only when you have thousands of users, will you have enough data to worry about scaling.
2) If you already have lots of content, scale _now_.
If you’re claiming you can search 140,000 academic publications in real time, you’d better be able to do it quickly, and if you’re crawling the web looking to put more into your database, you don’t want to over load it.
Anonymous Coward 03 Oct 06
I agree 100% with “scale later” in terms of building the application itself, however… if you know your app is going to have hefty bandwidth usage as your number of users goes up & up, you should think about how much that bandwidth is going to cost when you’re figuring out your business model.
It’s sort of the other side of “will it scale?” … “how much will the bandwidth and extra CPU cost if it needs to scale, and how does that affect the app’s ability to make money?”
Figuring _that_ out is different than worrying about the technical aspects of scaling prematurely (and it doesn’t take very much time to come up with a reasonable estimate of GiBs per month).
Thoughts?
elnate 09 Oct 06
But… some may say we have myspace BECAUSE friendster didn’t FEAR these things… No?
elnate 10 Oct 06
And then this in my inbox from the folks at PaidContent.org:
— Our Servers Today; And Changes Ahead [by Rafat]
As you may have noticed today, our servers have been very flaky, and you may be getting the database errors left and right. Our internal experience is that our sites are loading once out of every 4-5 tries, which is horrible and very, very frustrating.
This on a day when our traffic shot up dramatically (more than tripled) because of the Google-YouTube $1.65 billion deal news. There isn�t much we can do except wait for this weekend, when we are relaunching our sites, as well as moving to a much bigger and more reliable hosting company.
Our sites will probably be unavailable starting Friday until Monday, but we will notify you on that later this week. Thanks for your patience, and if the site doesn�t come up the first time, hit reload. But if you don�t need to, than spare some bandwidth for the others.