Great UI in Chrome here… I had about a dozen tabs open, and some audio was playing. It was an auto-play ad, so I didn’t initiate it and I didn’t know where it was coming from. I happened to look up in the tab bar and spotted a little speaker icon in one of the tabs (see the middle tab in the photo). I clicked it. Sure enough, that’s where the sound was coming from. When the video ended, the little speaker icon went away. Great little touch that answers a common question… “Where’s that sound coming from?”
“All that work, and that’s it?”
I remember thinking this to myself a few weeks ago. I’d been building a new homepage for Know Your Company. But I didn’t feel I’d made much progress.
I’d rewritten copy, collected stories from current customers, designed several new pages, made the site more mobile-friendly…
Yet despite these changes, the site didn’t look much different than before. I began to question if I’d accomplished much at all.
Luckily, as I started to feel this way, I happened to be chatting with Jonas, a designer at Basecamp. He’s also the original designer of Know Your Company, and has helped me transition the product into its own company.
Jonas said this to me:
“Claire, go read what the homepage had before.”
I went and did that.
“Okay. Go read it now with your changes.”
I went and looked at my new site.
“See? Before, people looking at your site didn’t know what customers thought about your product. Before, they couldn’t request a product demo as easily. Now, your revised design helps people do those things. So don’t get discouraged because your design doesn’t look different. If you read it, you’ll see it’s much better than it was before.”
What a great reminder.
I’d forgotten my progress simply because it didn’t look different. When you just look at the difference, you might not see much. Text looks like text, regardless of what it’s saying. But if you read the difference, you see how big your changes actually are.
Progress isn’t measured by how different something looks. It’s measured by how useful something has become. Is it making this person’s life easier? Is it doing the job you want it to do? Reading the difference, not just looking at it, reveals your progress.
So the next time you doubt how much progress you’ve made, don’t look at the difference. Read the difference.
You’ve probably accomplished more than you give yourself credit for.
Nearly every boss has said it. And just about every employee has heard it. Yet it’s one of the most meaningless lines ever spoken in the office:
“My door is always open.”
The statement usually is followed up with some version of, “If you ever have an issue with anything, please come talk to me.”
What’s wrong with this? Isn’t it important for your employees to know that you are open to hearing their suggestions, concerns, and criticisms? Of course it is.
But let’s be real here: In most cases, “My door is always open” isn’t really an invitation to speak up. It’s a cop-out. It makes the boss feel good but puts the onus entirely on the employees. You might as well say, “You find the problems and then take all the risk of interrupting my day and confronting me about them.” How many people have taken you up on that offer?
Your employees have lots of opinions about everything—your strategy and vision; the state of the competition; the quality of your products; the vibe in the workplace. There are tons of things you can learn from them.
But how many of these ideas and opinions have you actually heard? A tiny fraction, I’d bet. The reality is that companies are full of things that are left unspoken. And even when they are out in the open, the CEO is almost always the last to know.
I like to think of myself as a leader whose door is always open. But I recently learned that an open door isn’t enough…
Let’s take a break from the business news cycle. I love the news and I’ve covered it for a decade: what new startup is launching, why a stock price just moved, who’s being hired or fired. But there’s a whole universe of fascinating stories waiting to be covered about what’s old in business.
That’s why we’re launching The Distance, a new online magazine featuring original journalism about bootstrapped businesses that are at least 25 years old. If you’ve ever been curious about your favorite family-owned restaurant or that little shop on the corner, this is the publication for you. These businesses might not make headlines, but their owners have compelling stories about how they started, what they’ve learned, and why they keep doing it.
This is a heady time for people interested in great stories, whether it’s telling them or reading them. From newer sites like Vox and FiveThirtyEight and Narrative.ly to legacy media outlets that keep producing indispensable work (I still subscribe to two print newspapers), today’s readers have a lot of choices competing for their limited time. The Distance offers its own kind of storytelling – enjoyable reads about long-lasting businesses and the people behind them.
We’ll be publishing one story a month starting in May. We hope you find the companies of The Distance as interesting as we do and come back each month for more. (And while Basecamp is sponsoring the magazine, The Distance is editorially independent and we will not write about Basecamp customers.)
We’d also love to hear from you. If you know of any companies that would make good profile subjects, please let me know!
The annual Chicago Comics and Entertainment Expo is this weekend and we’re really excited to announce that Basecamp will be sponsoring designer toy maker, Shawnimals at the show!
Shawnimals uses Basecamp to make awesome stuff, from plush ninjas to recycled sketchbooks. The show runs from April 25th–27th, so come by booth 655 to say hi and pick up a limited edition Happy Sherpa plush!
Performance tuning is a fun sport, but how you’re keeping score matters more than you think, if winning is to have real impact. When it comes to web applications, the first mistake is start with what’s the easiest to measure: server-side generation times.
In Rails, that’s the almighty X-Runtime header — reported to the 6th decimal of a second, for that extra punch of authority. A clear target, easily measured, and in that safe realm of your own code to make it appear fully controllable and scientific. But what good is saving off milliseconds for a 50ms internal target, if your shit (or non-existent!) CDNs are costing you seconds in New Zealand? Pounds, not pennies, is where the wealth is.
Yet that’s still the easy, level one, part of the answer: Don’t worry too much about your internal performance metrics until you’ve cared enough about the full stack of SSL termination overhead, CDN optimization, JS/CSS asset minimization, and client-side computational overhead (the latter easily catching out people following the “just do a server-side API”, since the json may well generate in 50ms, but then the client-side computation takes a full second on the below-average device — doh!).
Level two, once reasonable efforts have been made to trim the fat around the X-Runtime itself, is getting some big numbers up on the board: Mean and the 90th percentile. Those really are great places to start. If your mean is an embarrassing 500ms+, well, then you have some serious, fundamental problems that need fixing, which will benefit everyone using your app. Get to it.
Keep going beyond even the 99th
Just don’t stop there. Neither at the mean or the 90th. Don’t even stop at the 99th! At Basecamp, we sorta fell into that trap for a while. Our means were looking pretty at around 60ms, the 90th was 200ms, and even the 99th was a respectable 700ms. Victory, right?
Well, victory for the requests that fell into the 1st to 99th percentile. But when you process about fifty million requests a day, there’s still an awful lot of requests hidden on the far side of the 99th. And there, young ones, is where the dragons lie.
A while back we started shining the light into that cave. And even while I expected there to be dragons, I was still shocked at just how large and plentiful they were at our scale. Just 0.4% of requests took 1-2 seconds to resolve, but that’s still a shockingly 200,000 requests when you’re doing those fifty million requests.
Yet it gets worse. Just 0.0025% of requests took 10-30 seconds, but that’s still a whooping 1,250 requests. While some of those come from API requests that users do not see directly, a fair slice is indeed from real, impatient human beings. That’s just embarrassing! And a far, far away land from that pretty picture painted by the 60ms mean. Ugh.
Finally, there was the true elite: The 0.0001%, for a total of 50 instances. Those guys sat and waited between 30 and 60 seconds on their merry request to complete. Triple ugh.
Since lighting the cave, we’ve already been pointed to big, obvious holes in our setup that we weren’t looking at before. One simple example was file uploads: We’d stage files in one area, then copy them over to their final resting place as part of the record creation process. That’s no problem when it’s a couple of 10MB audio files, but try that again with 20 400MB video files — it takes a while! So now we stage straight in the final resting place, and cut out the copy process. Voila: Lots of dragons dead.
There’s still much more work to do. Not just because it sucks for the people who actually hit those monster requests, but also because it’s a real drain on the rest of the system. Maybe it’s a N+1 case that really only appears under very special circumstances, but every time the request hits, it’s still an onslaught on the database, and everyone else’s fast queries might well be slowed down as a result.
But it really does also just suck for those who actually have to sit through a 30 second request. It doesn’t really help them very much to know that everyone else is having a good time. In fact, that might just piss them off.
It’s like going to the lobby of your hotel to complain about the cockroaches, and then seeing the smug smile of the desk clerk saying “oh, don’t worry about that, none of our other 499 guests have that problem… just deal with it”. You wouldn’t come back next Summer.
So do have a look at the far side of your histogram. And use actual request counts, not just feel-good percentiles.
Before he made The Simpsons, Matt Groening’s famous comics and illustrations graced the covers of Apple brochures. The writing inside—from 1989, mind you—still does a great job selling the Mac.
Instead of blanket marketing a one-size-fits-all message, Apple took the time to speak to every situation a person is in. If you’re feeling overwhelmed, the Mac is there to put order back in your life. If you’re unemployed, the Mac is there to help you chase a career. If you’re a habitual procrastinator, the Mac is there for your spark of productivity.
They’re listening to us, and our problems. Talk about empathy.
Plenty of marketing today, especially in software, is a package of feature sets, bells, whistles, and some boasting about how they’re better than the next guy. Perhaps there’s mention of “benefits,” sure, but we’re always left figuring out how a product is supposed to fit in our own lives.
Chances are, your product isn’t for everybody. It doesn’t have to be, either. Really listen to your audience—you’re lucky to have them. Instead of assuming what they need, ask where they’re coming from. How did they get to the point of finally asking your product for help? If you can figure that out, all of a sudden your marketing changes from “making sales” to “being there for friend.” That feels good.
So, a few years ago Dana Brunetti at Trigger Street Productions (Social Network and Captain Philips) got in touch with us after reading our book, REWORK. He loved the themes and the overall story of how our company came to be.
Some time passed and we hadn’t heard anything… Until a week ago. We’re super-excited to let you know that Netflix Originals has decided to take on the project and turn our little book into a feature film!
Martyn Burke was brought on to write the screenplay and Gwyneth Horder-Payton will direct. Filming is supposed to start sometime this spring and we just got these beautiful promo posters in the mail.
It’s rare that companies exercise options like the one they had on this book and we really can’t say how unbelievably excited and flattered we are to have our story told in a whole new way.
If there’s one thing everyone can agree on, it’s that computers and user interfaces peaked in the 1980s. Everything was clear and simple, without all of our modern annoyances like portability, color, touchscreens, and the Internet.
With that in mind, we took a long, hard look at Basecamp. We asked questions like:
- Is Basecamp as simple as it can be?
- Should we make Basecamp less colorful?
- What if Basecamp was stored on 800kb floppy disks?
- When is Knight Rider starting?
So today, after months of work, we’re excited to announce that Basecamp is going back to basics. We’ve rolled out an update to all our users that’s available right now. Just visit Basecamp in your web browser, and type the code
dogcow to get a sneak preview of our new direction. We think you’re going to love it.
Basecamp’s new system requirements:
- Macintosh System 6 or higher
- 512kb of RAM
- 800kb floppy disk drive
- 1200bps dial-up modem (recommended)
P.S. refresh your browser to exit the preview.
As we detailed in Basecamp was under network attack, criminals assaulted our network with a DDoS attack on March 24. This is the technical postmortem that we promised.
The main attack lasted a total of an hour and 40 minutes starting at 8:32 central time and ending around 10:12. During that window, Basecamp and the other services were completely unavailable for 45 minutes, and intermittently up and down or slow for the rest. In addition to the attack itself, Basecamp got put in network quarantine by other providers, so it wasn’t until 11:08 that access was restored for everyone, everywhere.
The attack was a combination of SYN flood, DNS reflection, ICMP flooding, and NTP amplification. The combined flow was in excess of 20Gbps. Our mitigation strategy included filtering through a single provider and working with them to remove bogus traffic.
To reiterate, no data was compromised in this attack. This was solely an attack on our customers’ ability to access Basecamp and the other services.
There are two main areas we will improve upon following this event. Regarding our shield against future network attacks:
- We’ve formed a DDoS Survivors group to collaborate with other sites who’ve been subject to the same or similar attacks. That’s been enormously helpful already.
- We’re exploring all sorts of vendor shields to be able to mitigate future attacks even faster. While it’s tough to completely prevent any interruption in the face of a massive attack, there are options to minimize the disturbance.
- Law enforcement has been contacted, we’ve added our statement to their case file, and we’ll continue to assist them in catching the criminals behind this attack.
Regarding the communication:
- There was a 20-minute delay between our first learning of the attack and reporting it to our customers via Twitter and status. That’s unacceptable. We’ll make changes to ensure that it doesn’t take more than a maximum of 5 minutes to report something like this again.
- Although we were successful at posting information to our status site (which is hosted off site), the site received more traffic than ever in the past, and it too had availability problems. We’ve already upgraded the servers that power the site and we’ll be conducting additional load and availability testing in the coming days.
We will continue to be on high alert in case there is another attack. We have discussed plans with our providers, and we’re initiating new conversations with some of the top security vendors.
Monday was a rough day and we’re incredibly sorry we weren’t more effective at minimizing this interruption. We continue to sincerely appreciate your patience and support. Thank you.