Highlights from this week’s 37signals staff posts at Twitter.

@mattlinderman From the “if it sounds like BS, it probably is” category: Goldman Sachs’ “synthetic collateralized debt obligation” http://nyti.ms/acXgJN

@dhh I mean, really, Apple, http://is.gd/buYUb and http://is.gd/buYTR – words fail me. (via timbray)—it’s a sad time to be an Apple “fan” :(

@rjs Bit the bullet and got a network drive. The WD My Book was really easy to set up. Plug into Time Capsule. Done. http://amzn.to/baWO0O

@dhh Someone should plot the historical likelihood of failure if a company reaches Series E funding. Gut says it’s towering.

@mattlinderman Art w/ push pins instead of pixels. “Georges Seurat meets Office Depot.” Neat how viewer distance changes the impact. http://bit.ly/aBJ0JN

@jasonfried “Knowing the name of something doesn’t mean you know the something.” -Richard Feynman (paraphrased)

@mattlinderman http://bit.ly/de08xT – Consulting business? Cut back client work 50% for a while & build a product. Even if it fails, you’ll learn a ton.

@rjs A product can be so compelling that it reaches out into its context and rearranges the constraining forces and incentives.

@dhh Thanks to awesome some volunteers, we’re in the process of bringing Basecamp to Norweigan, Russian, Greek, French, Swedish, German, and more

@rjs The more websites I work on, the more I understand why people say IA and graphic design are different skill sets.

@asianmack Companies spend $$$ on interactive microsites. Agencies make $$$ creating them. Who goes there or remembers them? iAds won’t be better.

@jasonfried Excess meetings, documents & policies are the saturated fats, sugars, preservatives & artificial ingredients of business.

Ning is laying off 40% of its staff and dumping free versions of its service. That’s a shitty day for the people who lost their job and the folks left behind without their coworkers. I went through a few rounds back in the dotcom days and fun it was not.

But I can’t help but be puzzled by the coverage of this. Here’s TechCrunch on the situation:

While the massive layoffs are obviously a big hit to the company, it isn’t all bad news for Ning: the service is still seeing its traffic grow according to comScore. But traffic growth is no longer good enough for the company — it needs to start generating some serious revenue, and advertising clearly isn’t cutting it.

Are you kidding me? The company has blown through $120MM of VC funding over six years, built up massive traffic, yet just had to slash and burn, and you’re saying that “traffic growth is no longer good enough”. How the hell was it ever good enough?

Ning’s problem is not a lack of eyeballs but its inability to turn them into cash money to pay the bills. Getting more of something that’s a net-negative is not going to make up for it.

That was always their problem. From day one. Just like it’s any other business’ problem. Acting all shocked and surprised now is just incredibly ignorant of our industry’s very recent past.

This is the same kind of ignorance that goes on to celebrate so-called businesses successes before they posted black numbers on the balance sheet. Until that happens it’s all conjecture and possible maybes.

The just-give-it-away-for-free-and-they-will-come-and-we’ll-be-rich automatron is as broken now as it was in 2001.

UPDATE: Just found this fantastic piece of cheerleader reporting from Business Insider from just last year: “Build-your-own-Facebook startup Ning is still on fire… How much money is Ning making with all that traffic? Bianchini wouldn’t comment. But by our back-of-the-envelope calculations, Ning could be on a nearly $10 million annual”. Oh boy.

David Christiansen, Founder of TroopTrack, sent us an email about apologizing well:

Over the weekend I broke the single sign on integration between my SaaS boy scout troop management software and ZenDesk, my help desk. It was broken for three days while I was sick, working on my regular job, and trying to enjoy some portion of Easter. I got about 30 emails from exception notifier, letting me know how my mistake was impacting users.

This morning I read your chapter on how to say you’re sorry. I already knew I needed to apologize, but it helped me to be human about it rather than corporate. Here’s what I sent:

Over the weekend I attempted to improve the single-sign-on feature between TroopTrack.com and TroopTrack Help Desk. Sadly, I didn’t do it right and caused two problems:
1) A brief outage over the weekend that impacted some of you.
2) Many of you are now unable to access the help desk.

The first problem was fixed within a few minutes, but it was still a pain for those of you who were online when it happened. I’m sorry about that.

I’m still working on the second problem. Hopefully it will be fixed soon. In the meantime, if you are having trouble accessing the help desk and need support, please email me directly or call me.

Thanks for understanding. Software is hard – I learn something new every day. Unfortunately sometimes I’m learning from my mistakes!

I appreciate the reminder REWORK gave me this morning to be myself.

Also, there was discussion in our Campfire room about how well done this was: Atlassian update on a security breach.

In summary — we’ve made mistakes, we’re sorry and we’re fixing them — and we’re going to be honest about what those mistakes are. Half of being a reliable and trustworthy vendor from a security perspective is the technical bits, and even though we erred here, we ultimately pride ourselves on how we handle security. The other half is being open and honest, which we’ll never fail at.

Related SvN posts on apologizing:
Hulu CEO: “We screwed up royally”
The bullshit of outage language
The goal is to apologize sincerely and be taken seriously
ThinkGeek: “We’d never get away with taking advantage of you guys, so why would we try?”
How to S.A.V.E. Customer Service

Time: 22:50 | 04/13/2010 | Download MP3

Mark, Joshua, and John on life as a 37signals Sys Admin
The Sys Admin team discusses hosting the 37signals apps, working with programmers, helping support, telecommuting, dealing with vendors, improving speeds in Europe, and more.

More episodes
Subscribe to the podcast via iTunes or RSS. Related links and previous episodes available at 37signals.com/podcast.

Spread the word
Like this episode? Please share it with your friends:

In 2009 we ran into some problems when failing over to the Basecamp secondary database. Basecamp relies on a keeping large working set of recently-accessed data in its InnoDB buffer cache for speed. Normal MySQL replication only sends writes, not reads, to the secondary. How could we ensure the data in the secondary’s cache is up to date?

We contracted Percona to build a solution into their Maatkit toolset based on their experiments with SELECT query mirroring. It involves a clever usage of tcpdump to capture and replay SELECT queries from the primary to the secondary database.

Here’s the resulting command.

/usr/bin/mk-query-digest --statistics --iterations 4 --run-time 15m --type tcpdump
--filter '$event->{arg} && $event->{arg} =~ m/^SELECT/i'
--statistics --execute \"h=db-secondary,P=3306,u=secondary,p=password,D=production\"
--execute-throttle 70,30,5

The tcpdump utility captures MySQL traffic from the primary and feeds the data into the mk-query-digest script. This script filters only the SELECT queries and executes them on the secondary database. The throttle argument sets the percentage of time the script should execute queries on the secondary, how often to check that value, and a percentage probability that queries will be skipped when the threshold is exceeded.

Here’s some sample statistical output:

# execute_executed      124668
# throttle_checked_rate     29
# throttle_rate_avg      29.84
# throttle_rate_ok          29

According to these values, the script didn’t reach the 70% query execution threshold we set. Our queries are executing on the secondary cleanly.

Since we began using this tool we switched production database servers without a performance reduction.

Note: This blog post was originally entitled “Keeping your slave warm”, and used the master/slave language throughout. We updated this to use the primary/secondary language in December of 2019, as the offensive nature of the original wording came to our attention.