I’ve been at 37signals for a couple of months now, and I thought I’d introduce myself and share the answers to a few questions I’ve been asked. I’m our “business analyst”, “data guy” or “stats guy”, but those all leave some explanation lacking of who I am and what I do.
Who are you?
I’m a mechanical engineer by training, but I spent the last couple of years working at McKinsey & Company, a consulting firm. I did a range of different things, but spent most of my time focusing on understanding and predicting individuals’ behavior as it relates to the healthcare system – how people use physicians, hospitals, and their health insurance policy. I live and work just outside of Pittsburgh, PA.
What would you say you do here?
I tend to think that there are three things that I’m trying to do.
- Look at data to try to improve our products- understanding how people interact with and use our products helps us make better decisions about how to improve them. Sometimes this is about new features; other times, it’s about the smaller details – where does a pagination break make sense based on how many projects people have?
We’re planning to share some specific examples of how we’re using data to inform design decisions over the next few months. - Use data to help grow – Too much of using data to “help businesses grow” is about predicting (guessing) revenue or profit in five years. This is kind of fun intellectually (and there are cleverly named tools like “Crystal Ball” to do this), but it’s less useful than finding real things that we can do today or in the near future that will meaningfully impact the business. This means focusing on why people cancel, how they interact with signup pages, etc.
- Answer those “I wonder” questions- one of my favorite parts of working at 37signals is watching products develop every day in Campfire and on Basecamp. While doing that, people occasionally start a sentence with “I wonder…”. I get to move from wondering to knowing, and that’s loads of fun to do.
How do you do what you do?
There’s nothing magical here – most of the challenge is just about knowing the right questions to ask and understanding the relationships between data. The actual tools matter a lot less.
We have a few reporting tools that we’ve built, and use a few commercial tools (Google Analytics, Clicky), but most analysis gets done from raw data. I do virtually all of my work within a statistical programming language called R (with a couple of custom functions to interface to our databases and APIs for Clicky, etc.) and with simple unix tools like awk/sed. This combination is incredibly simple, powerful, and adaptable, and R has a great community behind it. I use a couple of different tools to produce graphics (mostly ggplot2 in R and OmniGraphSketcher), and I’m experimenting with narrated screencasts, etc. to share results as well. Of course, my trusty HP-12C never leaves my desk.
Have any other questions? I’ll try to answer any below or in a future post or podcast.
Timothy
on 07 Feb 11No use of SAS?
Noob
matt
on 07 Feb 11Hmm…commenter ate most of my comment…retrying.
Yes. No use of SAS. SAS is a tool…like many others. For some things R works much better. R’s use is widespread in the predictive analytics world. I also do data analytics for a living and I don’t use SAS for most of it. Custom map reduce pieces end up kicking the crap out of SAS’s performance when it comes to multi terabyte data sets.So … appropriate tool for the job. It would be like saying you should use C for everything or python for everything. It’s naive.
Des Traynor
on 07 Feb 11Hey Noah,
Thanks for your post.
Re: podcast – I’d love to hear your thoughts about what makes for a good top level view of an applications success. Obviously on one hand you’ve “money in the bank”, but what other data do you collect to see how a new or existing product or feature is doing.
Obviously I’m not looking for actual numbers here, just some examples, e.g. do you raw numbers or percentages, or comparisons with previous months, or any of that. I’m wondering do you see much insight in the often quoted metrics such as Monthly Active Users, Cost per Acquisition, %New customers, Avg revenue per customer, etc.
I wrote about dashboards recently on the Contrast blog, but I’d love to hear your thoughts, as I’m guessing your data is far bigger and more long term than ours, or our clients.
Cheers, Des
Jon
on 07 Feb 11Interesting, I do something similar to your earlier job at McK and am thinking about focusing down in future – nice to hear a bit about your day to day.
Are you able to run live trials of slightly different designs to gather user metrics?
Alex
on 07 Feb 11Timothy: R is a sweet open source alternative with lots of community support. It’s also infinitely less expensive than SAS (read: free).
Richard
on 07 Feb 11Have you tried mixpanel.com for real-time stuff?
W
on 07 Feb 11Surely Timothy was being sarcastic?
Piet
on 07 Feb 11I find this very interesting! Can’t wait to see some more in depth articles. (Especially some cool graphs ;)
Timothy
on 07 Feb 11@Alex
It just seems strange to me that someone who came from McKinsey & Co. is accustomed to using free alternatives to SAS when most of their clients are Fortune 100 companies who pay big money for Business Intelligence software.
Hashmal
on 07 Feb 11I’d love to see some charts and graphs. There’s something sweet about them… I need to see some.
Jeff
on 07 Feb 11@Timothy
Actually, R is used by most of the “big” companies out there. Aside from being more powerful than SAS from a language perspective – it has an enormous user community. Most all academic research is being done with R, and incredibly vibrant communities in finance and bioinformatics are present. In fact I co-organize the US conference on R and Finance- which attracts names from the biggest and best banks and funds around the world. From a BI perspective, all of the needed tools (and many more) are present. Additionally there are numerous companies providing support for R to businesses around the world – including mine www.lemnica.com.
Most Fortune 500 pay for license fees to SAS because that is what new hires came in knowing - today they are more likely to be fluent in R – as it is portable from job to job and being taught in every major academic institution in the world.
NY Times Article on R
Kudos to Noah for sharing his toolkit, and next time you’re in Chicago - I’m just down the street from your offices. Drinks are on me.
Dhrumil
on 07 Feb 11This is fantastic. We are looking to hire a numbers guy for our team. Any suggestions on job board where we can find someone like you?
NL
on 07 Feb 11@Des – I could (and will) write a whole post about dashboards, but the short answer is that I think a lot of those metrics have value. The three key things with dashboards for me are: 1) The metrics should represent how things are going now, not a build up of historical performance (e.g., membership change is more useful than total membership). 2) There should be few enough metrics that you get comfortable with changes. For me, this is certainly no more than 10, preferably 6-8, metrics that I look at on a daily basis. 3) You actually have to look at them. The best dashboards in the world are useless if you never use them. For me, that means that I gravitate towards automated email dashboards waiting in my email when I wake up.
@Timothy—just because you can buy expensive software doesn’t mean you should. I overwhelmingly used free/open source versions of software while I was at McKinsey (including GNU Octave vs. Matlab). I don’t have anything against SAS (and having lived in NC, think they’re a great company), but R meets my needs well, and I never have to think about whether it’s worth the cost.
@Richard – only very briefly. I have it on my list of things to play with someday though. If you have any experience with it that you’d like to share, feel free to email me – noah@37signals
Nick Campbell
on 07 Feb 11As a college student, I guess my questions are pretty basic since I haven’t opened up R yet. My school tossed us into Mathematica for analysis and MiniTab for graphics. Which leaves me with a handful of questions about R (our teacher wanted us to get into using R, but we never got past the Mathematica portions).
When you were learning it, did you have any exercises to run with it or did you just begin by making things up? (Guess that’s a where did you begin question.)
Did you customize R in anyway to better suit your needs when you were starting out or ever?
Brandon Adams
on 07 Feb 11I’m curious, how’d you get into this field? Did you major in statistics? Or did you pick up all you needed on the job?
Nathan
on 07 Feb 11GregT
on 08 Feb 11I have no interest in this topic, just wanted to throw out there that I visited Pittsburgh for the first time last summer and was amazed at how awesome it is. I suspect that it may be the city with the greatest discrepancy between the popular, uninformed perception and the current reality.
Cheesehead
on 08 Feb 11Pittsburg also has this year’s second best football team.
Dude
on 08 Feb 11There’s nothing magical here
Matthew
on 08 Feb 11I do quite a bit of data analysis in my job (environmental data, not user based or financial based), and use a lot of free software. MATLAB is our choice for ease of use, but for our hardcore number crunching (2-10 million data points)we use FORTRAN (oldie but goodie).
Lots of database connectors and custom developed routines in perl, various *nix shells, and C for handing probabilities/bias corrections as well.
LA
on 08 Feb 11I’d appreciate seeing both the details of how you found the answer to a specific question, and details about your process of reading, parsing, and handling the log files. I have a background in statistics and have been thinking about this, but have not done anything yet. Thanks!
Paul
on 09 Feb 11Top notch company with a top notch stats guy… impressed!
Gideon B
on 09 Feb 11I’m with Des, I’d love to hear more about you analytic process
Igwe
on 10 Feb 11Would love to hear more from you noah. Read head first data analysis and have been hooked ever since.
This discussion is closed.