28 Apr 2004 | Jon Gales said...

I have always liked how PhoneScoop does it. If something is different, it puts the better feature in bold.

Here is a comparison of the Sony Ericsson T610 Vs. Sony Ericsson T630

28 Apr 2004 | Garrett Dimon said...

Putting a summary paragraph that would cover all the possible different combinations and comparisons would equate to writing that paragraph for n(squared) items. That is, you would have to have a comparison between every product so each product would have 40 different comparisons.

If n = 20 then you're talking 400 comparison paragraphs. If n = 40 then you're talking 1600 paragraphs. I'm sure you can quickly see the content generation difficulty there.

Similarly, from a technical standpoint, many times you are comparing items that, while similar from a big picture point of view, differ completely in their technical specs and measurements. The logic to handle those differences elegantly isn't exactly black and white. Often times the difference is much more complex than a black and white conversion. It's often apples to oranges on some features.

That all being said, I love the thought and thinking outside of the box. It's pushing the envelope like this that gets us to the better ideas.

28 Apr 2004 | Noah said...

Man, I'm so glad that Yahoo! Shopping compared the "Manufacturer Part No." for me. That's really important for my purchase decision.

28 Apr 2004 | JF said...

If n = 20 then you're talking 400 comparison paragraphs. If n = 40 then you're talking 1600 paragraphs. I'm sure you can quickly see the content generation difficulty there.

Think simpler. What are the top 3-5 items for a digital camera, for example? Maybe megapixels, overall size, zoom, included storage, and perhaps manual/automatic controls. I'm just guessing here, but I'm sure each product category has a top 3-5 most popular key feature list.

What I'm looking for is something that might say:

"Key differences: Model A has a better optical zoom (4x vs 3x). Model A is smaller overall (# vs #). Model B takes high resolution photos (5 megapixels vs 4 megapixels). Model A uses CompactFlash storage and Model B uses SmartMedia storage. Both come with 256 MB storage cards."

"If you are looking for a higher resolution camera, Model B should be your choice. If you are looking for a better zoom, consider Model A. If size is most important, choose Model A."

...or something like that. That could be written by a computer. Turning feature points into short sentences isn't really that difficult. But, I know it's not fair to say something "isn't that difficult" without thinking about all the other issues these sites have to deal with. I'm just putting some ideas out there.

So, yes, it's work, but if these sites really want to help people make decisions they need to think more about a customer's expectations. Comparing complicated data charts is not something that we expect to have to do and it's not something we should have to do.

28 Apr 2004 | Micah said...

A diff run two nearly identical product description pages would save you the trouble of reading the same text twice. A nice web interface to diff could actually be kind of useful, just feed it two URLs and it would return the distinguishing text. Has anyone seen anything like that before?

28 Apr 2004 | JF said...

Or... If we know the top 3-5 things most important to a digital camera shopper, how about this at the top:

The smaller camera is Model A
The higher resolution camera is Model B
The camera with better optical zoom is Model A

etc...

The point is that these comparison charts are treating nearly all the specs/info with the same priority. There are surely some key data points that are more popular than others and those should be clearly called out. I'll put these ideas into the 37better Comparison Grid when I get to it.

28 Apr 2004 | ML said...

Apple's iPod comparison page (regular vs. mini)

28 Apr 2004 | Ste Grainer said...

I like some of your suggestions, but what about differences that are subjectively better or worse? Some photographers might want a camera with extremely slow shutter speed (more than a minute) and others might want a camera with extremely fast shutter speed (less than 1/2000 second). How then does the software powering the comparison decide which camera (one that offers 1/1000 - 5 minute exposure and another that offers 1/5000 - 30 second exposure) has the "better" shutter speed? (I'm not even certain that 1/5000 is attainable - it's just an example.) Some things should be left up to the user to decide which feature they value more.

The thing I don't like about comparison sites is that they don't compare experience at all, just specs/statistics. One camera (since you used cameras, I can't think of anything else) might feel great in the hand and just take fair photos, while another might be almost painful to hold but take beautiful photos. I think the best comparison site should take ranked user reviews (where ranked is like Amazon's "x out of x people found this useful" feature) and run them side by side for each model - based on keywords (similar feature comparisons in each side-by-side review) and ranking (how many people thought the review was useful). *Then* they should feature the important/most requested specs with obvjective differences highlighted and more detail about subjective ones available. The less important specs should be minimized or available on a second page (because some guy is going to want to know the exact battery type or image sensor type).

28 Apr 2004 | Paul said...

JF: Funny you should mention these cameras, as I'm looking at pretty much these models. The sheer amount of information about digital cameras on the web is amazing - I've had to go to at least a dozen sites to figure out what camera is best for my needs.

I'd take your idea one further and suggest a simple, three- or four- question form as an alternative. I say what is most important to me, price range, and maybe megapixels. The comparison comes back and clearly points out what would be best for me - but doesn't exclude anything, unless I want it to do so.

The biggest missing piece here is, as you say, a hierarchy of the information presented. Maybe the person reading the page should have control over that hierarchy, too.

28 Apr 2004 | matthew said...

This is going to be a great discussion and I'm really looking forward to the 37better solution. Of the examples so far, I really like the PhoneScoop one. It's clean and clear and the differences are very obvious. At the same time, the Apple example is far too cluttered.

29 Apr 2004 | ek said...

Hey Paul, I think the problem with that approach is that no one wants to do any work (even though hitting a bunch of different Web sites is a lot more work than filling out a few questions ;-). The sort of survey you mention also, I think, has a bad rep., as none of the ones I've ever encountered has ever made a recommendation that was actually useful.

Looking at this from the business end, I guess I'd have to ask if there's a valid business case for doing this? Would the benefit of providing this sort of highly focused, highly editorialized comparison outweigh the real, and substantial costs in money and human resources?

Given the number of products and the frequency with which they're turned over, you'd really need a team of people just for digital cameras, another team for TVs (perhaps two; one for CRTs and one for flat screens), another team for PDAs, etc., etc. And these couldn't be just random people off the street — to provide comparisons of value they'd actually have to have some domain expertise to be able to figure out what people consider important for each of these products (and within each of the many sub-categories for each of these products).

Given how expensive people are I think you'd have a tough time making the case that it makes business sense to do this.

As such I guess I would say that, more interesting than a "better" version of an existing comparison shopping page that is perhaps not economically feasible, would be a "better" approach to achieving the same end result, which is to help people make an informed purchase decision based on their personal criteria with a cost to the comparison shopping site that's as close to zero as possible.

I think it can be done, but it would require some open minded thinking on the part of the shopping sites.

29 Apr 2004 | rye said...

i would like to see a remove row function for every row of those large comparison tables. it would be a great way to focus in on the differences that matter to me.

29 Apr 2004 | Paul said...

ek: I think the problem with that approach is that no one wants to do any work...

Totally valid and good point. But what if the comparison grid offers an 'advanced' mode that lets one fill out those few questions? That way, it's optional and not required. As a bonus if it doesn't fly, it can easily be yanked. :)

29 Apr 2004 | Bryan said...

What gets me about the comparison shopping sites is how they continue to claim they are giving you all these useful tools to make an informed and educated buying decision, but all they really do well is compare prices. But, if you asked them if there were a comparison PRICING site they'd say no, we're a comparison SHOPPING site.

They're either embarassed by what they really are or they are aren't delivering on their promises.

29 Apr 2004 | Ian Firth said...

The Phonescoop example is nice, but what if I feel that a slightly larger phone is better than a smaller one (easier to hold, etc) ?

Each column in the database would have to reference a column in another table that explained what was better (bigger/smaller, heavier/lighter) based on the product (smaller cameras are nicer, but bigger monitors are better).

This would be easy to design for a shopping site that focused on one product type (like PhoneScoop), but would be a bit of work to implement on a site like Shopping.com that sells everything.

29 Apr 2004 | JF said...

but would be a bit of work to implement on a site like Shopping.com that sells everything

Of course, but the web doesn't have to be all or nothing. How about a better comparison table on just the top 3 product categories to start? Test and iterate. Test and iterate. And stop.

29 Apr 2004 | dru said...

I really like the idea of removing rows of data. What I like about this is that it doesn't presume to know what I think is more important. It could store the rows removed in a cookie or something and allows the user to focus on what they want to see.

The tough part, of course, would be chunking the data.

29 Apr 2004 | Garrett Dimon said...

Jason, I wholeheartedly agree with you that comparison functionality needs help on almest every site and people do start needing to think outside the box.

I also think you're comment about focusing on the top 3-5 features is right on target. I wasn't by any means trying to shoot down your idea. I just wanted to throw level of effort into the mix.

I think focusing on the top 3-5 features would solve a significant portion of the problems indirectly, i.e. information overload, comparison of useless facts, etc.

I really look forward to your 37 Better version.

29 Apr 2004 | Ed said...

As much as I respect you guys for exploring this idea for a design/mockup perspective the truth is that you have to consider the data aspect of this as well. It's one thing to come up with a method that works for say, digital cameras, but apply that same methodology to say, toasters. My wife and I spent over 40 minutes comparing toasters at the store where we could touch them and make well-informed decisions. Did we make the right choice? I think so. And some might say, "It's only a toaster - what's the big deal?" But that's the thing - each of the sites that you mentioned has tried to be a complete and all encompassing comparison shopper site. If you're going to compare digital cameras online use dpreview.com. If you're comparing toasters online - then, you need to get out more.

Seriously... when you make your 37better do one for digital cameras, toasters, tshirts, oh, and one that does comparison shopping for computers. Try MAC vs PC. Oh, and one for video games and one for... I think you get the point.

I've had a lot of respect for 37 Signals but I guess it just seemed like you post made it seem like you guys can do it ALL better. The truth is it takes a lot of time and maintenance.

My respect goes to Garrett - he's the only one who's actually thought this through. JF - you're missing the whole boat on this one. What are the 3-5 top features of a tshirt? I know, it's a tough one but that's what a product comparison shopping site needs to consider as well.

I know the sites you listed don't do comparison shopping for all categories that they carry. So if we're only speaking specifically about digital cameras include dpreview.

29 Apr 2004 | JF said...

JF - you're missing the whole boat on this one. What are the 3-5 top features of a tshirt?

I'm not sure why everyone thinks everything has to work for everything. One size fits all doesn't fit anything particularly well. t-shirts obviously wouldn't fall under this particular compare feature. But for digital devices, for example, I think the suggested method could work quite well.

29 Apr 2004 | RH said...

I onced help develop a similar comparison grid solution for sprintpcs.com / pcsvision.com which had a checkbox the user could select to "show only differences". The user could then quickly identify what separates and distinguishes the phones in PCS's offering.

Makes sense to me.

29 Apr 2004 | ek said...

Hey Ed, not sure what you're getting all worked up about. If you need a site to provide comparison shopping tools to help you buy a t-shirt, then you really do need help.

And though I love dpreview, what if you're not obsessive compulsive and don't want to read through five or six 20-page reviews of digicams to find the one that best suits your needs? I know Phil provides a Pros & Cons list at the end of each review, but you have to be pretty knowledgeable about cameras for that list to really make sense.

You know, I think what would be a lot more useful than an improved version of the comparison functionality on the existing comparison shopping sites is an entirely different model for a comparison shopping site. I'd love it if one of these sites would get the backbone to actually recommend specific products instead of just chucking up every product under the sun.

So, for example, the site could say: "If you're looking for the best pocket-sized digital camera for $300 or less you should go for the Acme DigiPro Model X. Here are five reasons why, and here are the best prices for it...if you're willing to spend $400 you're best choice is Model Y, and so on."

Don't base it on megapixels, or optical/digital zoom-factor, or other techno mumbo-jumbo that 99% of the populace could care less about — base it on the recommendations of a knowledgeable human being. Ultimately, that's what we're looking for, isn't it?

To me, the existing sites are taking a Yahoo-like approach to comparison shopping; stuffing as much as possible onto every page to try to satisfy as many people as possible. I'd love to see someone take more of a Google-esque approach; keep it simple and give people what they want — an informed recommendation and a list of the best places to buy said product for the best price.

29 Apr 2004 | Heath Weaver said...

I have been thinking about comparision shopping. I am not sure if it exists, but there should be a way to compare anything on the web, by either entering your own criteria or by all companies that sell something on the web should have some standardized mark-up for products at least some basics, like product name, price, description. Then one site could be used to import the products by pasting the url of the product and then be able to compare.

29 Apr 2004 | Design GooRoo said...

I've been saying for years that we need a shopping comparison site comparison site.

29 Apr 2004 | PWH said...

Here's another idea: 'Add a feature'. When I was looking for a digital camera, I wanted to know if the model used standard batteries, rechargables or some proprietary design. You rarely see that on a comparison grid. If I request to 'Add that feature', maybe the site owner or other site visitors could populate that field for me.

29 Apr 2004 | jake said...

I think Ste is on to something with his point about using customer reviews as a point of comparison. But I think it would be nice to add in some stuff from PWH's comment. Basically a lot of people have a terrible fear about the anarchy that a Wiki will bring, which is unfounded when you look at the track record. By combining the wiki style of everyone being able to edit things with customer reviews and basic statistics that already exist makes most sense to me.

That way, the commenters who are more reputable are featured and when combined with extra information that others can supply, on top of a streamlined presentation of what we already have, you get everything you need to make a decision in a reasonable amount of time.

29 Apr 2004 | Ed said...

The point I was trying to make is that if you're going to say that Yahoo! does a poor job of comparison shopping it's because they are using the same engine for all of their products. Each time a new gadget prduct line comes out they don't want to have to write a new site crieria for ranking products.

ek: "I'd love it if one of these sites would get the backbone to actually recommend specific products instead of just chucking up every product under the sun."

Who has the time? If you recommend Product A today it becomes obsolete or surpassed in less than 6 months because Product B just came out. So the reasons why Product A were recommend become mute. But who is going to maintain that database? Comparison sites like Yahoo! and Shopping.com work the way they do because someone enters in the products info - info that will never change and is all based on quantitative analysis. Qualitative analysis is completely subjective. So who's to say that it's right? Oh, and don't forget - if a site recommends Product A over Product B that can be interpreted as an endorsement of that product. I believe that has got to be the number one reason why Yahoo! gives you "just the facts" and they leave it up to you to decide. They allow for customer reviews which have to be the best kind of recommendations.

29 Apr 2004 | hp said...

How about turning the comparison table upside-down in that the products (in this case cameras) are listed vertically and several key features (in this case: price, resolution, media supported, size, weight, additional software) are listed horizontally as sorting keys allowing each user to sort ascending or descending on each feature. This could be further extended by allowing users to determine sequence of sorting so, for example, one may want to sort first on resolution, then od media, price and leave the rest neutral. In addition to this, a brief review would be shown for each product.

On the site-level, one would need to add a layer that would associate each product with a category (cameras, mobile phones, cars, apparel, etc.) and then determine features to be picked up for each of these categories. Of course, an extensive comparison option like those that are standard now would always be available.

I have built an addition such as for an existing internal system of a large corporation and it greatly improved selection process. The development was complex but not too difficult.

Regards,

hp

29 Apr 2004 | ML said...

Anyone here subscribe to consumerreports.org?

29 Apr 2004 | ML said...

What do people think about Yahoo!'s SmartSort feature?
Digital Camera Smart Sort

29 Apr 2004 | ek said...

Ed, the fact that it's difficult is precisely why it's valuable. Obviously, chucking up all of this data is pretty easy because everyone is doing it. The hard part is teasing through all the data and making a call on what's "better" within a specific context (in the case of digicams, that would be price and size).

As people isn't that what most of us do when we go to a "real" store? At this point I've pretty much given up on employees at the big box retailers actually knowing anything, but on the off chance that they do, I still ask "so, which camera would you recommend?"

FYI: just for future reference, I think you meant "moot," not "mute" — a surprisingly common substitution.

29 Apr 2004 | Don Schenck said...

Same difference: A "mute" point is one that's not talked about.

:-)

29 Apr 2004 | Arne G said...

--I think you meant "moot," not "mute"--

Theres a surrogate Bill Bryson in every crowd -- and theyre always correcting me. Ive noticed this lately as I seem to be picking up the grammatical quirks of my five-year-old (I had-ed the best-est example, but I forgots it now).

29 Apr 2004 | ek said...

Ah, but Don, a point that's not talked about (mute) is not necessarily a point with no practical significance (moot). ;-)

To ML, I think that SmartSort tool is pretty cool. One thing is that I think the rationale as to why a particular product is listed above another should be emphasized more strongly and perhaps put into the form of a bullet list.

It's also a little confusing because it asks you to enter a price range, without asking you to weigh how important price is to you, and then proceeds to use price as a major ranking criterion.

And then there's the discrepancy between the tool's rankings and the "user rankings," which could be the cause of some confusion, particularly since the rankings display is more visually prominent than the rationale the tool provides.

But hey, nothing's perfect. It does seem to do a good job helping winnow the vast assortment of available options down to a more manageable handful.

29 Apr 2004 | ek said...

Ah, but Don, a point that's not talked about (mute) is not necessarily a point with no practical significance (moot). ;-)

To ML, I think that SmartSort tool is pretty cool. One thing is that I think the rationale as to why a particular product is listed above another should be emphasized more strongly and perhaps put into the form of a bullet list.

It's also a little confusing because it asks you to enter a price range, without asking you to weigh how important price is to you, and then proceeds to use price as a major ranking criterion.

And then there's the discrepancy between the tool's rankings and the "user rankings," which could be the cause of some confusion, particularly since the rankings display is more visually prominent than the rationale the tool provides.

But hey, nothing's perfect. It does seem to do a good job helping winnow the vast assortment of available options down to a more manageable handful.

29 Apr 2004 | ek said...

Whoops!

30 Apr 2004 | gn said...

JF, check out this page at dpreview.com

I used this side-by-side comparison when buying my digital camera and found it to be a very helpful tool.

30 Apr 2004 | TL said...

The " advantages write-up" on Mazda's site shows key differences without making you "read" a grid...

Their data is provided by the AIC.

05 May 2004 | K-Rock said...

This is a nice side-by-side. It doesn't appear to be dynamic but it is easy to read.

http://www.dpreview.com/reviews/nikond70/page20.asp

05 May 2004 | matthew said...

I hope you guys are still planning to do a 37Better version of this.