The Science of Aesthetics by Keith Lang (video) is a talk Lang gave at UXAustralia in 2009. In it, he talks about how some shapes are naturally friendly looking, like the rounded rectangle, while other shapes are harsher and more unfriendly. Here’s a cool example of how a UI could take advantage of this by using an aggressive, spiky shape:
Interesting idea. People would definitely think twice before clicking a button that looks like it’s going to carve up fingertips.