One of the things that got me excited about Apple’s multitouch UI for the iPhone is the potential for “handmade” shortcuts. I’ll call them trails.
I don’t know how this would work, or where/when you’d be able to do it, but let’s suspend those issues for a second and just talk about the potential.
What if I could make shapes or trails on the iPhone screen to visit a web site, sms someone, or speed dial? Here’s the idea:
In otherwords, make the shortcuts mine. I could make a Z trail to call my dad at his office number. Or an O to call my friend Oliver. Or a spiral to load up The New York Times’ web site. Or create parallel lines with two fingers to open up the sms app and start a conversation with David.
Anyway, just an idea. I’ve always thought squiggles or trails or shapes would be a great personal way to create shortcuts. It’s cool to maybe see that potential made possible with the iPhone’s Multitouch technology.
Luis Borba
on 15 Jan 07Hmm, very similar to the Palm OS handwriting techniques.
Eddie
on 15 Jan 07Yeah- the potential is pretty great. I think I mentioned in your earlier posts that it might help compensate for the lack of tactile feedback.
apart from the obvious (typing text), the feedback is important for those keys that are part muscle memory as well- like the send/end keys, or the “escape” key etc. If we could get a gesture for some of these types of commands that would could be executed regardless of the mode/context of the phone- that might be enough to overcome the fact that we can’t feel the buttons. I browse this way with mouse gestures and it’s nice to always have the “close” functionality available regardless of what my UI looks like.
Another downside is that I can’t see too many one handed gestures (apart from a sweep of the thumb) being used. You’re going to have to use two hands to be productive. Not to big a deal I guess.
Matt W.
on 15 Jan 07Hey, 2005 wants its ideas back. And it ask me to tell you these things are called ‘gestures’.
Bramus!
on 15 Jan 07Good idea, as this has a striking resemblance to Mouse Gestures, a commonly spread Firefox Extension.
Dan H
on 15 Jan 07Great Idea, I’ve got to call my girlfriend and tell her all about it…
Daniel Jalkut
on 15 Jan 07For a Mac-based example of user-defined gestures, check out FlyGesture from Flying Meat. I borrowed its UI in my crazy (not so crazy in retrospect) fan-fiction about an Apple/Nintendo phone, speculating that it would have gesture-based dialing :)
bv
on 15 Jan 07great idea!
Karl Nieberding
on 15 Jan 07In a multi-use device, I think quick shortcuts to common uses are absolutely key. This would be a good way to do them.
DanD
on 15 Jan 07The motion sensors could make for interesting interactions as well. You could give the phone two quick shakes to quit all applications or calls and return to the home screen, not unlike refreshing an Etch-a-Sketch.
Craig
on 15 Jan 07Check out Stroke It for Windows. I haven’t gotten around to trying it yet, but some of my friends use it all the time.
Dylan Bennett
on 15 Jan 07DanD, yeah I can just imagine that working really well while trying to use your iPhone on the bus. :)
Gustavo Beathyate
on 15 Jan 07DanD, according to Pogue the iPhone rotation only works counter-clockwise… so I don’t know if that would work that well…
Jeff Croft
on 15 Jan 07A great idea, indeed, Jason. I really hope Apple is listening to some of the suggestions/ideas popping up on the blogosphere. The potential for the iPhone to be truly amazing is so, so great—I hope it’s fully realized.
Martin
on 15 Jan 07“Trails”? What is wrong with the established name “gesture”?
Oliver Stör
on 15 Jan 07I think this is a great idea. I never really got mouse gestures – they took me longer than using the keyboard. But on a device like the iPhone they should work like a charm.
JF
on 15 Jan 07“Gesture” is fine too. I forgot about that term.
Mark
on 15 Jan 07I’m wondering how long it is before wannabes start making their icons with little red numbers in the top right corner…
Dennis Eusebio
on 15 Jan 07Great Idea. I think a solution for the loss of tactile feedback could use the phone vibration function. If any of you have used a wii, you’ll notice a slight vibration whenever you touch something that’s clickable.
They could use the same type of thing in the phone for dialing and other basic functions.
Thijs van der Vossen
on 15 Jan 07I’m still amazed that there’s no writing mode where you draw big letters on the display instead of using the on-screen keyboard, a bit like on a Palm or a Sony Ericsson UIQ phone.
Ted
on 15 Jan 07Does anyone smell a Newton like failure for the iPhone.
1. It’s too big 2. It’s too expensive 3. People will hate the touch screen for dialing and everyday use. 4. Apple will not let Cingular discount the phone, even to corporate users. Seriously, how many people will pay $500 or $600 for a phone. 5. Who cares about watching video on your phone. Yeah, that’s cool but my Blackberry Pearl also has that functionality and I haven’t used it once.
I really think the size of it will be a huge issue. How many people really want to carry, and I mean literally carry, around a phone that big.
Again, not to make too many comparison but my Blackberry Pearl is smaller than a razor and has all the functionality the iPhone has. Also, since I’m a corporate customer – guess what, I got the phone for FREE.
I’m purposefully not going to link to the bb pearl web site.
chris
on 15 Jan 07This is exactly how the Opera browser works. They have “mouse gestures” (this is what they call them) to help you get around. It’s definitely worth a look if you want to decrease your time to enjoyment while browsing. Although prepare for some of your favorite websites not to look as they should.
tom
on 15 Jan 07Gestures / Trails couldn’t possibly jive with usability… they aren’t guessable, intuitive… sure, they are a little faster, and all, but they seem wrong. It’s to easy to make a mistake, and to have a ‘confirm’ screen for an action defeats the purpose or having such a quick scheme.
Splashman
on 15 Jan 07Um, Ted?
1) Have you seen the specs? If so, why aren’t you also predicting doom for the Treo and a dozen other phones which are the same height/width and twice as thick?
2) They said the same thing about the iPod. Turns out, quite a lot of people will actually pay more for a superior product. Funny thing about that.
3) How often do you actually dial on your phone, as opposed to using a Recent or phone list? I dial maybe once a week, and that’s how I see most of my friends using theirs.
4) See #2. At this point, Apple is going for the upper end of the market, obviously, and that just happens to be where the profits are. Just like with computers, they’re not interested in selling free phones. And keep in mind this is an introductory product. Just as with the iPods, cheaper models are probably on the way. But SJ would probably barf up a lung if he ever saw his phone advertised as “Free.” And that’s just fine with you, because you’d never want one anyway, right? ;)
5) It’s not the iVideo. It’s an iPhone with enough power to play video, so why not? Did you see the part where Apple is selling kajillions of TV shows and movies via iTunes? Seems like a no-brainer to me. If you were designing the iPhone, would you purposely leave video out, even though adding it would cost little?
John Topley
on 15 Jan 07From what I’ve read, I don’t think Apple have finalised any sort of speed dial feature on the iPhone yet. At the moment it appears that you have to click on Phone and then Favourites and then pick who you want to call; not so speedy!
Dima Nikolaev
on 15 Jan 07Craig, I use StrokeIt for about a year or more – really handy, despite the fact that I use less than ten gestures. It works everywhere in a system, not only in a web browser.
Mrad
on 15 Jan 07Wouldn’t the ”||” gesture interfere with the two fingered scrolling? This is a great idea, they’d just have to figure out a way to keep your gestures significantly different that those already built into the phone.
I’d get pissed pretty quickly if I’m browsing my music and I accidentally call someone.
Vance
on 15 Jan 07Ted:
1) It looks to me to be a fine size. Sleeker than my 650, at least.
2) What phone/device are you comparing it to for pricing? Whatever your answer, you’re wrong. It’s not like other devices, so far. We’ll see if it is so great in person.
3) I use the touch screen of my Treo all the time and I love it. Im surprised people use that dorky wheel on the Blackberry’s, but that’s just me. Touch-screen is great most of the time.
4) I will, and it’s more than a phone, IMO. You don’t have to buy it.
5) I love watching videos on my iPod. When my wife or son are using the TV in the living room, I have sat next to them to watch the Office and other stuff. Not to mention waiting for flights. Never thought I would use it, but it’s great to have.
I don’t know where you are getting this “has all the functionality of” BS. That’s like saying the Malibu has all the functionality of the A4.
It all comes down to whether a divice is right for an individual. Judge for yourself when it comes out. I will.
Eddie
on 15 Jan 07tom-
Why not? Usability is about ease of use and efficient, activity centered design. It’s not about “dumbing it down” to the lowest common demoninator.1) They aren’t guessable- Fine, but they don’t get in the way for those that aren’t guessing. If you accidently fire one off, Opera asks you if you want to do keep using it, or not… first time, out of the way (or discovered on accident). Shortcuts are just that… shortcuts. Meaning slightly quicker ways to do your job more efficiently. If it was the only way to close a tab, then I’d think usability would suffer, but considering there’s about a kajillion ways to close a tab, I think usability is maintained.
2) They aren’t inuitive- that’s argueable. I say that any gesture package that allows you to “see trails” is admiting that it’s allowing you to be overly complex. I haven’t used strokeit in awhile, but I recall having to “write” letters of the alphabet with my mouse to execute some functions… I say gestures should reduced to your key browsing functions and shouldn’t be more than two movements (Like an “L’ shape), maybe three if you’re really into it (like me) and they should only be straight lines IMO.
I don’t know about seeming wrong… to me, I absolutely cannot browse without at least a “close” and “back” mouse gesture. The others I could do without, but I pull my hair out in the absence of those two.
eh
on 15 Jan 07the apple/jason friend response:
what? you don’t need this and we aren’t going to do it, because you’ll be happier without this feature. we know what you want and we’ll tell you what you want. and the world is better off that way. also, you can’t program this phone or mess with it no matter how much time you have on your hands and no matter how technically skilled you are, so you won’t be able to make this happen without a soldering gun and voiding the warranty. and you’re gonna pay through the nose for it. i can barely conceal the fact that i know that i’m a lot smarter than most people.
think different.
Piper
on 15 Jan 07Yea, I am fine with the size as well. But who knows! By the time they come out with it, it could change.
Eddie
on 15 Jan 07Oliver Stör-
I’m quicker with the keyboard too! ....when my hands are on the keyboard. And, although my browser has great spatial navigation (Opera) via keyboard, I still find that most of the time I’m “driving” with the mouse.
...in general, I try to be most effecient and do as much as I can with whatever is under my sweaty fingers at the time…. when I’m in the process of entering a URL or search phrase, I do as much as I can with the keyboard until I find myself reaching for the mouse to navigate, click, scroll etc.. then, once the mouse is in hand- I do as much as I can there before i return to the keyboard. Gestures are the best way to do that.
Thomas
on 15 Jan 07It makes me laugh that it has taken Apple this long to release an embedded OS.
People are all freaking out and getting excited like having an embedded OS based on your consumer OS is a huge deal.
Dare I say it. Here it goes, WINDOWS CE. And when did it come out, 2000.
Eddie
on 15 Jan 07I keep seeing this all over the place along with the “don’t tell you dail your phone without looking…” argument. Ok- you got us. We don’t dial every number, and we probably do glance at our phone when we do dial. That doesn’t completely disavow the need for tactile feedback. I don’t dial numbers, but the volume, end/send, vibrate, silent, escape, d-pad (or trackball in the case of my pearl) are all used with a rough, casual glance… muscle memory takes over from there with tactile feedback filling in the rest.
...as I said in my first post though, I think it’s possible to compensate (as someone mentioned with the wii vibrate), but please don’t casually dismiss tactile feedback & muscle memory just because you have a speed dial for your local pizza delivery joint.
George
on 15 Jan 07nice, reminds me of an idea i had a while ago which used this sort of idea of gesture/shape drawing as a ‘password’ (or pass-shape, as it were) which could be used instead of a standard password, for low security applications.
m
on 15 Jan 07I’m pretty sure apple is thinking of this (or has already thought of it) as this is a common feature on tablet PCs and windows mobile phones with touchscreens. Microsoft has a guide for developers on how to create custom gestures for their apps. Gestural / graffitti input is also incredibly useful in countries like china where our romanized keyboard layouts don’t make much sense.
Elliott R
on 15 Jan 07Hold on. I’m a bit slow here but….with the multi-touch screen technology why the hell arnt they using that to enter text? surely it wouldn’t be so hard to get handwriting recognition into the bundle.
George
on 15 Jan 0736 replies and no one has yet mentioned Black and White?
Released in 1999, it offered exactly the interface you describe, except for a computer game. Reasonably successful. Poor game underneath though.
Grant Hutchinson
on 15 Jan 071999? Black & White? Feh.
The Newton OS had programmable gesture recognition back in 1994 – optimized using Five Speed Software’s Dashboard in 1997.
Claus
on 15 Jan 07Neonode actually did something like this a few years ago… They made a proprietary version of windows CE with fullscreen gestures for quick navigation… it’s actually similar to the iphone in many ways…
Splashman
on 16 Jan 07Eddie, I’m not dismissing the need for tactile feedback. The lack of same is one of the few disadvantages that almost everyone will agree on, including me.
My point was that if you don’t dial much, the lack of tactile feedback is less of an issue. Obviously the lack will still be a problem with other operations (end call). Notice, however, that there are a few physical interface elements—volume control, for instance.
And the larger point, that I think everyone will agree on, is that every approach to an interface involves trade-offs, and no one interface is going to be best for everyone. You simply can’t have a touch interface AND a physical keyboard AND a large screen inside 3 cubic inches. Physical buttons have their limitations, as enumerated by SJ in the keynote. The iPhone’s touch interface has its own set of limitations, as ennumerated by many, many bloggers. SJ is obviously selling the notion that the touch interface, on balance, will work better for a lot of people. If you asked him whether a fast Blackberry texter will be happy with an iPhone, he’d probably say, “No, of course not. Stick with your Blackberry. Loser.” (That’s conjectural, of course.)
I’m guessing that I’d like an iPhone, but there are other limitations to consider (Cingular? EDGE?), and I’d have to play with it before I’d plunk down my money. If it doesn’t appeal to you, fine—just try not to over-generalize based on your own usage pattern.
Christopher Fahey
on 16 Jan 07Thos of you who think Jason is unoriginal because “gestures” existed before are profoundly missing the point. The idea of using one’s finger to make one single giant gesture on a portable device screen to trigger a simple, immediate action is a really clever and brand-fucking-new idea.
That’s what innovation is—building on existing knowledge to create something new.
knubile
on 16 Jan 07You can play with this concept now in a program called StreetDeck from mp3car. I have this running on my carpc now and gestures(you called them trails) are my primary input method. Its a great idea. Cheers
Riddle
on 16 Jan 07Believe me or not, I was thinking about the same thing while watching another video with another conclusion, that iPhones need more tactile feedback.
Gestures, in simple words.
We have mouse gestures in browsers and sophisticated UI in games (vide Black & White).
Great idea of course, I hope it will be introduced in iPhone ASAP.
David RealNameUnknown
on 16 Jan 07Gestures/Trails/Finger Squiggles (whatever you want to call them) rock. I use Maxthon and mouse gestures save me a bunch of effort/clicks/movement. The more apps that use them the better.
tom
on 16 Jan 07I defy you to make that large Z with one hand (thumb). Jobs uses his iPhone like a tablet PC; only my dad dials a cell phone with his pointer finger.
Sebhelyesfarku
on 16 Jan 07Your dad is a proto-Maczealot then.
jay
on 16 Jan 07wow jason, you finally understood the concept of gestures. WOW! So you see – you dont even need to invent a new word for that (trails…).
Greg
on 16 Jan 07Ugggggg, I absolutely hate mouse gestures. I was actually joking w/ someone about how I hoped the iPhone would not implement gestures. Such a bizarre and abstract form of UI. “If you want to go to this page, wiggle your mouse just so”.
Eddie
on 16 Jan 07Greg-
Keeping with the unorthodox notion of “going to a page” via a gesture, How is that different than clicking on a bookmark? Aren’t you essentially “wiggling” or otherwise maneuvering your mouse over to you favorites panel and clicking?
If there was an odd mouse gesture that closed a window by having you click a button, then move the mouse to the upper right corner of the screen, how would that be different than moving to the upper right of your screen and then clicking? With mouse gestures Fitt’s law comes in to play in that you don’t have to move your mouse to the edge of the screen- it’s already in the position for command execution. Seems like an efficient UI to me.
Just because some gesture packages allow you to write the letter “Q’ with your mouse to open quickbooks or something doesn’t mean that’s what mouse gestures are for.
Tobias
on 16 Jan 07Gestures are really great, I can’t even browse without mouse-gestures anymore.
Have been using them with Opera since at least 2003 I would guess (yes, they are no “firefox-extension-invention” ;-) ). Although you have to limit them to a few, otherwise you have to be really precise about them and the intuitiveness to it gets lost.
Also, customizing them isn’t the best idea because other people using your interface will get confused. My colleague e.g. always closes windows by accident on my PC. (ok, nobody would give away his/her iPhone to anybody)
Bob
on 16 Jan 07Christopher Fahey -
Let me guess, you work for the U.S. patent office? :)
Except that’s exactly what Apple has demonstrated already with their zooming/unzooming gesture. And like everything else in UI, multi-touch interfaces have been researched and developed in academic circles for years before this commercialization by Apple, with the state of the art now well beyond just simple gestures. That’s what innovation is—making money from well-known ideas by commercializing and productizing. And no, there isn’t anything wrong with that, but let’s not get carried away with our own “originality” here just because we don’t know what’s out there.
Kula bácsi
on 18 Jan 07Multitouch, large gestures = no one handed operation.
JeffreyR
on 20 Jan 07I agree that this feature would be an excellent addition to the iPhone. The simple “unlock” gesture already exposed, rang a bell for me from a science fiction novel where the character “signs in” to her handheld computer by drawing a simplified flower. So as George mentioned above this would be a great way for the iPhone to be more secure. OS X already allows assigning quick keys, adding an assigned trail or gesture would be a natural.
As far as the “me firsters” are concerned I read this book in 1992 (I just wish I could remember if it was from the Ender’s Game or Mote in God’s Eye series; would be a much stronger point :^]).
This discussion is closed.