Could you please enable download of this video? I'd like to watch it on my commute home :)

Cool concept until con10uum... you make the (wrong, imho) assumption that arbitrary window positions equal clutter and not functionality. This probably doesn't go over well with any power users I know, where windows are often purposely tiled in arbitrary ways, sometimes even almost entirely overlapping, to better view large amounts of information at once.. side by side on an 1D plane would be extremely limiting if you're looking at more than two things at a time.

Also, how do you factor out the keyboard?

I love this and the interface is beautiful, simple, and futuristic. I'm in love with the desktop itself and the Global and Local menus. I'm not sure about the con10uum though. I mean, as long as you'd be able to make a window consume the whole screen I'd be happy but that's something I don't like about Mac. You can't really full screen it. I do like the annotated thumbnail view though.. It seems I find many more pros than cons with this so I want it immediately. :)

I see two things which makes con10uum unacceptable as it is shown. First, I want to have windows which are visible all the time. Second, I want to switch very fast between certain windows. Is that possible?

I like a lot of this in theory. It seems to combine the strong control over workspace native to OSX with Windows simple focus on a single task (assuming that windows can be maxed to fullscreen- not demonstrated in the video). I would suggest that replacing the keyboard, with a 10Gui haptic 'force feedback' touch surface would be more ergonomic and practical than sticking on top of the 10Gui interface.

I find the subject interesting, but I find it hard to concentrate with the music in the background. Could you make a version without the music?

Great design, but I would also agree that the lack of arbitrarily placed windows is probably not a good match. I also don't particularly like the awkwardness of having to pinch the windows to fit a certain amount on the screen at the same time. Although I don't have a suggestion for this. I do particularly like Apple's approach using a gesture to zoom to a thumbnail view of all open windows in one screen, personally I use this a lot. One thing I have been thinking about in terms of organizing windows is to remember the time they were opened and, maybe with a gesture, open a view similar to Apple's time machine (awesome idea too) but just for the windows you're using now. I tend to remember the order in which I open windows and it's usually purposeful and related to what I'm doing. Another thought is grouping windows into logical task groups.

Pure Genius, Clayton. Period. One might further envision another gesture that provides a Z-axis rolodex-like "carousel" of linear con10uum's in fully zoomed-out mode for the most ultra-uber-meta-multi-taskers ... or for stored default con10uum linear arrays. Also, it's design genius to take one step at a time ... get us beyond the mouse first, help the humans get comfortable with multi-touch; keyboard emulation can come later, if ever, for by the time we can get 10GUI into default production lines, we should have, once-and-for-all, Voice-to-Text capabilities that practically eliminate the keyboard in most settings.

One traditional obstacle for voice input is that it's too noisy to use voice in shared work environments; however, noise cancellation is very close to obviating that complaint and should be able to fully solve that challenge by the time 10GUI ships as the default peripheral with ever Dell. Personally, I'd like to see that happen no later than 2014; although, really there's little or no excuse to not start seeing 10GUIs rolling down the production lines in equal numbers to declining mouse populations by 2012.

The only holdup is Human Behavioral Inertia. I remember people looking at me like I was the biggest fool on the planet for wearing "that stupid looking cyborg thing" on my ear. Today, only a fool is found NOT wearing a bluetooth headset.

So, Just ... Keep ... Going ... and please do put me on the list for prototype field testing! I'm anxious to start acquiring this next essential skill, NOW! ;-)

I think that it should be implement it only as a hardware device and adapt it to the existing OS's on the market.
Then it would be brilliant! :)

This looks great! Getting the ergonomics right on a 10GUI/Keyboard combination is going to be key to make this widely acceptable. This will even be even more challenging for a laptop.

It looks very nice, I’m interested to know if the sensor only detects finger pressure would it have problems with palms touching the surface? I’m also not a fan of “pinch-zoom” (I hurt my hand ice skating and that injury makes it painful to do fine finger movements) It might be nice to have regions on the touch surface that can be programmed to zoom automatically and carry out multiple tasks. (i.e. quick touch button) I think you are moving in the right direction very nicely done.

Nice design

Definitely an interesting approach. I've always loved alternate style UIs.

Some computer games do an interesting job as being a medium for new UI designs and concepts. This kind of interface would be interesting to use in a game that had you deal with cards or lots of "tabs" of some kind. Of course it doesn't deal with the hardware required (touch pad).

The one thing that really stands out as a major flaw is the requirements. Specialized touch hardware and 10 fingers that you can use. Touch pads will probably get cheaper, but a noteworthy percentage of users will be physically unable to use this interface. This is a big issue.

Very cool ideas, but what about handicap accessibility?

This system seems to cater only to 10-fingered, sighted users. While the linear workflow could really help blind users navigate with screen readers and voice input, other users with motor problems or missing fingers/prosthesis might find it too challenging to use effectively.

It doesn't look like I'd be able to perform day-to-day activities in 10/GUI with just one finger; making allowances for that would break the entire paradigm, no?

Great presentation regarding issues you would have working on a touch screen. Your proposed solution is good and aside from the possible issues that would be found during user test I think this is a great direction.

I'd love to see this, built into a mac for example. It would be a killer device, and the workflow will step one or two levels forward.

Congratulations and I hope it gets real soon.

Love it, when and where can I get it?

Looks great, hope to see it soon! At least in my lifetime

i suggest a version just for the right/left hand side (or both sides) to replace the mouse, gamers wont like the in front of the keyboard panel. i can see how this could replace the mouse.. keep the pad, use your fingers instead of the mouse :)

toosas: I agree, compatibility with keyboard usage is important. But as I didn't see any two-handed gestures in this video; it seems like it would work equally well with either hand singularly while leaving the other on the keys.

As for accessibility, I don't see it as a huge problem, as this system is mostly adding new input methods to speed up the interface. Users without full use of all their digits for multitouch gestures such as pinching for zooming or touch-finger-touch for switching applications can use keyboard shortcuts, just like all users currently use.

I think the funniest/saddest part of all of this is the way people are so quick to criticize a brilliant, innovative new idea. I guarantee you none of these critics have ever actually come up with an original idea of this scale, let alone implemented it. Great work, man. Brilliant stuff. It's a shame that there don't seem to be any real innovators working for the major computer companies these days. It's as if all of the innovation stopped in 1984. I, for one, am sick of the mouse!

There are some great ideas here, but the prevalent sideways text implementation is a deal breaker. You should rework the interface so that the text is in, you know, the orientation that we read.

Personally, I've been moving towards full window usage. I believe that every app should take up the entire screen. Distractions lead to... distractions. Give me full screen apps, and then come up with incredible ways to switch between apps and transfer information between apps.

Finally, we need keyboards. But there's no reason that the keyboard couldn't be the interface as well.

Great concept!

I was fully expecting him to take it another step, and turn the entire surface into a virtual keyboard with haptic 'force feedback', as dbspin suggested.
Then what sits on your desk won't be twice as large as your existing keyboard. You call up the virtual keyboard with touch bar (top of the control area? right=global, left=local, top=keyboard/control?) which could also be toggled within context. Like the control area automatically goes into virtual keyboard mode when you 'click'/select into a text field. In fact, you could just print the outlines of the keyboard onto the control area (not print the actual letters and numbers) so you can see where to put your fingers - and you can easily redefine the characters on the keyboard for alternate alphabets (European/Asian) and/or arrangements (Dvorak). Tiny physical bumps on the control surface for the "F" and "J" 'keys' may still be a good idea so people can find the 'home row' without looking.

I too was disappointed that he didn't cover alternate input methods for the handicapped, many which would still need 'single point' methods of input, and/or a method of movement around the interface for the blind. Could I navigate the interface using just the eraser end of a pencil? But fortunately, his concept appears to be easily adaptable to add such capabilities! That blank keyboard I mentioned above? Add a row of assignable function 'keys' above where the standard F# function keys go, for all the typical interface movement/interactions (zoom in/out, scrolling screens left/right, etc.)... Heck, even the fully 10 fingered folk may want such shortcut 'keys'.

Such an expanded virtual keyboard may even make the control surface closer to the same size ratio as the wider 'HD' screen ratios are these days.

This is a novel solution to the modern touch screen displays we are slowly adapting to. The only vice I hold with this concept, is the giant "pad" needed to include the control mechanism and the keyboard. I would want a system in which the touch pad actually displays the keyboard when I'm ready to type, activated by the OS, or possibly activated by a standard tactile button on the side of the pad.

I would also like to add that the limitations of the horizontal application system is still too limited. I can understand the ease of use this way, but why not instead add that second dimension back, and only restricting the use to either the x or the y axis instead of both at once. Visually this could mean having the same horizontal layout only this time per application. And each application is stored on its own "layer" vertically. This would better organize the applications, and lead to even quicker and easier management of the same system. Adding the little bit of edge space to the top and bottom, and removing the tactile keyboard in favor of a touch based via the same touch pad, would greatly enhance this concept.

man that is a great concept, you should really pitch your idea investinmyidea.com/ that will be sure to get some good interest!

Great work! One of the beste multi-touch concepts I've seen in a while.

Was this a private project or what's your lab?
What about an open source project? Having one of those new Wacom bamboo pads in front of the keyboard or maybe even my using the macbook pro trackpads and a modified ubuntu or something could do the job.

This isn't any sort of giant leap - Macbook laptops already come with big beautiful touchpads which detect up to four fingers and can do most of what what shown already.

Very bold and outrageous ideas. I love the fact you've sparked so many people's interest. Whether they are for or against your idea.

This is something big. I'm ready for it.

I hate to criticize such cool ambitious ideas but...

Let me look closer at your design exercise

The Good:
1) Foresees a future in which finger tracking pads can detect hovering fingertips with decent reliability (this is very difficult to achieve). This allows pointing at random at multiple targets without needing to haul a cursor across the screen. In other words you have the benefit of current touch screens without the drawback of body fatigue.
2) Takes the traditional laptop trackpad and expands it to accommodate two hands, allowing for simultaneous manipulation of more objects.

The Bad:
1) Your use cases do not envision much new functionality for multiple fingers/2 hands beyond manipulating windows and repositioning things (which is done relatively infrequently in day-to-day use)
2) Finally, The most glaring problem for you:
-Your tracking pad CANNOT BE 1:1 the same SIZE as your MONITOR, because that will ostensibly leave no desk room for a physical keyboard. (imagine scaling to a 24inch+ display, let alone even 13 or 15 inch). If it's so hard to get people to ditch physical keyboards on their cellphones where they don't type much, imagine trying to get rid of the keyboard where i type my term papers and whatnot.
-So you will HAVE to have some kind of distance multiplication algorithm going on, just like on current laptop trackpads, where one unit of finger movement results in less or more than one unit of cursor movement, depending on finger speed. This will make for interesting on-screen interactions-
-As it is, on traditional touch screens, buttons have to be made large to accommodate people who don't have surgical precision in hand placement.
-Imagine scaling the shake and clumsiness of a hovering finger onto a larger plane (the monitor) and trying to get fine control, for example, clicking or selecting small things on screen.
-This cannot be easily overcome, because when you use hovering fingers as your cursor, each spot on your trackpad corresponds to one specific spot on your monitor. You are stuck with (this may be the wrong jargon) absolute, and not relative control. You would have to do some fancy math to smooth out fine control without making cursor behavior unpredictable and annoying, this may or may not be practical.
-Unless you only have 1:1 size ratio between screen and trackpad, multi-finger operations get tricky. Imagine trying to squeeze two fingertips together to make two screen elements touch each other,and still having a 1cm gap on screen because you cannot get your fingertips close enough together.

Sorry to argue with your fine ideas, just had to say it. Great job on your presentation video though, and you're thinking about a very important and exciting problem, which is very cool. Best of luck, feel free to flay my critique.

I want it now on my next Macbook Pro. And I want Apple to adopt this paradigm for their next OS release!

Very interesting concept. There's a lot more ground to cover and expand, as it is a very radical shift in user interaction, for example: How precise does the finger clicks need to be done?, how this would work on users with a two-screen setup?, Wouldn't this become too cumbersome on applications with multiple tabs/palettes/windows (such as Adobe Photoshop)?. I would love to see this idea being worked further beyond. Keep it going!

Great idea! When will this be available?

Great idea. Great presentation. I like it very much!

Of course there are some things other mentioned which are good remarks and should be considered. I wouldn't tell those "criticism" - just feedback.

From my point of view (using an iPhone and get used to the touch keyboard very easily) - why not make an additional button on the lower end of the interface which opens a touch keyboad overlay?

Regarding those who like arbitrage windows, or are challenged with precises finger movements (hope this was formulated politically correct - I am not a native speaker ;-) ) - let them just stick to other operating systems or other GUIs! Why should everybody use the same UI?!

So, please go on with this concept, I would love to see it live and test it!

a) disabilities. software can be adapted for different numbers of fingers. hardware and software together for missing hands and having issues controlling movement. to be honest, some people just can not do the required actions to use a computer and require specialized systems. adaptation of the interface is all that is necessary, and come on who knows of any wide-spread operating system that by default has good support for disabled people apart from magnifying glass and "SAM"....

b) multitouch - i hate the mouse, always have. this idea makes use of more fingers, incorporate a keyboard activated by palm or finger touch and you have a pad doing the job of mouse and keyboard, occupying far less space and potentially being much more comfortable and natural to use. fingers are made for fine movements. they are not clumsy and you do not need great precision in your fingers for them to be well suited to the task ( you need look no further then mobile phone buttons, i have many friends who can type on a standard phone (no qwerty keyboard, just 1-9 with each button assigned letters) more then fast enough for it to be a convenient method of communication. as for hovering and all that... oh come on we all use keyboard and look at how well we can type with great precision with minimal error. now scale that ability to knowing where your finger is at all times and you suddenly have fewer errors :O :O

a prime example of the next point is the wii. you CAN learn to accommodate shake of hands in general. so you can definitely learn to accommodate for the slight shake of fingers.. if there was enough to be noticeable

and just as the mouse must be raised and moved before being placed upon the mousepad to continue using the mouse many times during ordinary computer usage (and it does you can not deny this) you can just move your hand.. if the pad is all up the size of a standard keyboard or slightly larger, you will not have to move your hand far. because you are using a touchpad *perhaps with an led wafer display* you can rearrange a keyboard to fit you, add custom buttons etc.. and position them wherever you please you can make buttons closer together.. smaller.. larger.. further apart, whatever the user is comfortable with. and perhaps some of the customizability can be saved to an online profile (i.e. igoogle and its widgets) accessible anywhere with a net connection, any computer would be comfortable to use. i hate using keyboards different to my own, and sometimes mice because they feel different.. imagine changing that?.

the operating system would have to be very customizable to make good use of the 10gui, and perhaps open to development by the entire community to allow creativity to come up with new applications for 10 fingers. most of the issues brought up in this huge thread can be addressed with a few additional settings for users to choose - having a high resolution pressure pad along side a decent resolution thin display (and before anyone mentions it breaking.. remember anything in a computer can be snapped... peripherals included) along with appropriate software could allow for interactivity upon the keyboard/mouse(s). any number of pointers could be enabled disabled?

so basically, i love the idea, it is extremely open to expansion because it only builds a base structure. perhaps all the useful suggestions and constructive criticism given by the community could be incorporated and addressed into prototypes of such an operatingsystem + pad?

beautiful. the paradigms suggested make for nice 'light' interaction which is common with people trying to employ multitouch in the current age, whilst also adding a new level of detail.
some fixes need to be made for power users, but i think developing the paradigm and making people get used to it is a much better approach than quick fixes like keyboard shortcuts or introducing elements which fit unnaturally in the paradigm

Nice, very nice! Love the interaction vocabulary that is set up.
Similarly, but on another scale, you guys should see LAb[au]'s "Touch" project: replace this LCD screen by a light tower building downtown Brussels !

lab-au.com/projects/touch/

Pretty neat!

The hardware sounds like basically the Touchstream keyboard ... I tried one out a few years ago.

Linear order of windows - interesting idea. Windows already imposes a linear order (Alt+Tab sequence), allowing you to find your way quickly, though it doesn't let you manipulate that order. From other comments, it seems like the jury is out on whether restricting placement on the screen is a good thing. I think in most cases it could be, especially if you have some freedom to arrange buffers *within* an app two-dimensionally.
I was expecting, when it zoomed out the annotation space, to show a set of several application spaces (streams) that you could switch between, for a sort of two-dimensional-but-more-organized working set.

What about the keyboard, when you need to use it for typing?
OK, the keyboard is behind the huge "mousepad". It's like having two keyboards, space-wise.
If you're typing a lot, do you get fatique from keeping your arms stretched across the large mousepad?
Could you have a single device (like touchstream) that toggles between mouse-like device and keyboard (like Touchstream does... I forget how it manages that)?

Would be most commercially viable if the hardware is decoupled from the software. You'd have a hard time selling both at once because it requires mfrs to take a gamble on the hardware w/o the software and behavior patterns being established in the marketplace.

You could envision some con10um-like "window manager" enhancements to Windows right now that could be offered w/o requiring hardware upgrades. I would like to try that.

I agreed w/ someone else that the music threatened to overpower the narration at times.

Anyway, neat ideas! Keep working on it!

@Stefan Goodchild.

"Found myself trying it out on my desk while the video was playing"

Me to :)

so basicly... a multi-touchpad for desktop?

Has for the keyboard, can't believe you didn't suggest a OLED screen on the touchpad that could show user info on what they're pressing(like a keyboard) that would only appear when the keyboard function is required.

This way you could get a custom keyboard for each apps. No more keyboard shortcut position limitation, has you could reposition every key on the screen.

For the poor's pad, you could put something over the touchpad with pre-defined key position. Like the current keyboard dresser you get in some games and apps.

Now on the software note, it is perfect right now. With few minor adjustement, everything could be adjusted

Amazing!

What if the touch area was split in 2, each sideways of the keyboard?

As people were discussing on AIfIA, the single area has some limitations: (1) brings the hands to an unusual position, as opposed to the mouse place, (2) limits movements by the crossing of hands and fingers, and (3) as Frederick van Amstel also said, it forces the keyboard away, which is ergonomically bad.

A solution would be 2 lateral areas to the keyboard, like 2 "mouse pads". This would (1) bring both hands to the natural and loose mouse position and (2) transcend the physical limits of matter.

Crazy?

"Found myself trying it out on my desk while the video was playing" - can´t stop!!! =)

Wacom, you better get on it while you have the chance!

Congrats! You just invented the Segway version of the mouse. So cool that no one uses it.

Learn from Edison:
Edison's first "legitimate" invention was an electric vote-recording machine which, despite its brilliance, was too far ahead of its time to be marketable and did not sell. Edison learned a valuable lesson in marketing, and from this point on, he vowed he would "never waste time inventing things that people would not want to buy."

Very interesting idea for a summer project, always good to see people thinking beyond normal convention. My two cents, the keyboard should be built into the touch screen and use an eight finger gesture to bring up the keyboard, I think we are all trained to move our hands to a home row position to begin typing.

not sure how the small windows would help, too much on the screen would distract me. nice try though

Very interesting - thanks

isnt this y they made the TaskBar?

It seems to me that most of these concepts are already working on the Palm Pre. Gesture area beyond the screen, sliding app management, multitouch gestures. The OS for this is already there... I do like your point about the positioning of the input device though.

Easy enough solution: make the touchpad another "touch monitor" so it can mimic the main screen, and using another's idea, have it so a sequence brings up a virtual keyboard (touch sreeen side only) to type into the text fields. I like this idea, but it needs a few revisions- but this can come AFTER first release. Being an MMO enthusiest, I know that a LOT can be changed after firs release, and this is one of those technologies that need change. Awesome first concept, though. Now- for the common person, you will need to set this up to work with gamers. Yes, gaming is a "waste" of time - but it is what seems to be driving a lot of technology's edges. maybe have it set up to interface with current programing mouse detection so that response time can be moved back to a linear set of inputs, but with the multiple input feel (action bar can be "clicked" with one finger while the other hand guides the avatar, as a real mouse would) - kind of like how original gui's looked like they multitasked, but really were more linear, with a fast response time. This can then be bumped over to regular applications so that all new applications would not have to be re-programmed to work with this new interface. Furthermore, it may mean that you don't need to change the OS – just have this overlapping your original Os for the Windows die-hards/vs Linux, vs. whatever.

Later in the over all modifications, the "touch pad" can start being contoured to better respond to the ergonomic needs of normal hand positioning (yes its not a perfect one to one, but I'm sure a slight slope may be better than a flat surface) - or make it so that the board can be "broken" and put back together so that they can fit a slope, or be a flat surface. - Furthermore, this "broken/reconnection" may allow to put an extra pad overlapping/ a 4X4 square, to allow for more intricate usage (another complain another poster had) - so now instead of having to zoom to work on a 1:1 ratio, you can have a 3:2 ratio for more detailed applications, then break the touchscreen down back o a 1:1 or a 2:3 ratio again. Something to think about. one last carry on: the separate part of the touch screen can then be used primary keyboard touch, while the other one goes back to your original idea. That way, the "breaking" is only a virtual one. As for the inclined slope - a small built in tilting device under the keyboard - since the touch pad may be able to be forming or bent (or at least made that way, with this intention) -

hope this helps.

oh ya- i hate current touch pads/synaptic on current laptops - because my hands at times get sweaty at work, or can transfer oil - which would look horrible on a touch screen over time, and makes the pointer "jump" - maybe invent special "touch finger tips" or gloves to increase screen cleanliness and longevity...

great video, great concept. ditch the music, makes it hard to hear the narrator.

Love the idea. A few thoughts:
Someone offered the addition of a pen for fine input. Rather than that, what about a fixed 'pen' gesture. Try this, hold an imaginary pen in your hand and then rest your hand on the surface of your desk. Feel those contact points? What if the system would recognize that gesture pattern and switch to a single 'pen' pointer. Open your hand again on the surface and you are back to 5 input points.

Pinning gestures: What if you could hold a window in one hand and then use the other to flip through the other windows so you can bring the two you want next to each other?

Overall gesture support thoughts.
5 fingers pinched to a point: Minimize/Fold application

10 fingers pinched in to the center: Minimize/Fold all applications. The con10um would then look like a book shelf with applications as 'book spines'. For those who may have LOTS of windows open it can make it easier to switch activities.

Keyboard thought:
If the keyboard is going to be a physical one, and I think it should, most of the time you will have your 'off' hand (left since I'm a righty) resting on/near the home keys. I'd say 70% of the time or more you would only be using 5 of the 10 inputs.

I heartily applaud the aims and the design, but note that there is nothing *requiring* the 1-dimensional window arrangement of con10uum. This could just as easily be optional, allowing power users to utilize 2-dominsional window arrangements. Every other aspect of the interface owuld remain unchanged.

Very cool. It's like taking a pen tablet to another level. I think it would be awesome if you could get the surface to mirror the desktop for certain applications, and like the suggestion of incorporating a pen for fine input. You're right that a touchscreen can get messy (not only blocking your view and tiring your arm, but all those yucky fingerprints), but for certain applications, such as retouching a photograph, pen tablets that are screens in themselves are awesome. Incorporate that and I'm sold on the hardware part.

For the windows... I'm torn. You do need to have tiling. I set up up to four programs side by side sometimes. Perhaps you could have two or three different possibilities, and let the user choose.

Question: what do five fingers do?

Sounds pretty cool, I'm not entirely sold on the con10uum but having it available as an option would be fine for most users.

I think you should have either a gesture or toggle that would have the surface display a standardized keyboard on it; perhaps using either an LCD, e-paper, TFT, or OLED display, depending on which interfered with the experience the least or kept production costs feasible.
Though if that would increase production costs too much perhaps the toggle would place the keyboard on-screen in software and it could function as a stand in, maybe not a permanent replacement, but a pretty cool option, and software might be easier to implement.
I merely mention it as many people still do, and most likely always will, look at their keyboard while they type.

Apart from that, pretty sweet stuff. :p

Great concept, very interesting. I would like to see two separate input devices though, standalone. This would allow each one to be put to the side of the keyboard in the same relaxed position the mouse occupies at the moment (but on each hand of course). This is how I found myself visualing the concept anyhow.

No

I'll be brief.

Touchpad = Keyboard.

Or at least the option. Oh, but you're afraid that children won't be able to adapt quick enough? They're children. They can adapt to anything. Less desktop clutter - and us veteran's will have an easy, easy time adjusting. I could type mid-air if the hardware / software options were available.

Excellent!

As others before me said, "Found myself trying it out on my desk while the video was playing"...and smiling!

Not sure if my default would be to open all apps at full screen, knowing that I'll often have multiple windows open simultaneously.

My first reaction to linear display of new windows was a small let down - however the demo of shrink-to-fit with page titles below allayed that disappointment.

Without having read all the myriad comments above, there's certain to be others who, like me, would really appreciate a 3-D interface to take this concept one step closer to "real life" manipulation.

I am absolutely anticipating the day I first get to use this technology!

Thank you.

The confusing display is easily fixed by the PC Cascading Display option. I only see this as a touch screen mouse pad, not a keyboard. Even if you could turn it into a keyboard, finger placement per key would be extremely difficult. It does have it's Apple Touchscreen benefits with zooming and moving app's around. Just don't see the pro's outweighing the con's though.

And Matthew Jackson, bull crap you could type midair. You think it would be easy but it's not. Just 10 mm to any side Matthew and your entire key range is off.

Put simply, I have a Radeon 9000 Pro Graphics Card I bought over 6 years ago and it support two graphics displays no problem. I've already solved the problem of too many open applications with this option. But, ofcourse, not everyone has two displays nor a dual output Graphics Card.

I love the mouse idea, but I would combine it with being able to bring up a keyboard by gesturing at the bottom of the screen and replace the keyboard also. This could also eliminate all kinds of ergonomic problems if the key spaces could be configured by the user. I like the option of being able to display windows as suggested, but I often intentionally overlap windows to be able to look at the data in one while interacting in another. Being restricted to one horizontal stream is not optimal.

Very well thought out. Probably makes the most sense in an ultra-light MacBook Air type of laptop, replacing the keyboard with one large touch-panel that exactly matches the screen in size.

Your ideas are innovative and a welcome fresh perspective on something many of us consider to be a necessarily evil aspect of computer interaction.

If I may, I would make three requests - as I'm sure better minds will have mentioned.

1. I often have too many windows open to accommodate a linear tiling. Given your use cases and gestures, I support your unwillingness to move past one dimension, but with thirty or more windows open I would appreciate the ability to add rows of applications (much in the way that Windows adds additional rows to the task pane).

This would allow for each row to group windows from a single app and address a previous comment that your text is vertical.

2. I'm sure that you have mused about adding a touch keyboard to your interface, but I applaud your resistance to such an endeavor. That said, I would find it useful to have a single line of buttons (even four or five 1cm blocks at the top) which support user customized functions or app calls.

It would be my hope that such an addition would address the 30% of the time one must move to the keyboard for only a few clicks ( Ctrl-C, Ctrl-V, etc. ) and thereby minimize the frequency of moving from the touch to the keyboard and back. (ala minimizing the interaction based context switching) This would also reduce the need for more complicated gestures (& learning) on the part of the user.

3. More of a musing than a request, as we move to more highly parallelized hardware, I envision extending your modifications to things like the "Open File" dialog. Where a user is instead presented with a Rolodex view of waiting windows which contain previewed files that can be dragged with the non-dominant hand into your active application context. (Clearly there would be security implications of auto-preview all files, etc.)

In other words, with such an elegant way to organize multiple windows why should one ever close anything? Why not instead have every possible window waiting at hand to be added to the active application context?

I would welcome further discourse.

- Matthew Tibbits

Pennsylvania State University
Department of Statistics

First: It is such a relief to see that you *didn't* fall into the all-too-seductive trap of adding a touch keyboard to the system. A normal keyboard has tactile feedback, lets you rest your fingers on the keytops without registering keypresses, and can be used efficiently without looking because the shape and location of each key can be felt with the fingertips as the user types. All the commenters asking for a keyboard as well simply haven't thought it through -- and certainly never had to use a tablet on-screen keyboard (or a laser keyboard or a cloth keyboard, or all the other kinds of non-tactile keyboards out there) for typing anything longer than an email.

Second: How do you expect to handle users keeping their fingertips above the touch surface at all times? Even when typing, people will occasionally rest their fingers on the keytops to relieve the stress on their wrists and forearms. The touchpad should somehow allow light touches (resting fingertips) without registering them as presses or clicks. This, to me, seems to be the biggest obstacle in turning this demo into an efficient input device.

I think your work is excellent, and I'd like to see it realized. Even if I'm partial to the several-app-windows-overlapping method myself for certain situations, I could definitely see myself switching to the 1-dimensional window layout for most day to day use. Add a few iterations of user feedback and I'm sure even my overlapping-window-situations could be accomplished as well or better using a variation of your system.

I think this video is very brilliant in the idea, visual content, speech content (and even the voice), and of course the music. It's perfect! And I love these pieces of music so much that I have to ask you their names. Could you please tell me their names by replying me or sending me messages, Mr Miller? Thank you very very much:)

sounds awesome!

thank you for your work. you are awesome.

amazing how practical it seems, this is definitely what I would expect of Macs next OS... true evolution

Great idea for a new GUI. With a bigger iPhone this kind of use of an interace would be possible. That would be nice.

Good idea, but not great. This is definitely not a breakthrough in desktop computing. I don't see any solutions presented. All you did was migrate the interface to a touch interface. All of which can be done easily using current keyboard and mouse devices.

I never buy a mouse that has no thumb buttons. I would love to get rid of the scroll wheel and see a scroll ball instead (like Apples mighty mouse). Scrolling can be done using the ball, zoom can be done using the thumb button and ball (horizontal for horizontal resize, vertical for vertical resize, and diagonal for either locked or non-locked aspect ratio diagonal resize with addition of ball press or another button press.) People might argue that is too many button pressed, but we are talking about using all fingers of the hands in addition to non-linear graphic interface. Even so, you are limited to 256 combination of touches with 10 fingers (2^10). While the combination of mouse (5 buttons) and keyboard (104 buttons) give you way much more than that. Since touches are touches you get either a 0 or 1. While using a keyboard, ctrl+A is not the same as ctrl+B. Sure currently OSes don't give much options the fact remains that they are capable of doing so.

Ofcourse adding the touch-pad to the keyboard doesn't even give you any more combinations, since utilizing the keyboard and the touch-pad ends up with keyboard + 5 fingers.

You are right in saying a 3D approach would be the wrong direction, simply because the monitor/OS desktop are two dimensional. And adding a third dimension is simply useless because you can simply send windows to the back behind other windows.

As with some of the comments about this gui being too restrictive for power users, I agree. When I'm coding, I have music/movie/tv-shows playing in a window in the lower right corner, nicely sized, then maybe three of four other windows tiled because I NEED to see them all at once. Having to change windows or move them around so I get some information and then head back is very tedious.

In addition, I don't see you mentioning anything regarding controlling multiple applications at once using multi-touch, or controlling multiple controllers within the same or even different applications at the same time. Which is what you hinted at by showing the multiple sliders.

The idea that multiple touches for the desktop ends up being used for zoom, move, application switch is in its essence faulty.

In real life, multiple touches, are used to manipulate multiple objects at once. For example, trying to find the right color by moving the three sliders of RGB, or changing volume control while scrolling through a webpage, comparing two documents by skimming through pages at the same time, or (which is my favorite) trying to move a file or folder to different folders [you hold onto a file/folder with one hand, and use the other hand to get to the folder you want, currently Mac OSX has the solution where you hold the dragged file/folder over a folder for a couple of seconds then it'll open the folder for you to navigate deeper into the folder, or if your organizing your files impulsively]

However, after all what I said, this is a great solution to Tablet PCs. The reason tablet PCs did not get user base it needed to be popular, because the OSes were the same as the desktop, people needed mice and are extremely limited with just a stylus, even the Wacom digitized stylus with buttons and an eraser. Thats why mostly artists and/or users of specialized software made for companies use them. The current desktop GUI is not suitable or fit for tablets. But con10uum gives a great starting point for a solution to tablet PCs. Its no big deal to have one hand covering part of the screen, because in reality, our palms will be curved when using it, and our fingers are going to be apart. Our brain makes a great job in filling in the spaces from memory and from interpolation. You see two edges of a window, you know its a square/rectangle, I'll bet you $50 you'll ball park that edge and be off by a few pixels. Cause all you need to see is just two edges to resize, move or rotate, let alone seeing three corners out of the four.

Plus with the tablet PC, you are 99% of the time either using a single application only, or an application in the foreground with an application on the side or some corner away running a video. And mostly that application you are using is full screen due to the sheer size of the tablets monitor.

So bottom line, definitely a no go for desktops, although interesting, maybe a modified version of our current GUI, but not the extent of con10uum. Cause I really see the linear 1D windows very restrictive, and the uses of the touch interface is merely a migration from current interface devices to something that is just different rather than an improvement.

And a big whopping YES solution to Tablet PCs with some tweaks that definitely will be brought up with user tests.

If this ever gets to the market, believe me I'll be one of the first people to try it out, it would love to even be part of testing it. So all in all, great concept and idea, the question is what/who/where is its market?

Good luck.

Great concept! Now let's get to some widespread beta testing! ;)

Awesome stuff--love the forward thinking.

I'd like to see a five-finger touch screen scenario because in day to day use cases there is a lot of keyboard-mouse combinations happening and it easier to switch back and forth between mouse pad and keyboard when only one hand has had to leave the keyboard to get to the mouse pad.

Think of all the times you've shift-clicked, ctrl-clicked, etc. Also I'm thinking back on the concepts in the video and realizing you only need five fingers to do most of what you want here.

The five-finger touchcount!

Very interesting ideas. I am not sure about the big touchpad below the keyboard, it could make typing uncomfortable?
maybe 2 touchpads at the sides, one for each hand?
or the touchpad can double as keyboard, it can have the letters painted in it, maybe with haptic feedback.
I guess these options and others are all possible and compatible, as we have different mice today, it depends on what you like.
I think the limiting factor now is software, all the UI now is built with a mouse in mind: only one cursor, only one click. It will take time to have an OS and software really designed to take advantage of this new kind of UI. It's interesting that mobile devices are showing us the future of desktop UI.

Interesting video, AWFUL music. Nuke the music, or get someone to do some that wasn't done by a living-room musician.

Second, this whole scrolling-window paradigm is really clunky; I would grow to hate that in about 5 minutes.

I'd much rather do as I've done, which is to use multiple monitors, and spread my work out so that I can see it at once.

Why would I want to go from app 1 to app 10 by having to either invoke that rather weak global view, where names were sideways, or scroll all the way from one end to the other, forcing me to remember where in the continuum my desired app currently resides?

While your 10-finger ideas are interesting, I suspect better concepts are already done at Apple.

brilliant

This looks great, but there are three problems that I see with the idea.

First, the 1D layout can restrict how well you can organize windows. This might be fixed by allowing multiple continuums to be open at once, arranged vertically and able to be resized, minimized, or maximised.

Second, the touch pad displayed in the video was too small to be comfortable. It needs to be the size of the keyboard, and could even replace the keyboard.

Third, pressing with fingers already resting on the pad doesn't seem like the most comforable way of clicking. I would prefer to lift my finger off the pad and tap to click.

Other than that, it's an awsome idea and I want one right now!

Also, to add to the first idea of multiple continuums, they can be controlled with 5 fingers exactly the same way as the 4 finger controls on a single continuum. In this way, it can act as a continuum of continuums. You can resize, scroll through, rearange, and see a global view of your continuums the same way as windows.

Damn, beat me to it! Sketched the same interface concept out in a brainstorm last night, glad to see someone working in this direction. :)

I personally have no issues with desktop computing and like it the way it is, but as an avid PC gamer I'd be very interested to see how this new kind of technology is incorporated into games in the future.

I would have liked it more if you split the touchpad into 2, one of each hand, on either side of the keyboard. Having it below the keyboard is just awkward, and, as many laptop users would tell you, is more prone to accidentally doing things by brushing against it.

Think about how this would be integrated into any keyboard: just slide the touchpads out from either side; Works for laptops or desktops.

I enjoyed it, and I believe it will be great in practice, but I was wondering if it will have the ability to have an application fill the screen; If I was viewing a website I would want the web browser to fill the screen.

You also need to show how typing would be done, as I've got many theories on how that would be accomplished, and would like to see where your head is at.

Great idea. Though swiping to switch applications seems a natural way, I want to handle multiple windows to operate complex stuff among applications opened. For example, drag-and-drop from one app to another.

Anyway, I enjoyed the movie. keep up your great work.

I think this is a great idea and have some suggestions that came to mind when watching the video. I find that even with the small touch pad on my laptop I can inadvertently move the mouse with my palm and I'm concerned that the same would occur with the design of the touch pad in front of the keyboard.

Perhaps you would consider these different options.
1. Separate the touch pad and keyboard and allow for a multilevel touch pad keyboard setup so the touch pad pulls out from under the desk like current keyboard trays.
2. Design a single unit so the keyboard is elevated above the touch pad and has some sort of resting edge for your hands and perhaps include smaller touch pads on the keyboard that allow for quick actions like scrolling through the windows.
3. Make a single unit with the touch pad separated on either side of the keyboard allowing the user to simply slide their hands out instead of pulling their arms back to access the touch pad.

Other than the design of the keyboard touch pad system I can't wait to see this in action!

Great concept. I think this has real potential. Nice work on the video as well.

Now just figure out how to tie this to a video panel touch screen that can bring up a keyboard, and you're golden.

I was thinking of using the presence of the meaty parts of your hands on the bottom edged of the panel or on the screen itself. Either the keyboard layout would come up when you had your hands rested on the surface, or when they were raised off of it depending on the user. Something along those lines would allow you to swap keyboard hands just by raising or perhaps shifting position of the meaty part of your hands in a specific way or applying more or less pressure on the surface. I think it could be the type of thing where adaptive software could learn a user's style of hand positions and tailor the appearance of the keyboard layout when it was apparent to the system that you wanted to type with one hand or the other.

I think a large portion of the audience will say, 'we don't like to type on non-feedback surfaces!'. To which I reply, 'I don't see any dinosaurs walking around these days.' ;)

this... is... THE FUTURE!!! This solution, once you hear it, it just makes so much sense, its one of those ideas that seems to have existed before anyone thought of it because its just so logical and so simple, natural and intuitive.

I will invest, sign me up, I'm sold!