We chatted about Simon, Newton, Spot, and the origins of touch interfaces. We discussed Kinect and the interactive poster advertisements of the future. Now, in the final part of my discussion with Microsoft principal researcher Bill Buxton, I ask him what the next big thing will be.
And he has an answer.
Are you aware of any relatively unknown technologies that exist now but are still waiting to be exploited in some new and popular way?
Yes. I can give you a really concrete one, and then maybe we can make note of the date and in three years see if I was right or not.
Back in 1992 I started working in Toronto on what was basically a drafting table. A big desk that was rear-projection and interactive with a three-foot diagonal display. It had a camera that recognized touch and a pen and so on and so forth. That work was consolidated and approved and in 2007 you saw it in something called Microsoft Surface.
Surface is out there now. It's in very small, specific, niche segments. The reason for that is the complexity of the technology and the cost of manufacturing. What it needs to work is such that it's simply not suitable, even if was cheap, for broad audiences. However, it's out there now while we work on perfecting it.
Right now it has five cameras in it and a projector and a bunch of other stuff. It's just a lot. What will happen is that Surface will become no thicker than a sheet of glass. That will more or less be true. It's not going to have any cameras or projectors because the cameras will be embedded in the device itself.
The best way to think about it is like a big LCD where there's a fourth pixel in every triad. So there's red, green, and blue pixels giving you light, and a fourth pixel which is a sensor that will capture stuff; go the other direction.
The way I talk about this is in terms of the paper cup walkie-talkies you make when you're a kid, where you've got a piece of string and connect to two cups and you can talk to your sister or brother. The key message here is that you've got one Dixie cup working as both the microphone and the loudspeaker. The lesson is that if every acoustic transducer-which is a fancy word for microphone or loudspeaker-could be bidirectional, then why can't every optical transducer-which is a fancy name for camera and display-be bidirectional? The answer is that they can be if you design them right.
The principle thing that we've been doing with Surface at Microsoft is that we've been making screens so they can not only emit light but also be like flatbed scanners. So if you put something against them they can see it at the pixel level. And what makes Surface different than anything else is that you can take a piece of paper or a magazine and hold it on top of the screen and the screen can read it, capture it like a scanner. Then it's right there and you can manipulate it. It's a really smooth interface between the physical and the virtual.
Right now we do this with cameras and all this Rube Goldberg kind of stuff. We needed to make something work so that we could live the future today. Meanwhile, we're engineering it efficiently and refining it so that when we can make these things thin and cheap and reliable and robust we'll already have had several years of experience with them and the software won't be rushing to catch up to the hardware.
What I predict is that sooner than you'd expect-but longer than I want-these things will come in at really cost effective prices and will start appearing in people's living rooms, dining rooms, game rooms, and so on and so forth.
And because of that bidirectional attribute and the fact that they're horizontal, this technology will augment and enhance in a dramatic way the nature of games. It's a very different thing to play a board game or checkers on a table like this. Even Dungeons and Dragons. You can have the whole table animated because the table can read the dice and recognize the characters and pieces. It will really transform the way we play games.
And I believe you will see that in three years. We can have this conversation then and you'll see if I'm right.
This talk of gaming on tables makes me think of a table-like arcade game [called Time Traveller]I played when I was a kid with a big bowl in the middle filled with holograms…
Absolutely! I know exactly what you're talking about!
So, what happened? Why aren't all of us playing holographic games now?
That was an optical illusion that took advantage of a very subtle thing to do with optics and physics. If you stand in front of a mirror, your image appears to be on the far side of the mirror equidistant from the side you are viewing from. If the mirror is concave, there's a double reflection and your image appears to be on the same side of the mirror as you are. It also appears to be inverted, so you're upside down, and it will be up above you depending on the viewing angle.
What they did with that game is they had a standard computer display shooting down onto this concave mirror, which was a bowl. The reason it created this eerie illusion is because of another attribute. If you have a hemisphere bowl and move your viewpoint, the image tends to follow you. So this game was not a hologram. It gave the impression of being a hologram but it was an optical illusion.
That game was a really nice case. Someone understood the optics, worked it through, made a game that made sense and used it to good effect. The fact that you remember it is really cool.
So, what about holograms? There are already a couple of companies that make holograms; proper 3-D displays. The problem with talking about 3-D is that everybody talks about it and everybody knows what it is, but if you ask each person to define what they mean by 3-D all bets are off. I worked for 10 years in 3-D, eight-and-a-half of them at the top 3-D computer graphics company in the world, and because I know so much about it I don't know what it is. I have no idea what people are talking about when they say 3-D because they could mean so many different things. In some sense the term 3-D means nothing until you get really specific.
For me true 3-D is when I walk to the side or around an image and it doesn't follow me. New parts of it are revealed. So I can walk to the other side and see the back. And if it's true 3-D that will be true for both you and me when we're in the same room and at different positions.
That's not what people are talking about with 3-D displays currently, as in 3-D movies. In 3-D movies what you're getting is a stereoscopic display. I'm happy to call that 3-D as well, but we need an additional word to qualify which type of 3-D we're talking about.
It's a really interesting space. And this is where I think people like you have a really important role to play. You have to help consumers cut through the marketing hype and explain the simple things that distinguish the different types of 3-D. You can map out the space and put each thing in its appropriate category so we don't have an apples-and-bananas conversation.
So, yes, there are holographic displays. But do I think that even within a decade they're going to have a major impact on, let's say, television content? No. But I do think they will have an important niche. Everything is best for something and worst for something else.
Follow me on Twitter: @ chadsapieha