Interactive user interfaces, virtual reality, augmented reality.
For many types of information retrieval, social interaction is critical to the experience. For instance, why do people go to live music concerts instead of listening to the CD? Partly because of the different kind of sound, but I believe it is also largely due to the social experience of being in an auditorium with the rest of the audience. This identifies one area where I do not think Virtual Reality will be successful. I believe that one of the biggest challenges for today's technical design- ers is to develop ways that computers can improve communication and interaction among people.
There are also times when it is appropriate to provide information without replacing the world for practical reasons. For instance, when giving traffic directions while driving, a typical approach is to put video on the dashboard with a map application, or to supply a printed piece of paper with directions. Both of these situations are potentially dangerous because they overload the visual channel by replacing it, at least temporarily.
One approach to these challenges is to use "Augmented Reality" - that is, to superimpose computer generated data on top of the real world, as the person moves within it. A number of groups are beginning to experiment with this notion, notably Steve Feiner at Columbia. My work is different in that it replaces the video augmentation that Feiner uses with audio. As you interact with the world, you are provided with relevant audio annotations which give you extra information about the current situation without interrupting what you are doing. This works by using a different and largely unused communication channel.
For a practical application based on this idea, it may be necessary for the entire computer system to be carried around, and for the computer system to know where it is in the environment. To gen- eralize this research question, I have begun to ask "What happens when a computer that you carry knows where it is?"
We have been applying these concepts by creating an automated tour guide for museums. The sys- tem will automatically describe pieces in the museum as they are approached. One advantage of this method over traditional taped tours is that people will have the choice to choose what they want to see, in what order, and for how long. We currently have a prototype demonstrating this technology. This concept could also be applied to the above mentioned traffic direction problem so that the driver automatically gets verbal traffic commands such as "turn right at the light" in situ.
I believe that Virtual Reality may be appropriate for certain kinds of information visualization, such as architectural walkthroughs - which can not be done any other way. However, it may not be able to replace other kinds of information retrieval, such as museum tours. I believe that this is due to the social aspect of the information retrieval process. Virtual Reality takes people away from people. It replaces the natural environment with a virtual one - and the society inherent in the natural one. Social situations in our physical world are important in our educational, entertaining, and creative expe- riences. It is one thing to replace social situations with technology, but the real challenge is to use technology to enhance them.
I'm at UNM most of the time at bederson@cs.unm.edu
Form last update: Dec 20th.