Intelligent Environments have the vision of enhancing our everyday environment and interaction with its objects by sensing, computing, and communication capabilities. Major characteristics of such environments are the increasing number of intelligent devices (ubiquity), their complexity, and their integration into the background (transparency). Devices will disappear or blend into the background and will be invisible to the user.
However, because of this transparency, users fail to develop an adequate mental model for interaction with such environments. To overcome these challenges, new types of user interfaces are required that will represent intelligent environments. Such representative user interfaces create an overall system image for intelligent environments in order to help users to better understand the intelligent environment. In this sense, representative user interfaces are Ubiquitous Meta User Interfaces (Ubi-MUI) that could increase the transparency and predictability of the whole system by visualizing the environments’ internal states, perception and decision making processes, available services and devices, as well as ongoing and adoption plans. Using Ubi-MUI users could observe, analyze, understand, control, and customize the adaptive behavior and context-dependent interactions of their surrounding.
Our research on Ubi-MUI includes:
Representing Ambient Intelligence and Smart Environments
- Visualization and animation of sensing activities, decisions, and implicit interactions of intelligent systems, which allows users to understand and predict the behavior of the system.
- Creating awareness for implicit interaction concepts using multimedia artifacts
- Using avatars, storytelling, or gaming to introduce the functional capabilities and adaptive behavior of smart environments to novice users
- Concepts for Meta User Interfaces
- Multimodal, multimedia interfaces to control and modify implicit interactions and environments’ responsive activities
- User interfaces to shut down the perception features or responsive behaviors of intelligent environments
- Game-based interfaces to observe, control and modify system behavior
- Haptic, multi touch, or tangible ubiquitous meta user interfaces
- Supportive user interfaces that create a system face for smart environments
- Natural meta interaction concepts allowing users an easy access to multimedia environments (search, exploration, manipulation and control of media and devices), touch and gesture based interfaces, 3D displays and audio immersive systems for augmented-reality.
- Metaphors and coordination algorithms for distributed conflict management,
- mechanisms that allow users an explicit interaction with smart environments.
Implementation, and Evaluation Aspects
Methods to evaluate the added-value of multimedia assisted meta user interfaces
User studies related to mental models for human-environment-interaction and meta user interfaces
Novel approaches to effectively manage the complexity of development,
real-time, de-centralized media-processing architectures,
middleware architectures for sensor and multimedia integration.
Activity analysis and domain observations related to phenomena such as loss of control and over-automation
Smart multimedia sensors with the ability to capture, store, and process audio and video signals for situation recognition and implicit interaction purposes.