A Social Robot and Data Sonification in the Caring Apartment

Estimated reading time: 5 minutes

“Technical systems are often operated using computer screens and keyboards,” says Thomas Hermann. “In our smart apartment, we also work to bring information out of the display screens and into the room.” Illuminated cabinet handles show where the dishes can be found in the kitchen. With small symbol-objects, users can control the light and ambient sound in the bathroom. If the user, for instance, places the shell on the shelf, sea sounds are played and the light changes to blue.

Dr. Thomas Hermann (left) und PD Dr. Sven Wachsmuth (right) are heading the project together with Prof. Dr. Britta Wrede. The avatar Flobi (center) provides support in the kitchen to the apartment’s guests. Photo: CITEC/Bielefeld UniversityDr. Thomas Hermann (left) und PD Dr. Sven Wachsmuth (right) are heading the project together with Prof. Dr. Britta Wrede. The avatar Flobi (center) provides support in the kitchen to the apartment’s guests. Photo: CITEC/Bielefeld University

In order to represent energy consumption in the apartment, the project uses sonification – the conversion of data into sound. With the light sound of a babbling brook, the apartment can make (if the user wants) baseline energy consumption audible. As soon as a device is turned on that uses a lot of energy, such as an electric tea kettle, the sound “swells” into a rain shower. With high energy consumption, apartment guests hear thunder rumbling. Increasing energy consumption is also visually represented: the “InfoPlant,” which is a green household plant, lights up yellow and lets one of its leaves “wilt” by sinking down.

A Lab for Researching the Future of Living

“That the apartment is smart and can provide assistance – that is one thing. But what also makes the apartment unique is that we can make it available to other research groups to use as an experimental environment,” says Dr. Sven Wachsmuth. In order to record human behavior in studies, the apartment is equipped with numerous sensors, including microphones, cameras, sensors to record movement, and tactile flooring. “By analyzing this data, the apartment learns how it can interact appropriately with its guests,” says Dr. Thomas Hermann. “It recognizes where people are moving, where they are looking, and it can listen to them and speak to them via Floka and the virtual robot Flobi.” At the “Human Agent Interaction” (HAI) conference in the CITEC building, CITEC researchers have discussed how the apartment can provide support to several people at the same time.

From the very beginning, this project has been about protecting user data in order to safeguard their privacy. “Recorded data is not in the Cloud, but is stored locally on our own network. Visitor data is encrypted with codes,” says Sven Wachsmuth.

Use After Project Completion

The “Cognitive Robotics Service Apartment” (CSRA) project began in October 2013 and will run through the end of this month. In the future, this experimental apartment is planned to be used for new projects on Smart Home research, such as with the KogniHome Cluster of Innovation. This CITEC-coordinated project with regional partners has developed an intelligent apartment that provides support without service robots.

12 research groups from the Cluster of Excellence have been working together on CSRA, and CITEC invested approximately 1.8 million Euro in the large-scale project. CSRA is one of four large-scale projects at CITEC. The other large-scale projects include the walking robot Hector, the virtual training environment ICSpace, and self-learning root hands (Famula). As part of the Excellence Initiative of the Deutsche Forschungsgemeinschaft (German Research Foundation, DFG), CITEC is funded by state and federal governments (EXC 277).

• < Previous Page