Building the Backbone of a Smarter Smart Home

Estimated reading time: 3 minutes

The state of artificial intelligence (AI) in smart homes nowadays might be likened to a smart but moody teenager: It’s starting to hit its stride and discover its talents, but it doesn’t really feel like answering any questions about what it’s up to and would really rather be left alone, OK?

William Yeoh, assistant professor of computer science and engineering in the School of Engineering & Applied Science at Washington University in St. Louis, is working to help smart-home AI to grow up.

The National Science Foundation (NSF) awarded Yeoh a $300,000 grant to assist in developing smart-home AI algorithms that can determine what a user wants by both asking questions and making smart guesses, and then plan and schedule accordingly. Beyond being smart, the system needs to be able to communicate and to explain why it is proposing the schedule it proposed to the user.

These challenges rely heavily on communication. And like that moody teen, communication is not currently one of AI’s strengths.

“So far, a lot of AI is successful in isolation,” Yeoh said. Feats such as winning a game of “Go,” or recognizing faces do not require significant user interaction; a computer can do these tasks mostly on its own. “But if AI is going to help people,” he said, “interaction with people is pretty important.”

The first step in this project is figuring out what the user actually wants: The temperature to be 70 degrees at home upon arrival; the car fully charged by 8 a.m.; and so on. And it must decipher what the user doesn’t want: The air conditioner set at full-blast all day long, or the car to charge during peak hours.

An AI system could meticulously and continuously ask a user for every one of his or her preferences. But because that is not possible (or would be, at the least, extremely annoying), the research sets out to determine how many questions should be asked and what should those questions be, as noted by Yeoh in his grant application. The reason for asking: to make the smartest possible decisions when the user has not supplied specific information.

The algorithm that determines which questions to ask will be restricted by how many questions it can ask, and so will have to decide which questions have answers that will yield the most useful information.

Once the parameters and constraints are set, and the system has all of the available information it’s going to have, it devises a schedule that honors the user’s preferences and maximizes the user’s comfort, while minimizing energy usage.

The system’s job isn’t finished, however, once the parameters are set and the scheduling is done; it still needs to work on its communication skills. Communication is a two-way street. Not only does Yeoh want to develop smart home systems that a user can tell what to do, he also thinks that the system should be able to explain itself to the user.

“The system needs to be able to explain to the user ‘why,’” Yeoh said. “The field has been working on getting good answers from AI. Now it’s time to get good explanations.”

Say a homeowner rarely goes in the basement, but heads down one night to get something out of storage. All of a sudden, the lights go out. Without being able to ask the system why, the user may be wondering, “Is there a power outage? Is someone here? Is my house out to get me?”

If the system could simply be asked what happened however, the homeowner would realize that in an attempt to save energy, the system has been turning off the lights in the basement each day after a certain time, because (via sensors, perhaps) it knows that the user rarely goes downstairs in the evening.

Yeoh’s project is developing the tools to give users the ability to ask those questions, vocally or by turning to a visual interface designed to give users access to a host of information, from details about specific devices to a broader view of energy consumption.

Having access to that information will allow the user to more narrowly tailor settings, working with the system instead of simply having to accept its settings. After all, there are lots of variables for a smart-home AI system to take into account if it’s monitoring the temperature, locking the doors, charging a vehicle, preheating the oven, and monitoring a security system — all while trying to conserve energy usage.

“Energy prices, weather, the status of the devices,” Yeoh said of all the changing conditions that AI will need to monitor. “And humans, the most fickle variable of all.”