�þ���ҹ����ӰԺ��ѹۿ�

DI Fellow Profile – Hadi Rahmati

When School of Visual Communication Design professor Hadi Rahmati met Computer Science assistant professor Raiful Hasan at �þ���ҹ����ӰԺ��ѹۿ�'s new faculty orientation in 2023, their conversation about their respective research interests led to an important project. They're working together to solve a big problem: how can self-driving cars communicate better with pedestrians?

Based on suggestions from Hasan, Rahmati read about 20 research articles on human-machine interaction and noticed something important was missing. None of the articles applied communication theory – the science of how people share information and build understanding.

This discovery became the starting point for their work.

After analyzing 60 existing external Human Machine Interface (eHMI) designs for Autonomous Vehicles (AVs), they realized that self-driving cars need to adapt their communication based on who they're talking to. Current systems treat all pedestrians the same, but people have different needs and situations.

When you’re walking and encounter a regular car, you can usually tell what the driver plans to do. Maybe they wave to let you know it’s safe to cross, or you make eye contact to confirm they see you. These small interactions help you trust that it’s safe.

But self-driving cars don’t have drivers. This creates a trust problem. “The absence of a driver can diminish the trust between pedestrians and AVs,” Rahmati explains. 

Car companies have tried to solve this by adding screens, lights and sounds to their vehicles. But these systems have a major flaw – they’re not smart about when and how they communicate.

Think about how human drivers behave. They don’t honk at every single pedestrian – only when someone seems distracted or there’s danger. But current self-driving cars frequently use the same signals all the time, regardless of the situation, leading to sensory overload in the pedestrian side of the communication.

Rahmati and Hasan are interested in designing a smarter system. They’ve created different communication approaches for multiple scenarios, depending on what's happening with the pedestrian. During their John and Fonda Elliot Design Innovation Faculty Fellowship, using Unity software, they developed a functioning virtual prototype to be used in their experiment with real participants to test the effectiveness of their design.

Originally, they started building their system to use LIDAR – a technology that uses reflected lasers to “see” the world around the car.  However, LIDAR only provides shape and distance data, so Rahmati and Hasan added volumetric cameras to improve depth calculation and provide image processing. This helps the car understand what pedestrians are doing and respond appropriately.

The two also want to make interactions feel more natural. “Imagine when you say, ‘thank you’ to a car for yielding, you would normally expect a response from a human driver,” Rahmati says. Their system could signal back “you're welcome,” making the interaction feel more human, providing what is called a bidirectional feedback mechanism (or feedback loop).

One of the most important parts of their research focuses on helping people with disabilities. Instead of designing for an average pedestrian, they want their system to prioritize whoever needs the most help in any situation – whether that's a child, someone in a wheelchair or a person with vision problems.

They're using emojis as part of their communication system because research shows they work well across different cultures and languages. The system combines visual signals (like lights and screens), sounds and vibrations that people can feel. “Just imagine if they're wearing smartwatches,” Rahmati says, “These are the needs of the communication in the future.”

The team plans to use virtual reality headsets to test how well their system works compared to current designs. They need this data to approach AVs companies to help prove their approach is better than what the companies are using now as well as to secure grant funding to refine and improve their system.

This research is about more than just technology though. It's about making sure self-driving cars work safely for everyone. By applying what we know about human communication to AVs' human-machine interface designs, Rahmati and Hasan are helping solve one of the biggest challenges facing self-driving cars.

Their work shows that successful self-driving cars need more than just good technology. They need to understand and copy the way humans naturally communicate with each other. This could be key to helping people feel comfortable sharing the roads with AVs in the future.

The John and Fonda Elliot Design Innovation Faculty Fellows Program is a six-week fellowship program, based in the Design Innovation Hub and generously supported through an endowed gift from John and Fonda Elliot, that creates space and time for faculty members from ANY academic unit to engage in collaborative, cross or trans-disciplinary projects utilizing human-centered design and innovation strategies. While advancing their project collaborations, Elliot DI Faculty Fellows cohorts also engage in activities grounded in our “Design Innovation Toolkit” resulting in certification as a “Design Innovation Change Agent” upon completion of the program.