Robot SAM

This project focused on the way that people experience and communicate with robots.

This project was performed for a course called Human-Robot-Communication. The project resulted in the creation of SAM, a robot waiter who could deliver a drink to customers in a coffee shop. SAM was created with a focus on making interactions smooth and intuitive. It was designed to resemble a robot but with human-like traits, avoiding the uncanny valley effect, where robots that are too human-like can feel unsettling.

SAM's design included expressive eyes and a mouth to convey emotions, wheels for movement, and the ability to process and respond to customer speech. SAM also had a platter for carrying drinks and could track and gaze at customers, making the interaction feel more natural. The project carefully analyzed and designed various interaction scenarios, ensuring SAM could handle situations like delivering the wrong drink or forgetting an order, ultimately creating a more engaging experience for customers. The whole purpose of the project was to explore how the relationship between humans and robots is constructed and how one can design for effective communication between humans and robots. An important aspect of interaction with waiters was the sense that the waiter would gaze at a customer instead of looking past them, that's why we also enabled SAM with the possibility to track the faces of customers and look in their direction. This can be seen in the video presented on this page.

This project was performed for a course called Human-Robot-Communication. The project resulted in the creation of SAM, a robot waiter who could deliver a drink to customers in a coffee shop. SAM was created with a focus on making interactions smooth and intuitive. It was designed to resemble a robot but with human-like traits, avoiding the uncanny valley effect, where robots that are too human-like can feel unsettling.

SAM's design included expressive eyes and a mouth to convey emotions, wheels for movement, and the ability to process and respond to customer speech. SAM also had a platter for carrying drinks and could track and gaze at customers, making the interaction feel more natural. The project carefully analyzed and designed various interaction scenarios, ensuring SAM could handle situations like delivering the wrong drink or forgetting an order, ultimately creating a more engaging experience for customers. The whole purpose of the project was to explore how the relationship between humans and robots is constructed and how one can design for effective communication between humans and robots. An important aspect of interaction with waiters was the sense that the waiter would gaze at a customer instead of looking past them, that's why we also enabled SAM with the possibility to track the faces of customers and look in their direction. This can be seen in the video presented on this page.

This project was performed for a course called Human-Robot-Communication. The project resulted in the creation of SAM, a robot waiter who could deliver a drink to customers in a coffee shop. SAM was created with a focus on making interactions smooth and intuitive. It was designed to resemble a robot but with human-like traits, avoiding the uncanny valley effect, where robots that are too human-like can feel unsettling.

SAM's design included expressive eyes and a mouth to convey emotions, wheels for movement, and the ability to process and respond to customer speech. SAM also had a platter for carrying drinks and could track and gaze at customers, making the interaction feel more natural. The project carefully analyzed and designed various interaction scenarios, ensuring SAM could handle situations like delivering the wrong drink or forgetting an order, ultimately creating a more engaging experience for customers. The whole purpose of the project was to explore how the relationship between humans and robots is constructed and how one can design for effective communication between humans and robots. An important aspect of interaction with waiters was the sense that the waiter would gaze at a customer instead of looking past them, that's why we also enabled SAM with the possibility to track the faces of customers and look in their direction. This can be seen in the video presented on this page.

The Process

We began by defining the robot's goal. This was limited to delivering a drink to a customer in order to keep the scope clear and achievable.

This task was chosen for its potential to generate interesting interactions. Our design process included research, a co-creation session, and scenario enactment to understand potential interactions. During the co-creation session, participants drew their ideal robot waiter, leading to key insights. They were comfortable taking a drink from a tray and consistently depicted the robot with a separate head, body, and wheels. From these findings, we identified three key requirements: (1) The robot should maintain eye contact while serving, (2) It should be kind, happy, and professional, (3) users expect different interactions from a robot, so it doesn't need to mimic human behavior closely. We then brainstormed interaction flows for SAM, incorporating verbal communication, emotional expression, gaze, and movement into the design, which guided the development of the prototype

My Role

The project was executed together with 3 other students from the University of Twente. My role was very multi-faceted during this project. I was involved in the research phase, the cocreation session, and took part in the scenario enactment. Thereafter I was the main technical power in the team. I wrote the software necessary for speech detection, processing, and response creation. I also took the lead in the creation of the hardware of the robot, combining all modalities into one working whole. Thereafter, I was present during the evaluation of SAM but was not directly involved with designing the evaluation practices that were used during this project.

User Experience
Design

User Experience Design

This project included the design, development, and evaluation of an interaction strategy for social robotics. It helped to shape a workflow for solving these kinds of interaction problems. This included the gathering of user input about appropriate interactions within a certain context, ideation about how we can shape these interactions, and implementing that practically.

Speech Processing

Speech Processing

For this project the speech of participants needed to be captured, processed, and thereafter an appropriate response needed to be sent by the robot. Google Dialog Manager was used to detect intents within the speech of participants and allowed for the creation of appropriate textual responses for the robot. Google's Text-to-Speech API was used to generate an vocal response. This was all implemented in a Python-based dialog manager script.

Hardware Design

In this project, a hardware prototype was developed that included a bunch of different modalities that all needed to be connected to a central microcontroller. These included different LED modules for the eyes and mouth, a sensor for detecting the placement of items on the delivery plate of SAM, and two continuous servo motors that the robot used for its movement. This further improved my skills in Arduino development and hardware design.