|
|
REP: 151
|
|
|
Title: Conventions, topics, interfaces for Human-Robot Interaction
|
|
|
Author: Séverin Lemaignan, <add your name!>
|
|
|
Status: Draft
|
|
|
Type: Informational
|
|
|
Content-Type: text/x-rst
|
|
|
Created: 04-Jul-2018
|
|
|
|
|
|
|
|
|
Abstract
|
|
|
========
|
|
|
|
|
|
This REP is written to provide a set of conventions and common interfaces
|
|
|
for Human-Robot Interaction (HRI) scenarios.
|
|
|
This interface is designed to promote interoperability and reusability
|
|
|
of core functionality between the many HRI-related software tools, from
|
|
|
skeleton tracking, to face recognition, to natural language processing.
|
|
|
|
|
|
By following the naming conventions and leveraging the interfaces defined
|
|
|
in this REP, both tools and libraries can be designed to be reusable between
|
|
|
different frameworks and experiments.
|
|
|
|
|
|
These interfaces are designed to be relevant for a broad range of HRI
|
|
|
situations.
|
|
|
|
|
|
Rationale
|
|
|
=========
|
|
|
|
|
|
There are a lot of developers around the world using ROS in contexts involving interactions with human partners.
|
|
|
As such there are also many different implementations which
|
|
|
are similiar but not exactly compatible. A short list of existing known
|
|
|
bridges and implementations follows:
|
|
|
|
|
|
- ...
|
|
|
- ...
|
|
|
|
|
|
Due to a lack of common interfaces the majority of the above packages
|
|
|
cannot be used together. By implementing common interfaces elements of
|
|
|
each package can be used in conjunction with elements of a different
|
|
|
package. This will allow much better collaboration between projects and
|
|
|
allow a reduction in duplicate efforts to implement the same
|
|
|
functionality.
|
|
|
|
|
|
Sub-domains covered by this REP
|
|
|
===============================
|
|
|
|
|
|
1. [Physical HRI](phri) (including haptics, fatigue)
|
|
|
1. [Teleoperation](teleoperation)
|
|
|
1. [Human postures, gestures](gestures), [action recognition](action-recognition)
|
|
|
1. [Speech, Dialogue, Non-verbal communication](communication)
|
|
|
1. [Emotions](emotions), [facial expressions](facial-expressions)
|
|
|
1. [Gaze, Attention](gaze)
|
|
|
|
|
|
Other interface definitions
|
|
|
===========================
|
|
|
|
|
|
There are already existing messages that relate to HRI. These messages definitions
|
|
|
were used to help define this set.
|
|
|
|
|
|
- ...
|
|
|
- ...
|
|
|
|
|
|
Reference Interactions
|
|
|
======================
|
|
|
|
|
|
```mermaid
|
|
|
graph LR
|
|
|
|
|
|
skel(skeleton viewer)
|
|
|
|
|
|
|
|
|
A[face reco]
|
|
|
B[face detection<br/>+ pose estimation]
|
|
|
C[facial landmarks]
|
|
|
D[emotions reco]
|
|
|
E[skeleton tracker]
|
|
|
URDFGEN[human URDF generator]
|
|
|
F[age/gender detector]
|
|
|
H[human database]
|
|
|
S[speech reco]
|
|
|
M[skel/face matcher]
|
|
|
|
|
|
B -- /tf --> client1[ ]
|
|
|
B -- /faces_roi + face frame id --> C
|
|
|
B -- /faces_roi + face frame id --> F
|
|
|
C -- /facial_landmarks + face frame id --> A
|
|
|
C -- /facial_landmarks + face frame id --> D
|
|
|
|
|
|
E --> M
|
|
|
B --> M
|
|
|
M -. " (update)" .-> H
|
|
|
F -- /age --> client3[ ]
|
|
|
F -- /gender --> client4[ ]
|
|
|
|
|
|
A -. "(update)" .-> H
|
|
|
F -. "(update)" .-> H
|
|
|
|
|
|
E -- /tf --> URDFGEN
|
|
|
URDFGEN -- " (urdf)" --> rviz
|
|
|
|
|
|
H -. "(retrieve)".-> client5[ ]
|
|
|
|
|
|
classDef green fill:#9f6,stroke:#333,stroke-width:2px;
|
|
|
classDef orange fill:#f96,stroke:#333,stroke-width:4px;
|
|
|
class client1,client2,client3,client4,client5 green
|
|
|
class skel orange
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
Coordinate Frames
|
|
|
=================
|
|
|
|
|
|
Cf [human posture](gestures) page.
|
|
|
|
|
|
Where meaningful, the coordinate frames used for humans follow the conventions set out in [REP-120](http://www.ros.org/reps/rep-0120.html)
|
|
|
|
|
|
|
|
|
Proposed Interfaces
|
|
|
===================
|
|
|
...
|
|
|
|
|
|
Summary
|
|
|
=======
|
|
|
|
|
|
Existing interfaces
|
|
|
-------------------
|
|
|
|
|
|
|
|
|
References
|
|
|
==========
|
|
|
|
|
|
Copyright
|
|
|
=========
|
|
|
|
|
|
This document has been placed in the public domain. |