Output
SENSR Output
To listen to SENSR output, you have to connect to the Master Node. The algo nodes only communicate with the Master Node.
Secure Output
SENSR supports secure output mode. In this mode, all outgoing/incoming packets are secured by SSL. By default, the secure mode is turned off. To enable this, we have to set up 2 things before running SENSR.
- Generate certificate files. You can make it by running
generate_certificate.shwith your ip address of the publishing packets. (E.g.$./generate_certificate.sh 192.168.0.100) The certificate file will be located under$HOME/seoulrobotics/keysChangesecurity.enable_sslto true inbin/data/app_config/app_settings.toml.
SENSR encodes data processing results with Protobuf and sends it through TCP using Websocket. With SENSR SDK, you can easily receive SENSR output in your C++ client. SENSR SDK code is here
We also provide JavaScript and Python interfaces inside the SDK to easily work with SENSR output in those languages. For more information on how to use JavaScript and Python, please refer to the relevant readme in the SDK repository.
Communication Specification
| Communication Protocol | TCP (Websocket) |
|---|---|
| Data Encoding | Protobuf 3 |
| Output Port | 5050 |
| Protobuf Message | OutputMessage |
| Communication Protocol | TCP (Websocket) |
|---|---|
| Data Encoding | Protobuf 3 |
| Output Port | 5051 |
| Protobuf Message | PointResult |
WS and WSS
SENSR can secure the output channel. To enable secure websocket, please follow steps in section SENSR output.
Output Data Specification
The Protobuf data specification for all output messages can be found and downloaded from these links:
Streaming Data (TCP and Websocket)
This is a live data stream that can be parsed and processed by external applications.
Output Message
This is a main message which includes all the output results during runtime of SENSR product. But SENSR does not always fill all the fields because of bandwidth issues. SENSR has update rates internally and each field of this message will be filled according to these rates.
Type: Continuous Stream (every frame)
| Field | Field size | Data type |
|---|---|---|
| timestamp | 1 | TimeStamp |
| stream | 0-1 | Stream message |
| event | 0-1 | Event message |
| custom | 0-1 | Custom message |
Stream Message
This message includes output results of SENSR. Each field will be filled according to the update rates of each field. objects : every frame zones : every 10 sec. system : every 1 sec.
Type: Continuous Stream
| Field | Field size | Data type |
|---|---|---|
| objects | 0-* | Object |
| zones | 0-* | ZoneConfig |
| health | 0-1 | SystemHealth |
| first_cloud_arrival_timestamp | 1 | TimeStamp |
Object
This is the new standard object tracking output. points and histories are optional fields. You can receive these optional fields by setting a specific configuration(“common.output.publish_point_cloud”).
Type: Continuous Stream (every frame)
| Field | Size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | |
| label | 1 | Enumeration | 0 = None 1 = Car 2 = Pedestrian 3 = Cyclist, 4 = Misc |
| confidence | 1 | float | 0.0 - 1.0 |
| bbox | 1 | BoundingBox | |
| velocity | 1 | Vector3 (float) | m/s + direction |
| tracking_status | 1 | Enumeration | 0 = NONE 1 = VALIDATING 2 = INVALIDATING 3 = TRACKING 4 = DRIFTING 5 = EXPIRED (VALIDATING : Checking validity in the early stage of tracking INVALIDATING : Short term prediction when tracking is lost in VALIDATING status TRACKING : Stable tracking (RECOMMENDED VALUE TO USE FOR TRACKING, IGNORE THE REST) DRIFTING: Short term prediction when tracking is lost in TRACKING status EXPIRED: Expired tracking) |
| points | 0-* | byte | Object shape (optional) |
| history | 0-1 | History | Object tracking history (optional) |
| prediction | 0-1 | Prediction | Object position prediction (Not Available) |
| zone_ids | 0-* | int32 | List of Zone ids this object currently occupies. |
Zone Config
This is a low frequency message to communicate the zone shapes and locations for you who do not want to use the REST API.
Type: Continuous Stream (low frequency, 1 per 10 seconds or slower)
| Field | Size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | Zone id |
| name | 1 | string | Zone name |
| pbox | 1 | PolygonBox | Array of X,Y,Z |
| type | 1 | Enumeration | 0 = None 1 = Event 2 = Exclusion |
Event Message
This message will be filled whenever SENSR detects events. There are 3 types of events. zone, losing, system. Before parsing each field, you need to check the size of each field. The size 0 means no event related to that field.
Type: Trigger on event
| Field | Field size | Data type | Note |
|---|---|---|---|
| zone | 0-* | ZoneEvent | Zone id |
| losing | 0-* | LosingEvent | Zone name |
| system | 0-1 | SystemHealth | Array of X,Y,Z |
Zone Event
This is an event triggered message. This message gets sent when an object interacts with a zone. Four interactions are: Enter Zone, Exit Zone, Loitering(more than X seconds in a zone) and Velocity(speed above X m/s in a zone).
Type: Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| timestamp | 1 | TimeStamp | Event trigger time. |
| id | 1 | int32 | Zone id |
| type | 1 | Enumeration | 0 = None, 1 = Entry, 2 = Exit, 3 = Loitering, 4 = Exceed speed |
| object | 1 | ZoneEvent_Object | Object information |
Object in ZoneEvent
ZoneEvent saves an object of this type.
| Field | size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | |
| position | 1 | Vector3 (float) | Object location at time of trigger |
| heading | 1 | Float | Yaw (0.0-2.0Pi) |
| velocity | 1 | Vector3 (float) | m/s + direction (optional) Filled in case of overspeed events. |
Losing Event
SENSR will send this message when an object is no longer detected.
Type: Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| timestamp | 1 | TimeStamp | |
| id | 1 | int32 | Object id |
| position | 1 | Vector3 (float) | Last known location of object |
| heading | 1 | float | Yaw (0.0-2.0Pi) |
Custom Message
This message includes additional information of SENSR.
Type: Continuous Stream
| Field | size | Data type | Note |
|---|---|---|---|
| field_of_regard | 0-* | PolygonBox | |
| bg_learning_progress | 1 | float | Background learning progress (0.0 - 1.0) |
| replay | 1 | ReplayInfo | Information on current running replay data. |
| record | 1 | RecordingInfo | Information of recording state |
| profiling | 1 | ProfilingResultSet | Profiling result of current frame |
Point Result
This is an optional message. If a user wants to get raw point clouds, the user can use this. To get this, the user needs to enable a specific configuration(“common.output.publish_point_cloud” = 2) in advance. This message increases lots of burden in transfer bandwidth.
Type: Continuous Stream (low frequency, 1 per 10 seconds or slower)
| Field | size | Data type | Note |
|---|---|---|---|
| points | 0-* | PointCloud | |
| uid | 1 | string | deprecated |
PointCloud
| Field | size | Data type | Note |
|---|---|---|---|
| type | 1 | Enumeration | 0 = None 1 = Ground 2 = Background 3 = Raw |
| id | 1 | string | Identifier of sensor Format: In Calibration mode(algo uid + “#” + lidar topic) In Runtime mode(meaningless) |
| points | * | byte |
System Health
This is part of the Health Monitoring System. This is both a continuous low frequency stream (acting as a heartbeat) and an event trigger stream.
Type: Continuous Stream + Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| master | 1 | Enumeration | 0 = None 1 = OK 2 = Storage Shortage 3 = SlowDown Error 4 = Internal Error |
| nodes | 1-* | map<string, NodeHealth> | Each algo node health |
| points | * | byte |
Node Health
This is part of the System Health message.
| Field | size | Data type | Note |
|---|---|---|---|
| status | 1 | Enumeration | 0 = None 1 = OK 2 = ROS Error 3 = Lost Connection 4 = Invalid GPU Configuration |
| sensors | 1-* | map<string, Enumeration> | 0 = sensor dead 1 = sensor alive 2 = sensor erroneous(Sensor is obstructed.) 3 = sensor tilted 4 = sensor suspended(Sensor points are not transmitted temporarily.) |
Shared Data Types
Bounding Box
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| position | 1 | Vector3 (float) | In meters ( 1.0f = 1m ) |
| size | 1 | Vector3 (float) | X: Relative to Yaw (longitudinal) Y: Relative to Yaw (lateral) Z: Height |
| yaw | 1 | Float | Heading (0.0-2.0Pi) |
Polygon box
This is part of the Zone Config message.
| Field | size | Data type | Note |
|---|---|---|---|
| points | 3-* | Vector2 (float) | Shape of zone in 2D |
| min_z | 1 | float | Base point in z axis |
| max_z | 1 | float | Height of zone |
History
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| states | 0-* | History.State | List of history state |
History.State
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| positions | 1 | Vector3 (float) | object’s tracked XYZ position. |
| timestamp | 1 | Timestamp | timestamp of the tracked XYZ position. |
Prediction
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| positions | 0-* | Vector3 (float) | List of predicted positions of an object |
ReplayInfo
Replay detail information.
| Field | size | Data type | Note |
|---|---|---|---|
| current_index | 1 | int32 | Current playing frame index in replay |
RecordingInfo
Recording detail information.
| Field | size | Data type | Note |
|---|---|---|---|
| saving_progress | 1 | float | Progress of saving file(0.0 - 1.0) |
ProfilingResult
This indicates the profiling status of the latest frame.
| Field | size | Data type | Note |
|---|---|---|---|
| processing_times | 1 | map<string, uint64> | Map of processing time(Unit: ns) Valid key is below. FrameOverall UIRuntimeState-OnUpdate |
| mem_usage | 1 | map<string, uint64> | Map of memory usage(Unit: bytes) Valid key is below. Current-RSS Peak-RSS VM-Usage |
| characteristics | 1 | map<string, Characteristics> | List of special characteristics data Valid key is below. Num-Points Network-packet-size Packet-roundtrip-time |