Output Specification
Output Data Specification
The Protobuf data specification for all output messages can be found and downloaded from these links:
Streaming Data (TCP and Websocket)
This is a live data stream that can be parsed and processed by external applications. Here you can see websocket endpoints.
Output Message
This is a main message which includes all the output results during runtime of SENSR product. But SENSR does not always fill all the fields because of bandwidth issues. SENSR has update rates internally and each field of this message will be filled according to these rates.
Type: Continuous Stream (every frame)
| Field | Field size | Data type |
|---|---|---|
| timestamp | 1 | TimeStamp |
| stream | 0-1 | Stream message |
| event | 0-1 | Event message |
| custom | 0-1 | Custom message |
Stream Message
This message includes output results of SENSR. Each field will be filled according to the update rates of each field. objects : every frame zones : every 10 sec. system : every 1 sec.
Type: Continuous Stream
| Field | Field size | Data type |
|---|---|---|
| objects | 0-* | Object |
| zones | 0-* | ZoneConfig |
| health | 0-1 | SystemHealth |
| first_cloud_arrival_timestamp | 1 | TimeStamp |
Object
This is the new standard object tracking output. points and histories are optional fields. You can receive these optional fields by setting a specific configuration(“common.output.publish_point_cloud”).
Type: Continuous Stream (every frame)
| Field | Size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | |
| label | 1 | Enumeration | 0 = None 1 = Car 2 = Pedestrian 3 = Cyclist, 4 = Misc |
| confidence | 1 | float | 0.0 - 1.0 |
| bbox | 1 | BoundingBox | |
| velocity | 1 | Vector3 (float) | m/s + direction |
| tracking_status | 1 | Enumeration | 0 = NONE 1 = VALIDATING 2 = INVALIDATING 3 = TRACKING 4 = DRIFTING 5 = EXPIRED (VALIDATING : Checking validity in the early stage of tracking INVALIDATING : Short term prediction when tracking is lost in VALIDATING status TRACKING : Stable tracking (RECOMMENDED VALUE TO USE FOR TRACKING, IGNORE THE REST) DRIFTING: Short term prediction when tracking is lost in TRACKING status EXPIRED: Expired tracking) |
| points | 0-* | byte | Object shape (optional) |
| history | 0-1 | History | Object tracking history (optional) |
| prediction | 0-1 | Prediction | Object position prediction (Not Available) |
| zone_ids | 0-* | int32 | List of Zone ids this object currently occupies. |
Python Sample Code using SENSR-I SDK
Get point, bbox, prediction
for obj in message.stream.objects:
float_size = ctypes.sizeof(ctypes.c_float)
object_point_num = len(obj.points) // (float_size * 3)
print('Obj ({0}): point no. {1}'.format(obj.id, object_point_num))
print('Obj ({0}): bbox {1}'.format(obj.id, obj.bbox))
print('Obj ({0}): prediction {1}'.format(obj.id, obj.prediction))
Get the velocity value and position angle of the objects
import math
for obj in message.stream.objects:
Velocity = math.sqrt(math.pow(obj.velocity.x,2)+math.pow(obj.velocity.y,2))
Angle = math.atan2(obj.velocity.y,obj.velocity.x)
Zone Config
This is a low frequency message to communicate the zone shapes and locations for you who do not want to use the REST API.
Type: Continuous Stream (low frequency, 1 per 10 seconds or slower)
| Field | Size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | Zone id |
| name | 1 | string | Zone name |
| pbox | 1 | PolygonBox | Array of X,Y,Z |
| type | 1 | Enumeration | 0 = None 1 = Event 2 = Exclusion |
Python Sample Code using SENSR-I SDK
Get trigger message of the zone
for zone in message.event.zone:
if zone.type == sensr_output.ZoneEvent.Type.ENTRY:
print('Entering zone ({0}): obj {1}'.format(zone.id, zone.object.id))
elif zone.type == sensr_output.ZoneEvent.Type.EXIT:
print('Exiting zone ({0}): obj {1}'.format(zone.id, zone.object.id))
Event Message
This message will be filled whenever SENSR detects events. There are 3 types of events. zone, losing, system. Before parsing each field, you need to check the size of each field. The size 0 means no event related to that field.
Type: Trigger on event
| Field | Field size | Data type | Note |
|---|---|---|---|
| zone | 0-* | ZoneEvent | Zone id |
| losing | 0-* | LosingEvent | Zone name |
| system | 0-1 | SystemHealth | Array of X,Y,Z |
Zone Event
This is an event triggered message. This message gets sent when an object interacts with a zone. Four interactions are: Enter Zone, Exit Zone, Loitering(more than X seconds in a zone) and Velocity(speed above X m/s in a zone).
Type: Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| timestamp | 1 | TimeStamp | Event trigger time. |
| id | 1 | int32 | Zone id |
| type | 1 | Enumeration | 0 = None, 1 = Entry, 2 = Exit, 3 = Loitering, 4 = Exceed speed |
| object | 1 | ZoneEvent_Object | Object information |
Python Sample Code using SENSR-I SDK
Get trigger message of the zone
for zone in message.event.zone:
if zone.type == sensr_output.ZoneEvent.Type.ENTRY:
print('Entering zone ({0}): obj {1}'.format(zone.id, zone.object.id))
elif zone.type == sensr_output.ZoneEvent.Type.EXIT:
print('Exiting zone ({0}): obj {1}'.format(zone.id, zone.object.id))
Object in ZoneEvent
ZoneEvent saves an object of this type.
| Field | size | Data type | Note |
|---|---|---|---|
| id | 1 | int32 | |
| position | 1 | Vector3 (float) | Object location at time of trigger |
| heading | 1 | Float | Yaw (0.0-2.0Pi) |
| velocity | 1 | Vector3 (float) | m/s + direction (optional) Filled in case of overspeed events. |
Python Sample Code using SENSR-I SDK
Get Point Cloud
for point_cloud in message.points:
float_size = ctypes.sizeof(ctypes.c_float)
num_points = len(point_cloud.points) // (float_size * 3)
if point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.RAW:
print('Topic ({0}) no.of points- {1}'.format(point_cloud.id, num_points))
elif point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.GROUND:
print('Ground points no. of points - {0}'.format(num_points))
elif point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.BACKGROUND:
print('Environment points no. of points - {0}'.format(num_points))
Losing Event
SENSR will send this message when an object is no longer detected.
Type: Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| timestamp | 1 | TimeStamp | |
| id | 1 | int32 | Object id |
| position | 1 | Vector3 (float) | Last known location of object |
| heading | 1 | float | Yaw (0.0-2.0Pi) |
Custom Message
This message includes additional information of SENSR.
Type: Continuous Stream
| Field | size | Data type | Note |
|---|---|---|---|
| field_of_regard | 0-* | PolygonBox | |
| bg_learning_progress | 1 | float | Background learning progress (0.0 - 1.0) |
| replay | 1 | ReplayInfo | Information on current running replay data. |
| record | 1 | RecordingInfo | Information of recording state |
| profiling | 1 | ProfilingResultSet | Profiling result of current frame |
Point Result
This is an optional message. If a user wants to get raw point clouds, the user can use this. To get this, the user needs to enable a specific configuration(“common.output.publish_point_cloud” = 2) in advance. This message increases lots of burden in transfer bandwidth.
Type: Continuous Stream (low frequency, 1 per 10 seconds or slower)
| Field | size | Data type | Note |
|---|---|---|---|
| points | 0-* | PointCloud | |
| uid | 1 | string | deprecated |
PointCloud
| Field | size | Data type | Note |
|---|---|---|---|
| type | 1 | Enumeration | 0 = None 1 = Ground 2 = Background 3 = Raw |
| id | 1 | string | Identifier of sensor Format: In Calibration mode(algo uid + “#” + lidar topic) In Runtime mode(meaningless) |
| points | * | byte | Byte order : LittleEndian |
Python Sample Code using SENSR-I SDK
Get Point Cloud
for point_cloud in message.points:
float_size = ctypes.sizeof(ctypes.c_float)
num_points = len(point_cloud.points) // (float_size * 3)
if point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.RAW:
print('Topic ({0}) no.of points- {1}'.format(point_cloud.id, num_points))
elif point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.GROUND:
print('Ground points no. of points - {0}'.format(num_points))
elif point_cloud.type == sensr_pcloud.PointResult.PointCloud.Type.BACKGROUND:
print('Environment points no. of points - {0}'.format(num_points))
System Health
This is part of the Health Monitoring System. This is both a continuous low frequency stream (acting as a heartbeat) and an event trigger stream.
Type: Continuous Stream + Trigger on event
| Field | size | Data type | Note |
|---|---|---|---|
| master | 1 | Enumeration | 0 = None 1 = OK 2 = Storage Shortage 3 = SlowDown Error 4 = Internal Error |
| nodes | 1-* | map<string, NodeHealth> | Each algo node health |
| points | * | byte |
Python Sample Code using SENSR-I SDK
Get system health informantion
system_health = message.stream.health
print(‘System health: {0}’.format(system_health.master))
if len(system_health.nodes) > 0:
for node_key in syste_health.nodes:
node_health = system_health.nodes[node_key]
print('Node ({0}) health: {1}'.format(node_key, health.status))
if len(node_health.sensors) > 0:
for sensor_key in node_health.sensors:
Sensor_health = node_health.sensors[sensor_key]
print('Sensor ({0}) health {1}'.format(sensor_key, sensor_health))
else:
print(‘ No sensors are connected’)
else:
print(‘No nodes are connected’)
Node Health
This is part of the System Health message.
| Field | size | Data type | Note |
|---|---|---|---|
| status | 1 | Enumeration | 0 = None 1 = OK 2 = ROS Error 3 = Lost Connection 4 = Invalid GPU Configuration |
| sensors | 1-* | map<string, Enumeration> | 0 = sensor dead 1 = sensor alive 2 = sensor erroneous(Sensor is obstructed.) 3 = sensor tilted 4 = sensor suspended(Sensor points are not transmitted temporarily.) |
Shared Data Types
Bounding Box
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| position | 1 | Vector3 (float) | In meters ( 1.0f = 1m ) |
| size | 1 | Vector3 (float) | X: Relative to Yaw (longitudinal) Y: Relative to Yaw (lateral) Z: Height |
| yaw | 1 | Float | Heading (0.0-2.0Pi) |
Polygon box
This is part of the Zone Config message.
| Field | size | Data type | Note |
|---|---|---|---|
| points | 3-* | Vector2 (float) | Shape of zone in 2D |
| min_z | 1 | float | Base point in z axis |
| max_z | 1 | float | Height of zone |
History
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| states | 0-* | History.State | List of history state |
History.State
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| positions | 1 | Vector3 (float) | object’s tracked XYZ position. |
| timestamp | 1 | Timestamp | timestamp of the tracked XYZ position. |
Prediction
This is part of the Object message.
| Field | size | Data type | Note |
|---|---|---|---|
| positions | 0-* | Vector3 (float) | List of predicted positions of an object |
ReplayInfo
Replay detail information.
| Field | size | Data type | Note |
|---|---|---|---|
| current_index | 1 | int32 | Current playing frame index in replay |
RecordingInfo
Recording detail information.
| Field | size | Data type | Note |
|---|---|---|---|
| saving_progress | 1 | float | Progress of saving file(0.0 - 1.0) |
ProfilingResult
This indicates the profiling status of the latest frame.
| Field | size | Data type | Note |
|---|---|---|---|
| processing_times | 1 | map<string, uint64> | Map of processing time(Unit: ns) Valid key is below. FrameOverall UIRuntimeState-OnUpdate |
| mem_usage | 1 | map<string, uint64> | Map of memory usage(Unit: bytes) Valid key is below. Current-RSS Peak-RSS VM-Usage |
| characteristics | 1 | map<string, Characteristics> | List of special characteristics data Valid key is below. Num-Points Network-packet-size Packet-roundtrip-time |