Overview
Integration with SENSR
Integration elements
SENSR is built to be integrated with other software solutions. This section describes how to do just that.
- SENSR's output data described in the Output Specification page,
- Data can be integrated through the SENSR Software Development Kit that supports Python, C++, Javascript,
- SENSR can be controlled through its REST API.
Software releases and backward compatibility
SENSR software benefits from frequent updates, allowing to keep the LiDAR drivers up to date, add more features to the software and continuously improve performance.
For each release, a Release Note can be provided under SENSR > Release Note
in this online documentation.
SENSR is designed as backwards compatible between minor releases, (i.e.: 3.X). This ensures that applications built on top of SENSR keep operating smoothly after updates.
Basic Policy
New SENSR releases must not change in an incompatible fashion between any two consecutive minor versions.
- For example, version 3.2 shall be compatible with versions 3.0 and 3.1. Note: major version compatibility will not be guaranteed.
Official announcement
- The release note will list all the changes in parameters, SDK, and output.
- The user manual will have provide default values of parameters, and if the default parameters are changed, it will also be stated in the user manual.
- To-be-deprecated parameters will be highlighted and kept in 2 consecutive minor versions before being officially removed.
- If there’s an exception, it will be announced six months in advance.
Process SENSR output data
SENSR Output
SENSR outputs data through protobuf-encoded TCP websockets on two ports:
- Output stream: port 5050
- Pointcloud stream (optional): port 5051
The full specification can be found on the Output Specification page. The data is transmitted as 2 different messages: OutputMessage and PointResult (optional)
Output Stream
The Output stream is made of the following elements, each sent with a different frequency
- Object information (every frame, default = 0.1 second): contains all data about objects
- Event zone information (every 10 second): contains information on zones drawn in SENSR
- Health status (every 1 second): contains health reports for each component of the system
- Event information (every frame, default = 0.1 second): contains data on events triggered by the system
The Pointcloud stream deliver the site’s pointcloud every frame. As this can represent a significant amount of data streamed each frame, that output is optional
Object information
The objects have the following attributes:
- ID: Unique for each object detected and reset when SENSR restarts,
- Label: Vehicle, Cycle, Pedestrian,Miscellaneous,
- Confidence: confidence level for classification,
- Bbox: bounding box of the object, yaw, coordinates of the volume’s center are provided,
- Velocity: velocity of the object in the referential system in m/s,
- Yaw_rate: yaw value difference of the object between the previous frame and the current frame is divided by the time delta.
- Tracking_status: Object tracking status,
- Validating: Object detected, checking object’s validity,
- Tracking: Stable tracking of object,
- Drifting: Short term prediction when tracking lost in Tracking mode,
- Invalidating: Short term prediction when tracking is lost in validating mode,
- Last_observed_timestamp: Last time offset relative to first pointcloud arrival timestamp,
- History: (when activated) historical location per frame of the object with timestamp,
- Prediction: (when activated) predicted location of the object per frame,
- Zone_ids: List of zones occupied by the object,
- Retro_reflective: flag showing if a given object is retro-reflective or not,
- Points: sequential list of points defined as part of the object.
- Intensities: sequential list of intensity values for points defined as part of the object.
Health information
Every 1 second, SENSR reports the health status of its components.
Master:
- OK: Fully functional
- Storage Shortage: Computer disk over 80% full
- SlowDown Error: Processing performance longer than 100 ms
- Internal Error: Non functional
Node:
- OK: Fully functional
- ROS Error: Error with ROS layer
- Lost Connection: Disconnected from master node
- Invalid GPU Config: Error with GPU drivers
Sensor:
- Sensor alive: Fully functional
- Sensor erroneous: Sensor obstructed
- Sensor tilted: Sensor tilted
- Sensor Dead: disconnection from algo node
Event information
When objects are in a zone, they can trigger events based on the settings in SENSR. That event is relayed through the event message and contains:
- timestamp: Event trigger time,
- id: Zone id,
- type: Type of event triggered Entry, Exit, Loitering, Exceed speed,
- object: Object information,
- ID: ID of the object,
- Position: Object location at the time of trigger,
- Heading: Yaw of the object,
- Velocity: expressed in m/s.
Software Development Kit
Seoul Robotics provides a basic SDK for SENSR to enable users to readily utilize the output data. SENSR's output data can be received and processed through the SDK by connecting to the Master Node. It is recommended to deploy the SDK on the machine that will be used for integrating with SENSR.
Installation instructions can be found on the SDK's page. The SDK supports Python, C++ and JavaScript for processing the output data.
The SDK is built with the following functions:
- Protobuf decoding
- Websocket connection
- Connection error handling logic
- Basic use cases for using SENSR data
The use cases covered by the SDK are:
- Record output data in a file
- Visualize output data in a terminal
- Record pointcloud data
Integration with production-ready applications
For production-ready applications, it is recommended to use C++, JavaScript, or other languages such as Go-Lang to ensure high performance.
Production-ready applications can be built using the SDK or from the ground up.
- Deploy Protobuf: Ensure Protobuf is deployed in your environment.
- Establish Connection with SENSR’s Output Server: Connect to SENSR’s output server on port 5050.
- Decode Messages Using Protobuf Declaration Files: Utilize Protobuf declaration files to decode messages received from SENSR.
- Build Logic to Consume Data: Develop the necessary logic to consume the data as per your application's requirements.
- Compile Code Including Protobuf: Compile the code, including Protobuf, for easy deployment and execution.
Calibration with other systems
Depending on the type of integration, calibration with other systems can be needed. This section describes how to calibrate SENSR with other systems.
Time synchronization
Some systems may require time synchronization with SENSR. SENSR's data is timestamped using the Master Node host's clock. SENSR can be synchronized with external system by simply synchronizing the Master Node host's clock with the desired system.
This synchronization can be achieved using the PTP, NTP or any other OS-level synchronization protocols.
All Timestamps are defined as the arrival date of the first pointcloud processed for a given frame. This way, time-critical applications can directly use the timestamp provided by SENSR for object detection.
Spatial calibration
All data is expressed in the Master node’s cartesian referential. See here for more details.
To calibrate SENSR with another system, a reference map can be used.
The reference map should define the referential,s origin and axes. The map can be as simple as a PNG file See here for importing a PNG map into SENSR as a background map..
LiDARs’ location in the Master Node referential can be adjusted to set the coordinates origin according to a landmark for calibration against other systems.
GPS coordinates
For a certain number of applications, the usage of GPS coordinates is needed. SENSR does not provide conversion between cartesian and GPS coordinate systems at the moment. That conversion should be performed at integration level. As an approximation, the conversion can be done using 3 points with known coordinates in both SENSR's referential and the GPS coordinates system.
Control SENSR through REST API
SENSR can be remote-controlled through its REST API. It allows to:
- Control Algo Nodes,
- Change parameters,
- Add and remove zones,
- Restart the pipeline,
- Record data...
The REST API is fully documented with usage examples on REST API specification page.