RAINBOW Human-Robot Collaboration in Industrial Ecosystems Use Case

Reliable indoor positioning enables several innovative location-based services, because such accuracy levels essentially allow for real-time interaction between humans and cyber-physical systems. For safety-critical industrial IoT, real-time indoor localization services that monitor the movement of objects and detect human worker’s position with respect to the machinery (e.g., heavy-payloads robots), to prevent collisions and accidents. Specifically, the production process demands the involvement of humans and robots to assemble heavy and complex entities like car engines or power supply units, with robots assisting on carrying these heavy products for assembly.

The demonstrator (a typical human robot collaborative setup in a shop floor) is represented in Figure 1 with the following installation and entities:

Figure 1: Demonstrator overview before RAINBOW

• Industrial Robotic arm: Used for lifting heavy parts for assembly (e.g., transformer).
• Industrial PC (IPC) and Programmable Logic Controller (PLC): The control unit and communicates control signals to the robot arm via PLC.
• IoTGateway: Acts as an adapter to communicate information between IPC (using Profinet) and CAP services (using MQTT, OPCUA).
• Node: Each Personnel in the shop floor carries a Node device unit all the time. The node device essentially provides instantaneous 3D position coordinate and motion dynamics of a personnel. The node’s microcontroller performs digital signal processing (filtering) on extracted data from motion sensors and tag/s before sending the processed data to Data Aggregator as shown in the Figure 2.

Figure 2: Block representation of Node device

• Data Aggregator: Primary purpose of Data aggregator is to receive telemetry data from multiple Node devices and send these received data to subscribed devices using MQTT (Message Queue Telemetry Transport) with TLS enabled. The Data aggregator consists of UWB Anchors (distributed over work-place), MQTT Broker as shown in Figure 3. UWB Anchor serves two-fold purpose. First, it acts a reference beacon in distance ranging for Node devices in a work-place area.

Figure 3: Block representation of Data Aggregator

• Collision Avoidance Processing unit (CAP): Primary purpose of CAP unit is to avoid fatal collision between robot arm and personnel. This is done by estimating probability of collision between robot arm and personnel in proximity, ahead of specified time, by taking account of current location (3D coordinates) and motion dynamics of both robot and personnel. Based on estimated probability CAP takes a decision to slow down or to stop the robot. The CAP consists of three distinct services:

1. Personnel Localization and motion capturing service
2. Robot motion tracking service
3. Collision prediction and avoidance service

The data obtained from localization sensing are often noisy and are affected by environment factors such as interference, multi-path fading etc. To obtain best estimate of measurement probabilistic algorithms are used such as Robust Adaptive Linear Quadratic estimator, Multi-model adaptive Kalman filters etc. Adding to this, using probabilistic algorithm it is also feasible to predict future motion trajectories time ahead with certain confidence in each region. These algorithms are computationally intense and are required to run in hard-real time constraints. Thus, the need to run these algorithms on powerful multi core processors.

Robot Motion Tracking service (RMT)
This service tracks robot arm movement and also provide future motion path at every joints. One instance of this service is assigned to exactly one Robot in the work-place area.
The service provides the following:

• Instantaneous 3D Coordinate of the robot joints
• Future motion path of robot joints ahead of time

As the Industrial PC provides instantaneous joint angles of robot arm. This service performs Forward Kinematics operations to extract 3D coordinate position of end effector. Since the Robot motion planning unit knows ahead of time about future motion attributes (like joint angles) of robot. These attributes are obtained to predict future coordinates/regions of presence.

Collision Prediction and Avoidance service (CPA)
This service combines the information from PLMC services and RMT services. And uses probabilistic algorithm to predict the probability of collision between a given personnel and robot in a work-place area time ahead. If possibility of collision is detected, based on likelihood, safety distance and velocities of approaching Personnel and Robot, CPA service either slows down the robot or stops the Robot by sending appropriate control signal to PLC via IPC. One instance of CPA service is assigned to a group of Robots and personnel(s) in a work-place area.
This service provides the following:

• Probability of collision between given Personnel and Robot time ahead
• Stop or slow Robot based on the likelihood, safety distance and velocities of
  approaching Personnel and Robot
• Calculate safety region/distance required between Robot and Personnel