WO2019000859A1

WO2019000859A1 - Safe anti-collision system and method for lifting appliance based on three-dimensional recognition of container area outline

Info

Publication number: WO2019000859A1
Application number: PCT/CN2017/116803
Authority: WO
Inventors: 刘旭; 林凡雨; 王伟; 崔冲肖
Original assignee: 北京国泰星云科技有限公司
Priority date: 2017-06-29
Filing date: 2017-12-18
Publication date: 2019-01-03
Also published as: CN107089599A; CN107089599B

Abstract

A safe anti-collision system and method for a lifting appliance based on three-dimensional recognition of a container area outline. The anti-collision system comprises a 3D laser unit and a 2D laser scanner; the 3D laser unit comprises an embedded controller, and a laser scanner, a motor driver and a dual-DGPS differential positioning system which are connected to the embedded controller, separately; and the embedded controller is in a two-way communication with an RTG/RMG electronic control system. According to the system and method, the container area outline of the stowage location and the spatial position of the lifting appliance are recognized in real time by means of 3D laser point cloud data calculation, three-dimensional recognition of the container area outline, cart positioning and quick feedback of 2D laser plane data by combining the real-time 3D laser unit with positioning and attitude determining functions with the 2D laser scanner, real-time position detection on the lifting appliance and container stacks in the stowage location and adjacent stowage locations in an operating area is effectively achieved when a container wharf RTG/RMG conducts container grasping and unloading operation in a field area, and lifting appliance collision accidents are comprehensively avoided.

Description

Spreader safety anti-collision system and method based on three-dimensional identification of container box area contour

Technical field

The invention relates to a spreader safety collision avoidance system and method based on three-dimensional identification of a container box area, and a tire type container crane (hereinafter referred to as RTG) and a rail type container crane (hereinafter referred to as RMG) applied at a container terminal, through a container The three-dimensional identification of the terminal box area combined with the RTG (or RMG) condition status judgment and the spreader position identification finally realizes the real-time safety collision avoidance function of the spreader in the three-dimensional space with respect to the entire container area.

Background technique

At present, domestic container terminals have encountered the need to accurately identify and locate containers in the yard during the development of automation. The use of a 2D laser scanner or an existing 3D laser device can depict the outline of the box area in the scanning plane to a certain extent, but the position and posture information of the container in the adjacent position cannot be detected.

In the process of target object recognition and position detection using a 2D laser scanner, since the 2D laser can only perform laser scanning in one scanning plane, only one section of the scanning target can be detected. Normally, the surface of the scanned target object is not a regular shape. If only one section is scanned and identified, the complete surface information cannot be obtained. Moreover, the existing 3D laser products are complete sets of products produced by foreign companies using multi-line laser or double prism reflection technology. If these 3D laser products are used to identify and locate containers, the acquired data is sparse, aiming at the target. There is a large error in object scanning and recognition, which cannot meet the needs of the application. Or take long-term static scanning to obtain accurate contour information, but also can not meet the high efficiency of the terminal operation, high real-time Spreader protection needs.

Summary of the invention

In order to overcome the above disadvantages of the prior art, the present invention provides a spreader safety collision avoidance system and method based on three-dimensional identification of a container box area.

The technical solution adopted by the present invention to solve the technical problem thereof is: a spreader safety collision avoidance system based on three-dimensional identification of a container box outline, comprising a 3D laser unit and a 2D laser scanner disposed under the trolley platform, the 3D laser The unit comprises an embedded controller and a laser scanner, a motor driver and a dual DGPS differential positioning system respectively connected to the embedded controller, the motor driver being connected to the servo motor, the laser scanner being arranged on the rotating head, rotating The pan/tilt is connected to the reducer of the servo motor through the coupling; the laser scanner is connected to the embedded controller through the data line, and is used for receiving the instruction issued by the embedded controller for scanning, and transmitting the scanned data to the embedded in real time. a controller; the motor driver is connected to the embedded controller through a signal line, and is configured to receive an instruction from the embedded controller to control the servo motor to drive the pan/tilt rotation; the dual DGPS differential positioning system passes the data line and the embedded controller Connection for transmitting position change information and attitude change information of the detected 3D laser unit as a whole Give to the embedded controller;

The 2D laser scanner is connected to the embedded controller of the 3D laser unit through a data line, is used for contour scanning under the RTG/RMG, and transmits the scanned data to the embedded controller in real time;

The embedded controller of the 3D laser unit performs bidirectional communication with the RTG/RMG electronic control system to obtain the working status information of the RTG/RMG, and sends the protection command of the spreader in the container area to the RTG/RMG electronic control system.

The invention also provides a safety protection anti-collision method for a spreader based on three-dimensional identification of a container box area, comprising the following contents:

1. During the movement of the cart, when the cart moves to a new shell position, or the spreader locks up a container, the laser scanner of the 3D laser unit performs a 3D fast scan on the underside of the cart; In the process, the embedded controller of the 3D laser unit performs real-time scanning position adjustment on the laser scanner according to the height of the spreader, and performs 3D tracking scan on the protection distance position of the traveling direction of the trolley at the current height of the spreader;

2. The 2D laser scanner scans along the traveling direction of the trolley to obtain 2D scan data of the cross section of the container area;

3. The embedded controller of the 3D laser unit calculates the 3D point cloud data (x _n , y _n , z _n ) of the dynamic origin of the 3D laser unit as it moves with the trolley, and uses the random sampling consistency segmentation method to point cloud data. The lines and planes in the cluster are clustered to obtain the smallest convex polygon that can surround all the input points, and then the vertices of the smallest convex polygon are extracted as the contour points of the container area and saved; meanwhile, the embedded controller uses the 2D laser scanner to scan The 2D scan data of the cross section of the container area is calculated to obtain the cross-sectional profile information of the bay area and the spatial position information of the spreader;

4. The embedded controller of the 3D laser unit utilizes the real-time spatial coordinates of the spreader center, the two axial movement trends of the spreader in the space, and the current working state of the spreader to determine whether the spreader and the container area contour will collide. According to the judgment result, the protection command of the spreader in the container area is sent to the RTG/RMG electronic control system to the RTG/RMG electronic control system.

Compared with the prior art, the positive effect of the present invention is that the present invention utilizes a real-time 3D laser unit with a positioning and positioning function, combined with a 2D laser scanner, through a pair of 3D laser points. The calculation of cloud data, the three-dimensional identification of the box outline, the car positioning and the fast feedback of the 2D laser plane data identify the bin area outline and the spatial position of the RTG/RMG spreader in real time, effectively solving the presence of the container terminal RTG/RMG. When the area is engaged in the loading and unloading operation, the real-time position detection of the shell and the adjacent-bay container pile in the spreader and the working area is comprehensively avoided, thereby avoiding the collision accident of the spreader.

DRAWINGS

The invention will be illustrated by way of example and with reference to the accompanying drawings in which:

Figure 1 is a logic block diagram of the spreader safety collision avoidance system;

2 is a top view of each scanning surface of the system during field bridge operation;

Figure 3 is a schematic diagram of a 3D tracking scan;

Figure 4 is a schematic diagram of scan data of a 2D laser scanner;

Figure 5 is a schematic diagram of scan data of a 3D laser unit.

Detailed ways

A spreader safety collision avoidance system based on three-dimensional identification of a container box outline, as shown in FIG. 1 , comprising: a 2D laser scanner, a 3D laser unit, and an RTG/RMG electronic control system PLC, wherein:

The 3D laser unit includes an embedded controller and a laser scanner, a motor driver, a dual DGPS differential positioning system, and the like respectively connected to the embedded controller; the motor driver is connected to the servo motor, and the laser scanner is disposed on the rotating head The rotating pan/tilt is connected to the reducer of the servo motor through the coupling; the laser scanner is connected to the embedded controller through the data line, and is used for transmitting the two-dimensional data of the real-time scanned container to the embedded controller, and simultaneously pressing Control commands issued by the embedded controller for three-dimensional scanning; the motor driver The signal line is connected to the embedded controller for accepting an instruction from the embedded controller to control the servo motor to drive the pan/tilt to rotate in a vertical direction; the dual DGPS differential positioning system is connected to the embedded controller through the data line, The position change information and the attitude change information of the detected 3D laser unit as a whole are transmitted to the embedded controller.

The 2D laser scanner is responsible for contour scanning under the RTG/RMG at a frequency of 25hz, and transmits the scanned data to the embedded controller of the 3D laser unit through the network switch of the 3D laser unit.

The RTG/RMG electronic control system PLC performs two-way communication with the embedded controller of the 3D laser unit at a frequency of not less than 2hz, receives the collision information and deceleration stop information of the current spreader, and simultaneously inputs the embedded controller to the 3D laser unit. Send information about the current RTG/RMG cart encoder, cart encoder, spreader size and spin lock status.

The 3D laser unit and the 2D laser scanner are installed under the RTG/RMG trolley platform, and the 3D laser unit is installed in front of the center position of the spreader at a distance of 20 cm from the edge of the spreader, so that the laser scanning surface of the 3D laser unit is perpendicular to the direction of the trolley. The 2D laser scanner is installed on the center of the spreader at a position of about 2 meters, and the 2D laser scanning surface is parallel to the direction of the trolley and perpendicular to the ground, and the 2D laser can scan the container and spreader to the bottom without the driver's cab platform. The occlusion is shown in Figure 2.

All laser data scanned by the 2D laser scanner and the laser scanner of the 3D laser unit are collected into the embedded controller of the 3D laser unit through the network switch built in the 3D laser unit. The embedded controller performs real-time processing on the point data scanned by the laser scanner in the 3D laser to obtain the outline of the stacking container, and at the same time, obtains the real-time spreader position by processing the laser data of the external 2D laser scanner. Stacking box at The relative positional relationship of the laser scanning plane projection in the direction of the trolley can be used to predict the collision avoidance; the embedded controller of the 3D laser unit and the PLC system of the RTG/RMG electronic control system exchange data through the network or other communication means to obtain the RTG/ RMG's current working status, and send the protection command of the spreader in the container area to its PLC, and the PLC finally sends control commands to the actuator (RTG/RMG cart inverter, trolley inverter, lifting inverter, etc.).

The invention also provides a real-time anti-collision method for a spreader based on three-dimensional identification of the contour of the container box, which comprises the following contents:

1. During the movement of the cart, when the cart moves to a new shell position, the laser scanner of the 3D laser unit is not completed before the full scan contour of the current shell position is completed, or when the spreader lifts a container 3D fast scanning of all 40 40-foot shells near the RTG/RMG (18 meters on each side of the laser), which can complete the scanning angle parallel to the direction of the cart up to 180 degrees in 1 second. The scanning angle of the trolley direction can be up to 60 degrees. And scan data once every 40 milliseconds (25 Hz) (including point cloud data of container outline, spreader and hanging box outline under the cart).

During the movement of the trolley, the embedded controller of the 3D laser unit performs real-time scanning position adjustment of the laser scanner according to the height of the spreader, and performs 3D tracking scan on the protection distance position of the traveling direction of the trolley at the current height of the spreader, and observes the hanging in real time. In the direction of travel, whether there are three adjacent shells in the work box area (current shell position and left and right adjacent shells), whether the container interferes with the travel route of the spreader:

3D tracking scanning means that after the current shell contour and the contour of the adjacent shell position have been acquired in the fast scanning process, as long as the current shell position is unchanged, the laser scanner of the 3D laser unit does not continuously oscillate and scan, but Tilt the scanning surface by a front angle θ before the spreader Fang, anti-collision protection detection of the container outline in front of the spreader.

As shown in Fig. 3, during the movement of the trolley, the 3D laser unit works in a 3D tracking scan mode, and the pan/tilt rotation angle θ of the 3D laser unit can be calculated according to the height Δh of the spreader:

The laser is kept to scan the spreader anti-collision distance D to obtain the traveling direction of the trolley, the local bay position of the anti-collision protection area and the laser scanning point of the adjacent shell position.

During the 3D data acquisition of the 3D laser unit, the attitude torsion and positional change of the 3D laser unit are always completed by its integrated dual DGPS differential positioning system.

Second, the 2D laser scanner scans along the moving direction of the trolley at a frequency of 25hz, and obtains the real-time position of the 2D laser scanner and the trolley relative to the cart and the container, and calculates the cross-sectional contour information of the shell-box area and the spatial position of the spreader. information;

3. The embedded controller of the 3D laser unit controls the laser scanner of the 3D laser unit to complete the longitudinal direction of the 3D laser unit (ie parallel to the direction of travel of the trolley) from 0° to 60° in one second, horizontally (ie large 3D scanning of the range of -5° to 185° in the direction of travel of the car.

The raw data is the 2D laser scanning ranging data of the 3D laser unit in different rotation planes. If the laser scanning angular resolution is 0.1667 degrees, then in a 3D laser unit scanning cycle, a laser scanning range of -5° to 185° will be obtained, and a total of 1141 laser points will be obtained in the form of:

{d ₀ ,d ₁ ,d ₂ ...d ₁₁₄₁ } (2)

According to the principle of polar coordinate and Cartesian coordinate system, each scanning point is in this laser sweep The coordinates in the drawing plane are:

x' _n =d _n ×cos(n×0.1667-5) (3)

y' _n =d _n ×sin(n×0.1667-5) (4)

If the 3D laser rotating pan/tilt is rotated by θ° with respect to the initial position at this time, after the 3D laser encoder data is fused, the origin can be obtained as the laser center, the traveling direction of the cart is the x-axis, and the traveling direction of the trolley is the y-axis. 3D data with lifting axis z-axis:

x” _n =x'_n; (5)

y” _n = y' _n × sin θ; (6)

z” _n = y' _n × cos θ; (7)

A dual DGPS differential positioning system integrated in the 3D laser unit obtains the spatial coordinates of the two GPS antennas in the geodetic coordinate system {x _g1 , y _g1 , z _g1 } and {x _g2 through two GPS antennas mounted on the top of the trolley , y _g2 , z _g2 }. In space, one of the antennas is used for positioning and the other is used for direction finding. The positioning antenna acquires the displacement difference in every two GPS data periods (100ms), and can be converted into the displacement difference of the 3D laser unit {Δx, Δy, Δz} according to the initial installation displacement of the positioning antenna and the 3D laser unit. The angle between the direction finding antenna coordinate and the positioning antenna coordinate in the space and the respective axial direction is converted into {Δα, Δβ, Δγ} according to the initial installation angle difference between the straight line connecting the positioning antenna to the DF antenna and the 3D laser unit. .

During the actual operation of the RTG, the laser is always in motion due to the movement of the cart and the cart. In order to solve the problem of laser origin movement, the high-precision positioning and positioning of the 3D laser unit is realized by the dual DGPS differential positioning system integrated in the 3D laser scanner, and the three-axis displacement of the 3D laser unit is obtained every 100 ms {Δx, Δy , Δz} and the three-axis rotation angle {Δα, Δβ, Δγ}. Obtaining the 3D laser unit when moving with the trolley Dynamic origin 3D point cloud data:

x _n =(x" _n +Δx)cosΔα (8)

y _n =(y" _n +Δy)cosΔβ (9)

z _n =(z" _n +Δz)cosΔγ (10)

The 2D laser scanner scans along the moving direction of the trolley to obtain 2D scan data of the cross section of the container area, and converts the coordinates of the Cartesian coordinate system according to formulas (2) to (4). It is positioned by the PLC encoder of the PLC to complete its dynamic coordinate system translation and data alignment.

In the PLC data: the position of the absolute value encoder of the trolley is combined with the position of the absolute value encoder of the lifting device and the installation position relationship between the spreader and the 3D laser unit, and the real-time spatial coordinates of the center of the spreader are calculated; the speed encoder and spreader of the trolley The lifting speed encoder obtains the two axial movement trends of the spreader in the space; the current working state of the spreader is obtained by the spreader lock and the box sensor.

4. In the working state of the system of the present invention, the 3D laser unit obtains the point cloud data of the box area during the real-time scanning process of the box area. First, the filtering algorithm of the point cloud raw data is performed to remove the clutter point to prevent its influence on the subsequent data processing. The specific steps are as follows:

The first step: calculate the input point cloud (total number of dots N), K (the number of neighboring points) mean, wherein the calculated K i-th point p _i D _i is the mean

Step 2: Calculate the K-mean standard deviation of the input point cloud

The third step: according to the set standard deviation coefficient μ and the number of neighboring points K, it is determined whether the detection point p _i is a wild point (outlier point), and the condition is a wild point, and filtering is performed.

d _i >μσ (13)

Then convert the laser point cloud data according to the steps in the previous section to obtain {x _n , y _n , z _n } for each laser point in the absolute coordinate system.

On this basis, the point cloud segmentation in the space is started, and the lines and planes in the point cloud data are clustered according to the random sampling consistency segmentation method.

Split plane method:

Step 1: Determine the plane from the random sampled minimum data volume (3 data points), and calculate the coefficients A, B, C, and D of the plane; 3 of the data points are p ₁ (x ₁ , y ₁ , z ₁ ), p ₂ (x ₂ , y ₂ , z ₂ ), p ₃ (x ₃ , y ₃ , z ₃ );

The second step: calculating the internal point set and the outer point set corresponding to the plane obtained in the first step, the point p _i satisfies the distance d _i from the plane is less than the set threshold d is the internal point M;

The third step: iteratively repeats the first two steps to find a plane with enough internal points; to improve the accuracy, the PCL library adds error evaluation parameters for finding the optimal plane, and after determining the optimal plane, the internal point can be used again. A further estimate of the plane coefficient is made.

Split line method:

The first step is to randomly sample the minimum data amount (2 data points) from the input point cloud to determine the straight line, and calculate the coefficients A, B, C, D, E, and F of the straight line; 2 of the data points are respectively p ₁ (x ₁ , y ₁ , z ₁ ), p ₂ (x ₂ , y ₂ , z ₂ );

The second step: calculating the inner point set and the outer point set corresponding to the straight line obtained in the first step, the point p _i satisfies the distance d _i from the straight line is less than the set threshold d is the internal point M;

The third step: iteratively repeat the first two steps to find a line with enough internal points; to improve the accuracy, the PCL library adds the error evaluation parameters for finding the optimal line, and after determining the optimal line, the internal point can be used again. A further estimate of the straight line factor is made.

Finally, the smallest convex polygon that inputs all points can be surrounded by calculation, and then the vertices of the polygon are extracted as the contour points of the point cloud. Save each bin outline point in the container area.

In the process of 3D laser calculation, the 2D laser always maintains the real-time scanning state, and the current job shell section is contoured to make up for the 3D laser unit to obtain the shell data in time when the container stacking level is high. In case, the contour recognition of the 3D laser unit is complemented in real time. See Figure 4.

Obtain the real-time dynamic position of the spreader center through the PLC data. In the space, use the mathematical cube to model the spreader, and combine the speed of the current trolley axis and the lifting axis to determine whether the spreader and the container area are contoured. There will be a collision. According to the judgment result, 3D The embedded controller in the laser unit sends a deceleration or stop command to the RTG/RMG PLC system to achieve the anti-collision function of the spreader in the box area.

Claims

A spreader safety collision avoidance system based on three-dimensional identification of a container box outline, comprising: a 3D laser unit and a 2D laser scanner disposed under the trolley platform, the 3D laser unit comprising an embedded controller and respectively a laser scanner connected to the embedded controller, a motor driver and a dual DGPS differential positioning system, the motor driver being connected to the servo motor, the laser scanner being disposed on the rotating pan/tilt, rotating the pan/tilt through the coupling and the servo motor The speed reducer is connected; the laser scanner is connected to the embedded controller through the data line, is used to receive the instruction issued by the embedded controller for scanning, and transmits the scanned data to the embedded controller in real time; the motor driver passes the signal The line is connected to the embedded controller for accepting an instruction from the embedded controller to control the servo motor to drive the pan/tilt rotation; the dual DGPS differential positioning system is connected to the embedded controller through the data line, and is used for detecting the 3D The position change information and the attitude change information of the laser unit as a whole are transmitted to the embedded controller;

The 2D laser scanner is connected to the embedded controller of the 3D laser unit through a data line, is used for contour scanning under the RTG/RMG, and transmits the scanned data to the embedded controller in real time;

The embedded controller of the 3D laser unit performs bidirectional communication with the RTG/RMG electronic control system to obtain the working status information of the RTG/RMG, and sends the protection command of the spreader in the container area to the RTG/RMG electronic control system.
The spreader safety collision avoidance system based on the three-dimensional identification of the container box area according to claim 1, wherein the 3D laser unit is installed in front of the center position of the spreader at a distance of 20 cm from the edge of the spreader, the 2D laser The scanner is installed at a position 2 meters away from the center of the spreader.
The spreader safety collision avoidance system based on three-dimensional identification of a container box outline according to claim 1, wherein a laser scanning surface of the 3D laser unit is perpendicular to a trolley direction; and a laser scanning surface of the 2D laser scanner Parallel trolley direction and perpendicular to the ground.
The spreader safety collision avoidance system based on three-dimensional identification of a container box profile according to claim 1, wherein the dual DGPS differential positioning system comprises two GPS mounted on the top of the trolley. Antennas, one for positioning and the other for direction finding.
A safety safety anti-collision method for a spreader based on three-dimensional identification of a container box area, characterized in that it comprises the following contents:

1. During the movement of the cart, when the cart moves to a new shell position, or the spreader locks up a container, the laser scanner of the 3D laser unit performs a 3D fast scan on the underside of the cart; In the process, the embedded controller of the 3D laser unit performs real-time scanning position adjustment on the laser scanner according to the height of the spreader, and performs 3D tracking scan on the protection distance position of the traveling direction of the trolley at the current height of the spreader;

2. The 2D laser scanner scans along the traveling direction of the trolley to obtain 2D scan data of the cross section of the container area;

3. The embedded controller of the 3D laser unit calculates the 3D point cloud data (x n , y n , z n ) of the dynamic origin of the 3D laser unit as it moves with the trolley, and uses the random sampling consistency segmentation method to point cloud data. The lines and planes in the cluster are clustered to obtain the smallest convex polygon that can surround all the input points, and then the vertices of the smallest convex polygon are extracted as the contour points of the container area and saved; meanwhile, the embedded controller uses the 2D laser scanner to scan The 2D scan data of the cross section of the container area is calculated to obtain the cross-sectional profile information of the bay area and the spatial position information of the spreader;

4. The embedded controller of the 3D laser unit utilizes the real-time spatial coordinates of the spreader center, the two axial movement trends of the spreader in the space, and the current working state of the spreader to determine whether the spreader and the container area contour will collide. According to the judgment result, the protection command of the spreader in the container area is sent to the RTG/RMG electronic control system to the RTG/RMG electronic control system.
The method for safely colliding a spreader based on three-dimensional identification of a container box area according to claim 5, characterized in that: 3D point cloud data (x n , y n , z n of the dynamic origin of the 3D laser unit as the vehicle moves) The calculation method is:

The first step is to convert the polar coordinates d n of each laser scanning point obtained by the 3D fast scanning into rectangular coordinates (x' n , y' n ) in the current laser scanning plane;

In the second step, calculate the 3D coordinates (x" n , y" n , z with the laser center as the origin, the direction of the cart as the x-axis, the direction of the cart as the y-axis, and the lifting direction of the spreader as the z-axis. ” n ):

x” n =x'n;

y” n = y' n × sin θ;

z” n = y' n × cos θ;

Where θ is the angle at which the rotating pan/tilt of the 3D laser unit rotates relative to the initial position;

In the third step, the 3D point cloud data (x n , y n , z n ) of the dynamic origin of the 3D laser unit as it moves with the trolley is calculated as follows:

x n =(x" n +Δx)cosΔα

y n =(y" n +Δy)cosΔβ

z n =(z" n +Δz)cosΔγ

Here, {Δx, Δy, Δz} is the displacement difference of the 3D laser unit, and {Δα, Δβ, Δγ} is the angular difference of the 3D laser unit.