KR102546156B1 - Autonomous logistics transport robot - Google Patents

Abstract

The present invention relates to a logistics transport robot used in large-scale stores, factories, etc., and the self-driving logistics transport robot according to the present invention includes a body unit 10 for loading logistics and a transfer unit 20 for moving the body unit 10 And composed of a lidar unit 30 provided in the body unit 10, when the lidar unit 30 detects a set target in a 3-dimensional space and generates 3-dimensional lidar data, the body unit 10 A self-driving logistics transport robot that processes 3D lidar data to follow the target, and the transfer unit 20 moves the body unit 10 according to a control command, wherein the body unit 10 is the transfer unit A motor 13 for transmitting power to 20, a motor driving board 14 for controlling the motor 13, a PC 15 for controlling the motor driving board 14 and the lidar unit 30, and It is composed of a battery 16 and a power supply board 17 that supplies power to the motor driving board 14 and the PC 15 by controlling the battery, and the PC 15 controls the adjacent data discrimination module 15-1. ), an intensity-based following target classification module 15-2, a following target determining module 15-3, and a following module 15-4, and real-time location data of objects in the working space of the lidar unit 30. The data collection step (S10) of collecting is performed, and the adjacent data classification module (15-1) performs the adjacent data classification step (S20) of grouping reflection points collected as 3D LiDAR data into a candidate group to be followed, The intensity-based follow-up target classification module 15-2 extracts a preferred follow-up target candidate group with a high possibility of being a follow-up target from the point group determined to be the follow-up target group derived from the adjacent data classification module 15-1. A candidate group extraction step (S30) is performed, and a following target determining module (15-3) performs a following target determining step (S40) in which the self-driving logistics transport robot determines a target to be followed, and the following module (15-3) is performed. 4) is technically characterized in that the self-driving logistics transport robot performs a target following step (S50) in which the self-driving logistics transport robot moves along the target while continuously following the target determined in the following target determination module 15-3.

Description

Autonomous Logistics Transportation Robot {AUTONOMOUS LOGISTICS TRANSPORT ROBOT}

The present invention relates to robots, and more particularly, to logistics transport robots used in large-scale stores, factories, and the like.

Autonomous driving means that mechanical devices such as cars, airplanes, and robots travel freely by relying on various sensors and computing systems installed on their own without borrowing external power. As the degree of development has reached a certain level, it is becoming a common technology in the 2020s.

The field where self-driving issues are highlighted the most recently is the automobile field. Simple road driving and parking are already being implemented, and unmanned cars and cars that do not require driver's operation are expected to become a reality.

Any field to which autonomous driving can be applied can be any device that involves movement of location, but the field of attention of the present invention is robots, in particular, logistics transportation that can be used in large marts or factories (hereinafter referred to as 'marts, etc.') This is the field of robotics. In an environment such as a mart, shelves, storage stands, work tables, etc. are scattered throughout the space, and workers move frequently, so in the case of an autonomous logistics transportation robot, it is necessary to be able to detect any obstacles in the space where the robot moves in real time. do. In particular, human detection is more important because it is closely related to the possibility of an accident.

Various technologies related to such human detection or tracking have been developed. HDL (High resolution real time 3D Lidar)-based tracking algorithm is based on 3D Lidar data and based on SLAM (Simultaneous Localization And Mapping). It is an algorithm that follows a person by removing background data and learning the upper body data of a person in advance using SVM (Support Vector Machine) technique.

TANet is a learning-based human-following algorithm using lidar and a depth camera. After converting the running result into a depth image-based algorithm, the exact position of the person is tracked using lidar.

In addition, 'Non-Patent Document 1' discloses an algorithm that follows a person by distinguishing and learning human leg data from Lidar data based on CNN (Convolutional Neural Network).

However, since these conventional algorithms are based on learning, there is a problem in which classification is impossible in a state different from the data used for learning (eg, when a person bends or shows a side, or when the upper and lower body cannot be distinguished). In addition, since the data used for learning is vast, there is a problem in that a lot of learning time is required. In the case of HDL, since the background is removed based on the prior map, there is a problem in that the generation of the prior map must be premised.

Therefore, when trying to implement an autonomous robot using conventional algorithms, the problems of conventional algorithms appear in the robot as it is.

Claudia Alvarez-Aparicio et al. People Detection and Tracking Using LIDAR Sensors. ROBOTICS. MDPI. 2019. 8. 31., 8, 75

The present invention has been made to solve the above problems, and the problem to be solved by the present invention is to provide an autonomous logistics transport robot capable of autonomously driving in real time without relying on learning and without requiring a prior map. will be.

The self-driving logistics transport robot according to the present invention is composed of a main body for loading logistics, a transfer unit for moving the main body, and a lidar unit provided in the body, and the lidar unit detects a set target in a three-dimensional space and When dimensional lidar data is generated, the body unit processes the three-dimensional lidar data to follow the target, and the transfer unit is an autonomous logistics transport robot in which the body unit moves according to a control command, wherein the body unit moves the transfer unit. Composed of a motor that transmits power, a motor drive board that controls the motor, a PC that controls the motor drive board and the lidar unit, and a battery and a power supply board that supplies power to the motor drive board and the PC by controlling the battery. The PC is composed of a neighbor data classification module, an intensity-based follow target classification module, a follow target determination module, and a follow module, and the lidar unit performs a data collection step of collecting real-time location data of objects in the work space, The adjacent data classification module performs the adjacent data classification step of grouping reflection points collected as 3D LiDAR data into a target candidate group to be followed, and the intensity-based following target classification module determines the candidate group to be followed derived from the adjacent data classification module. A priority target candidate group extraction step of extracting a preferred follow target candidate group with a high probability of being a follow target for the point group is performed, and a follow target determination step of determining a target to be followed by the self-driving logistics transport robot is performed by the follow target determination module. and performs a target following step in which the self-driving logistics transport robot moves along the target while the following module continuously tracks the target determined by the following target determination module.

The self-driving logistics transport robot according to the present invention can set an autonomous route by recognizing a target in a workspace in real time.

In addition, in the case of tracking a person as a target, robust real-time tracking of the target is possible by reflecting guesswork and measurement for each LiDAR measurement.

1 is a perspective view of an autonomous logistics transport robot according to the present invention
2 is a configuration diagram of an autonomous logistics transport robot according to the present invention
3 is a configuration diagram of a module assembly for detecting and following a target in a PC
4 is a flowchart of a detection and tracking method performed in a lidar unit and a module assembly
5 is a reflective vest that is an example of a reflector
6 is a result of the adjacent data classification step for a person
7 is an exemplary view showing the relationship between the distribution and center of two point groups;
8 is a result of a follow-up target determination step targeting a human
9 is a detailed flow chart of a goal following step
10 is data subsampled results
11 is a result of reinforcing reflector data

Hereinafter, an autonomous logistics transport robot according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a perspective view of an autonomous logistics transport robot according to the present invention, and FIG. 2 is a configuration diagram of an autonomous logistics transport robot according to the present invention. The self-driving logistics transport robot according to the present invention is composed of a main body 10 for loading logistics, a transfer unit 20 for moving the main body 10, and a lidar unit 30 provided in the main body 10, When the lidar unit 30 detects a set target, such as a carrier, in a 3-dimensional space and generates 3-dimensional lidar data, the body unit 10 processes the 3-dimensional lidar data to follow the target, and the transfer unit 20 The main body 10 is moved according to the control command. The self-driving logistics transport robot according to the present invention may be configured to further include a camera unit 40 to supplement the lidar unit 30.

The main body unit 10 is a space in which the self-driving logistics transport robot according to the present invention moves from the three-dimensional data collected by the lidar unit 30 (hereinafter referred to as a 'work space'. It may be inside a logistics warehouse, inside a factory, etc. ), it is a component that enables autonomous driving by estimating the position and speed of objects (which can be plural) in real time, and determining and following the target among the objects.

The transfer unit 20 is composed of wheels or endless tracks and is a component that transports the body unit 10 under the control of the body unit 10 .

The lidar unit 30 is a component that emits light and senses the intensity of the emitted light reflected from an object in the work space and transmits it to the body unit 10. In the present invention, a person is set as the main target, Since there is a reflection point defined in a three-dimensional space {the point where light is reflected from an object and captured by the lidar unit 30, even if the light is reflected, the lidar unit 30 reflects weak light enough to be determined as intensity 0 It is configured to include a 3D lidar 31 to detect a point cannot be a reflection point}, and to further include a 2D lidar 32 to collect 2D data to be used for position correction of an object. can The 2D lidar verifies the accuracy of the position of an object that can be obtained through the 3D lidar data collected from the 3D lidar 31 in the present invention, or detects a temporary target when the 3D lidar 31 is abnormal. And it is provided for the purpose of performing control.

The camera unit 40 is a component for generating an image for image analysis, and is used to supplement tracking target classification by 3D lidar data obtained from the lidar unit 30. For example, by detecting the color of the tracking target, it is possible to know whether the tracking target predicted by the 3D LIDAR data is appropriate or not, and when the target cannot be distinguished in the region of interest, the target can be specified by color. When the camera unit 40 is configured as a depth camera, it may also be verified that position estimation based on 3D LIDAR data is accurate.

The body unit 10 includes a sensor 11 such as an infrared sensor and an ultrasonic sensor, an MCU board 12 that collects data sensed by the sensor 11, a motor 13 that transmits power to the transfer unit 20, and a motor ( 13), a motor driving board 14 controlling the MCU board 12, a motor driving board 14, a lidar unit 30, a PC 15 controlling the camera unit 40, and a battery 16 and a battery It is composed of an MCU board 12, a motor driving board 14, and a power supply board 17 that supplies power to the PC 15 by controlling. Data and control commands may be transmitted and received between the PC 15 and the MCU board 12, the motor driving board 14, and the power supply board 17 through RS232 or CAN communication. The sensor 11 and the MCU board 12 can be omitted if there is no necessary data other than the 3D lidar data by the lidar unit 30.

The PC 15 processes the sensed 3D LIDAR data to detect and follow a target, and this target detection and tracking may be performed in terms of firmware or software. In the present invention, firmware and software are not distinguished, and the concept of a module (which may be implemented in hardware or software, but is the same in terms of technology, so it is not distinguished) is approached.

3 is a configuration diagram of a module assembly for detecting and following a target in a PC, and FIG. 4 is a flowchart of a detection and tracking method performed in a lidar unit and a module assembly. In the present invention, the PC 15 is composed of a neighbor data classification module 15-1, an intensity-based following target classification module 15-2, a following target determining module 15-3, and a following module 15-4. It is configured to include a module assembly, and a data sub-sampling module 15-5, a strength data recovery module 15-6, and an initialization module 15-7 to be followed may be further included in the module assembly.

The lidar unit 30 performs a data collection step (S10) of collecting real-time location data of objects in the work space as a preliminary step of each step performed in the module assembly. Since the collection of 3D lidar data is performed in the lidar unit 30, it is preferable that a reflector is provided at the target so that the lidar unit 30 can detect the target well. 5 is an image of a reflective vest that is an example of a reflector that can be adopted when the target is a person.

3D LiDAR data is collected based on the self-driving logistics transport robot (in other words, a relative coordinate system), and the 3D LiDAR data collected from the lidar unit 30 can be expressed as a plurality of reflection points in a 3D space. A set is called a cloud point.

The neighbor data classification module 15-1 performs a neighbor data classification step (S20) of grouping reflection points collected as 3D lidar data into a candidate group to be followed. When there are multiple people wearing reflective vests, point clouds with dense reflection points are generated corresponding to the location of people, so classifying adjacent data is similar to recognizing a candidate group to follow located in the workspace.

If an object is in the same or close line of sight when viewed from the lidar unit 30, it is necessary to distinguish it because cloud points reflected from the object overlap. 6 illustrates the result of the neighbor data classification step for a person, and it can be seen that human-shaped points in an orange box form a point group (thus, they become a candidate group to be followed).

The step of classifying adjacent data (S20) is performed through segmentation and recombination of point clouds. First, if there are a plurality of point clouds and the distances between the plurality of point clouds are sufficiently far apart, it is sufficient to divide the point clouds because each candidate to be followed can be regarded as a case where the point clouds correspond 1:1.

Next, if the distance between two or more point clouds is adjacent, one follow-up target candidate can be composed of the above two or more point clouds. do. In the present invention, the distance between point clouds is used for this determination. Since the present invention is based on the premise that a robot follows a human, when determining a point cloud corresponding to a human, if the point cloud and the point cloud are within a range smaller than the size (height, shoulder width, etc.) of the human, the corresponding point clouds can be determined as one point cloud. there is. For example, no matter how large a person's shoulder width is, it does not exceed 1 m, and since the center of a plurality of people's bodies is rarely within 50 cm, ) If the distance is 50 cm or less, the point cloud can be recombined as belonging to one follow-up target candidate.

The classification of adjacent data can be problematic when two or more point clouds are close to each other at a similar distance. In this case, recombination of point groups can be determined by considering the distribution of point groups.

7 is an exemplary view showing the relationship between the distribution and center of two point groups. For simplicity of explanation, two point clouds (first and second point clouds) and a 2-dimensional distribution are given as an example, but the technical means of solving the problem is the same even in the case of three or more point clouds and a 3-dimensional distribution (in the case of a 3-dimensional distribution) multivariate Gaussian distribution can be used). When the standard deviation (σ) of the first point group is small as shown in FIG. 7(a) and the standard deviation of the first point group is large as shown in FIG. ) distance is shorter in the case of FIG. 7(a) than in the case of FIG. 7(b), but since the standard deviation of FIG. 7(a) is smaller than that of FIG. The probability of belonging to the first point cloud is higher in the case of FIG. 7(b), and therefore it is more probabilistically correct that the second point cloud belongs to the first point cloud of FIG. 7(b) and recombines. That is, in the present invention, the recombination of adjacent data is carried out between corresponding point clouds when the center distance between point clouds is less than a predetermined value (50 cm in the above example), and if there are a plurality of second point clouds whose center distance from the first point cloud is less than a predetermined value It is made between the first point cloud and the second point cloud in which the value of (central distance between the first specific point cloud and the second point cloud) ÷ (the standard deviation of the first specific point group above) is the smallest.

The location of each candidate to be followed can be determined by the point cloud classified in the above way. As the self-driving logistics transport robot continues to move, the front of the robot changes, making it difficult to simply display the position of the tracking target with (x, y) coordinates. ) Including θ, the data form is (x, y, θ), and the x, y coordinates are also measured with the robot's position as the origin (x, y axes do not rotate, only the origin is moved in parallel to the robot's position ), and sets a predetermined region in front of the robot as a region of interest (ROI) to collect 3D lidar data only within the region of interest. The setting of such a region of interest has a great effect on reducing the amount of computation together with data subsampling, which will be described later.

The intensity-based follow-up target classification module (15-2) is divided and recombined in the adjacent data classification module (15-1), and preferential follow-up is performed to extract a preferred follow-up target group with a high possibility of being a follow-up target for the point group determined as a follow-up target group. A target candidate group extraction step (S30) is performed. The reason why the step of extracting the candidate group to be followed (S30) is necessary first is that the low-density point cloud in the present invention is not an actual candidate group to be followed, but is highly likely to be noise caused by diffuse reflection of an object or floor. As an extreme example, if there are one or two points belonging to a point group determined as a candidate to be followed, and the distance is sufficiently far from other point clouds, there is no need to determine this as a candidate to be followed. Therefore, if there are more than a predetermined number of reflection points having a reflection intensity of a predetermined value or more in the point group determined as a candidate group to be followed, it can be classified as a candidate group to be followed first. According to the experiment of the present inventors, when a person is wearing an appropriate reflector such as a reflective vest, whether there are 100 or more points with a reflective intensity of 255 or more in the point group is an appropriate criterion for determining a candidate group to be followed first. Above, the reflection intensity 255 is an example (8 bits) of a value where the reflection intensity is saturated when the reflection of a reflective vest is measured in a general environment. Depending on lidar performance, the resolution of reflection intensity can be further increased. In other words, since the point with the above reflection intensity of 255 or more is the reflection point corresponding to the reflective vest, it is safe to consider it as a candidate to be followed first.

The following target determining module 15-3 performs a following target determining step (S40) of determining a target to be followed by the self-driving logistics transport robot. That is, the following target determination step (S40) is a step of determining one target to be followed by the self-driving logistics transport robot according to the present invention. FIG. 8 illustrates the result of the step of determining a follow-up target targeting a person, and shows a state in which only the target is displayed with other follow-up target groups shown in FIG. 6 removed. In the present invention, one follow-up target is determined as a preferred follow-up target closest to the self-driving logistics transport robot according to the present invention among preferred follow-up target candidates extracted by the strength-based follow-up target classification module 15-2. This is because the self-driving logistics transportation robot according to the present invention is driven first by a carrier using the robot, and therefore is closer to the robot than other candidates for priority following.

The following module 15-4 performs a target following step S50 in which the self-driving logistics transport robot moves along the target while continuously following the target determined by the following target determination module 15-3. 9 is a detailed flow chart of a goal following step. Target tracking estimates the current position (χ _t , ψ _t ) from the previous data (x _{t -1} , y _{t - 1} ) of the target, and measures the current target position (measurement) in the lidar unit 30 Kalman filtering that corrects the measured current target position data with the input value of the next position estimation when x _t , y _t ) is within a predetermined error range with the estimated position (χ _t , ψ _t ) can be performed by If the distance between the measured target position and the estimated position exceeds the predetermined error range (e), it means that the measured position may not be the target that the self-driving logistics transport robot wants to follow, so it follows the wrong target. In order to prevent it from being followed, the next location estimation is performed again by using the current location data (x _t , y _t ) = (χ _t , ψ _t ) estimated from the previous location data of the target without correction as an input value. If the next position (χ _{t +1} , ψ _{t +1} ) and the position (x _{t +1} , y _{t + 1} ) measured by the lidar unit 30 are again within a predetermined error range, it is determined that the target has been captured again Otherwise, the next position (χ _{t +1} , ψ _{t +1} ) is estimated based on the estimated position data (χ _t , ψ t ). In addition, if the estimation continues for more than a predetermined number of times (3 in the example of FIG. 9 ) without correcting by measurement, the following target determination step (S40) in the region of interest may be performed again after the reset.

The self-driving logistics transportation robot according to the present invention performs target detection and tracking in the PC 15, but since it is difficult to have a high-performance PC 15, it is necessary to reduce the amount of calculation by an algorithm, and for this purpose, data subsampling A module 15-5 and a strength data recovery module 15-6 may be further provided.

The data sub-sampling module 15 - 5 may perform a data sub-sampling step ( S60 ) of sampling only a part of the data collected by the lidar unit 30 . Since the data is reduced, the amount of calculation in the processing of subsequent modules such as the adjacent data classification module 15-1 can be significantly reduced. According to the experiment of the present inventors, the quality of the final target tracking did not significantly deteriorate even when only about 1/10 of the collected data was subsampled. 10 illustrates data subsampled results, and it can be seen that several point clouds appear in the region of interest. For reference, the red pipe represents the x-axis and the green pipe represents the y-axis.

The intensity data recovery module 15 - 6 performs a reflector data augmentation step ( S70 ) of adding high reflection intensity reflection points reflected from the reflector to the subsampled data. Accordingly, data dilution due to subsampling can be greatly alleviated. FIG. 11 illustrates the result of reinforcing reflector data, and it can be seen that more 'V'-shaped reflection points of the reflective vest are added compared to FIG. 10 . The reflector data can be configured by extracting points where the beam intensity measured by the lidar unit 30 is equal to or greater than a predetermined threshold value (reflection intensity 255 in the case of the example above).

When the data collected in the data collection step (S10) is collectively subsampled, the reflection points of the reflector that strongly indicate the position of the target are uniformly reduced, and accordingly, the density of the point group corresponding to the target is lowered, making it difficult to determine the distribution. Inter-distance judgments also become inaccurate. Adding the reflection points of the reflector to the subsampled data significantly increases the number of reflection points around the peak of the point group (the center of the point group has the largest number of reflection points even if it is collectively subsampled), so the distribution of the point group becomes more rapid ( sharp) Therefore, the effect that the position of the target is well specified appears. Subsequently, if the adjacent data classification step (S20) is performed, the division and recombination of point clouds can be performed more clearly.

As described above with reference to FIG. 9, the follow-up target initialization module 15-7 determines that the target follow-up becomes impossible when the estimation continues for more than a predetermined number of times (3 in the example of FIG. 9) without correction by measurement, and follows A target initialization step (S80) is performed. Therefore, after the reset, a following target is set through the following target determination step (S40) again in the region of interest.

10 body part 11 sensor
12 MCU board 13 motor
14 motor driving board 15 pcs
15-1 Adjacent Data Classification Module 15-2 Intensity-Based Tracking Target Classification Module
15-3 Tracking target determination module 15-4 Tracking module
15-5 Data Subsampling Module 15-6 Intensity Data Recovery Module
15-7 Follow Target Initialization Module 16 Battery
17 power supply board 20 transport
30 lidar part 31 3-dimensional lidar
32 2D lidar 40 Camera unit
S10 data collection step S20 adjacent data classification step
S30 Priority Follow-up Target Candidate Extraction Step
S40 Tracking Goal Determination Step S50 Goal Tracking Step
S60 data subsampling step S70 reflector data augmentation step
S80 follower initialization step

Claims (5)

It consists of a body part 10 for loading logistics, a transfer part 20 for moving the body part 10, and a lidar part 30 provided in the body part 10, and the lidar part 30 is set. When a target is sensed in a 3D space and 3D lidar data is generated, the main body 10 processes the 3D lidar data to follow the target, and the transfer unit 20 performs a control command on the main body. As an autonomous logistics transport robot that moves the unit 10,
A reflector is provided on the target,
The body part 10 transmits power to the transfer part 20, the motor 13, the motor driving board 14 controlling the motor 13, the motor driving board 14 and the lidar part 30 ) and a battery 16 and a power supply board 17 that supplies power to the motor driving board 14 and the PC 15 by controlling the battery,
The PC 15 includes a neighbor data classification module 15-1, an intensity-based follow target classification module 15-2, a follow target determination module 15-3, a follow module 15-4, and a data subsampling module. (15-5) and intensity data recovery module (15-6),
The lidar unit 30 performs a data collection step (S10) of collecting real-time location data of objects in the work space,
The adjacent data classification module 15-1 performs the adjacent data classification step (S20) of grouping reflection points collected as 3D LiDAR data into a candidate group to be followed,
A preferred follow-up target in which the intensity-based follow-up target classification module 15-2 extracts a preferred follow-up target candidate group with a high possibility of being a follow-up target for the point group determined to be the follow-up target group derived from the adjacent data classification module 15-1 A candidate group extraction step (S30) is performed, but the priority target candidate group is a target candidate group having a predetermined number or more reflection points having a reflection intensity of the reflector of a predetermined value or more among the target candidate groups grouped by the neighbor data discrimination module 15-1.
The following target determination module 15-3 performs a following target determination step (S40) of determining a target to be followed by the self-driving logistics transport robot;
The following module 15-4 performs a target following step S50 in which the self-driving logistics transport robot moves along the target while continuously following the target determined in the following target determination module 15-3,
In the data collection step 10, after the data subsampling module 15-5 performs a data subsampling step S60 of sampling only a part of the data collected by the lidar unit 30, the intensity data recovery module (15-6) performs a reflector data augmentation step (S70) of adding high reflection intensity reflection points, which are points at which the beam intensity measured by the lidar unit 30 is equal to or greater than a predetermined threshold, to the subsampled data. Self-driving logistics transport robot.
The method of claim 1,
The neighbor data segmentation module 15-1 performs segmentation and recombination of a point group that is a set of reflection points,
An autonomous logistics transport robot characterized in that it determines whether two or more point clouds should be viewed as one follow-up target candidate.
The method of claim 2,
The self-driving logistics transport robot, characterized in that for recombining the point clouds as belonging to one follow-up target candidate when the target is a person and the central distance between the two or more point clouds is 50 cm or less.
The method of claim 1,
The follow-up target determination module 15-3 determines the preferred follow-up target candidate closest to the self-driving logistics transport robot among the preferred follow-up target candidates extracted by the intensity-based follow target classification module 15-2 as a target to be followed Self-driving logistics transport robot, characterized in that.
The method of claim 1,
The tracking module 15-4 estimates the current position (χ _t , ψ _t ) from the previous data (x _{t -1} , y _{t - 1} ) of the target, and the current target measured by the lidar unit 30 When the position (x _t , y _t ) is within a predetermined error range with the estimated position (χ _t , ψ _t ), the measured current target position data is corrected with the input value of the next position estimation to guess the next position Autonomous driving logistics transport robot.

Images