KR101919463B1 - Gripper robot control system for picking of atypical form package - Google Patents

Abstract

The present invention discloses a logistics robot control system capable of atypical picking. A control system for a robot that can pick up atypical digits according to the present invention includes a package detector for extracting stereoscopic image data, image data, and distance measurement data, analyzing stereoscopic image data, image data, and distance measurement data, And a robot driving control unit for driving the picking robot to grasp and move the gripping position of the package according to the gripping control signal, By using the laser detection module, it is possible to recognize the atypical package automatically and accurately analyze the position of the package, so that the package can be grasped and moved accurately.

Description

TECHNICAL FIELD [0001] The present invention relates to a robot control system capable of picking up atypical digests,

More particularly, the present invention relates to a control system for a robot capable of picking up a product, and more particularly, to a control system of a robot capable of picking up a product, To a robot control system capable of picking irregular packages.

In logistics centers such as logistics storage and storage, a wide variety of logistics and parcels are transported and loaded. In recent years, a picking robot for gripping and sorting the goods being transported through the transporting conveyor or the transporting device has been applied to reduce the transporting time and manpower.

However, since many kinds of parcels are transported and processed in a single unit at a distribution center such as a market or a shopping mall, utilization of the picking robots is remarkably lowered. That is, conventional picking robots have difficulty in processing various kinds of irregular packages due to limitation of package detection and shape adaptive gripping technology. Since the picking tool according to the prior art has structural limitations in general-purpose gripping of objects of various shapes and materials, and it is difficult to ensure the grip reliability of the picking for the picking center. Accordingly, in the process of processing a package of a variety of goods to be transported in a single unit, it takes more manpower and time.

Especially, in the logistics process of the online shopping mall, since many kinds of parcels are mainstream, the peaking process which requires the greatest manpower and time, that is, the automation of the location confirmation and the loading / unloading, is more urgently required.

In order to automatically pick all the packages packed in various sizes, such as packaged packages, packaged packages of various sizes and shapes, box packages, plastic packages, plastic packages, etc., a picking operation Robots must be developed. To this end, it is necessary to develop a picking tool with a built-in detection sensor and an integrated picking operation control technology in conjunction with a mobile platform.

However, first of all, the most important technology that needs to be developed and commercialized is packet recognition technology. Up until now, due to limitations of package recognition technology, the technology of automating the picking operation itself has been insufficient, and the development of picking robots for transporting goods has not been developed yet. Therefore, it is necessary to develop a package detection and control module that can more quickly and accurately grasp and load packages.

Korean Patent Publication No. 2014-0081177

SUMMARY OF THE INVENTION The present invention has been made to solve various problems as described above, and it is an object of the present invention to provide a 3D vision and a laser detection module that automatically recognize irregular parcels, accurately analyze the grip positions of parcels, The present invention provides a control system for a logistics robot capable of picking up irregular packages.

Particularly, it is an object of the present invention to provide a control system of a logistics robot which can utilize a picking robot and minimize manpower and time consumption through a picking tool and a package recognition technology which can be applied universally to picking operations for multi-item packages. have.

In addition, a stereoscopic image camera, an image capturing camera, and a laser sensor are used to accurately detect the distance between the picking robot and the corresponding package in three dimensions and automatically analyze the package to the gripping position, thereby reducing the picking time and improving the accuracy. And a control system.

According to an aspect of the present invention, there is provided a control system for a robot that can pick up atypical digits, comprising: a package detector for extracting stereoscopic image data, image data, and distance measurement data; A central processing unit for analyzing the data to detect the position of the package and the grip position of the package, and outputting a grip control signal according to the detection result, and a central processing unit for operating the gripping robot to grasp the grip position of the package according to the grip control signal And a robot drive control unit.

In order to achieve the above object, a parcel detecting unit includes a stereoscopic image extracting unit for picking up an image within a grip range of a picking robot and outputting a stereoscopic image signal in units of at least one frame, A distance measurement unit for outputting a distance signal by emitting a laser signal to a grasping range of the picking robot, and a distance measuring unit for measuring a distance between the 3D stereoscopic image signal and the image signal and the distance signal And a signal conversion storage unit for outputting the 3D stereoscopic image data, the image data, and the distance data at least every one frame.

In addition, the central processing unit may include a package detection unit for receiving the stereoscopic image data, the image data, and the distance data in units of at least one frame, analyzing the position of the center point of the package by real- A grip position extracting unit for calculating distance data between the outer shape of the package and the outer shape of the package to extract gripable positions of the package and distance data between the outer shape, size and outer shape of the package, A grip position setting unit for setting a grip position and a grip control signal for generating and outputting a grip control signal so as to be grasped by the picking robot according to a result of setting one grip position, And a grip control section for further outputting a movement control signal do.

According to an embodiment of the present invention having various technical features as described above, a control system for a robot that can pick up atypical digits automatically detects irregular packages using 3D vision and a laser detection module. In addition, an accurate analysis of the grip position of the package allows the package to be grasped and moved accurately.

In particular, the picking tool and the package recognition technology, which can be generally applied to the picking operation for a multi-item package, increase the utilization of the picking robot and minimize labor and time consumption. If a picking robot system capable of picking unmanned is developed, it is expected that more than 90% of the logistics processing automation will be possible in the field of automatic picking automation.

In addition, in the present invention, the distance between the picking robot and the package is precisely detected in three dimensions using a stereoscopic image camera, an image capturing camera, and a laser sensor. By automatically analyzing the position of the package to the gripping position, it is possible to reduce the picking time but improve the accuracy. Accordingly, the present invention can anticipate economical effects such as an increase in the throughput of the logistics, a cost of the logistics, and a decrease in the peaking error rate.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram specifically illustrating a control system for a logistics robot capable of picking up atypical digests according to an embodiment of the present invention; FIG.
FIG. 2 is a block diagram specifically illustrating the configuration of the package detection unit and the central processing unit shown in FIG. 1;
3 is an image diagram for explaining a stereoscopic image extraction and a package detection process in detail.
FIG. 4 is an image diagram for explaining a package position detection process of the package detection unit shown in FIG. 2; FIG.
FIG. 5 is an image diagram for explaining the procedure of extracting a parcel position of a parcel by the parcel position extracting unit shown in FIG. 2;
FIG. 6 is an image diagram for explaining the procedure of setting a parcel position of a parcel by the parcel position setting unit shown in FIG. 2;
FIG. 7 is a graph showing the results of the parcel detection accuracy and the grip position extraction accuracy according to the present invention.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. It is to be understood that the embodiments and terminology used herein are not intended to limit the invention to the particular embodiments, but are to be construed to cover various modifications, equivalents, and / or alternatives of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram specifically illustrating a control system for a logistics robot capable of picking up atypical digests according to an embodiment of the present invention; FIG.

The control system shown in FIG. 1 includes a package detecting unit 100 for extracting 3D stereoscopic image data, image data, and distance measurement data, and a controller 110 for analyzing 3D stereoscopic image data, image data, and distance measurement data, A central processing unit 200 for detecting a grip position and outputting a grip control signal according to the detection result, and a robot drive control unit (not shown) for driving the picking robot 400 so as to grasp and move the grip position of the package according to the grip control signal 300).

The package detection unit 100 may include various sensors for 3D image, color image, distance measurement, etc., a conversion unit for converting the respective sensing signals, and a minicomputer. Accordingly, the picking robot 400 may be configured separately from the periphery, or some components may be provided on the inner and outer case of the picking robot 400 or the like.

Specifically, the package detection unit 100 detects a 3D image through a stereo camera capable of outputting a 3D stereoscopic image, a CCD and RGB camera capable of extracting a color image, and a laser sensor capable of measuring a distance between the objects facing the stereoscopic image, Extracts stereoscopic image data, image data, and distance measurement data. The 3D stereoscopic image data, the image data, and the distance measurement data extracted by the package detector 100 are data on the parcels within the range that the picking robot 400 can pick and the background data.

The central processing unit 200 may include a central processing unit (CPU) including at least one micro controller unit (MCU). The central processing unit 200 may be separately provided outside the picking robot 400, but may be provided in the picking robot 400.

The central processing unit 200 analyzes 3D stereoscopic image data, image data, and distance measurement data extracted in real time from the package detector 100 in real time. Based on the analysis result, the center point position of the package is analyzed and the appearance shape, size and distance data of the package are calculated based on the center point position. When the exterior shape, size, and distance data of the package are calculated, the gripping position of the package that can be gripped by the picking robot 400 is detected based on the external shape, size, and distance data. And outputs a grasping control signal so as to be grasped by the picking robot 400 according to the detection result. In addition, a movement control signal may be further output to move the gripped piece to a predetermined position.

The robot driving control unit 300 may be built in a barping robot 400 including at least one mini computer or an MCU. The robot driving control unit 300 controls various motors of the picking robot 400 and driving elements such as a hydraulic pump so that the picking robot 400 can grasp the gripping position of the package according to the gripping control signal from the central processing unit 200. [ Lt; / RTI > When the picking is completed, the picking robot 400 drives and controls the picking robot 400 so as to move the picked up pile according to the movement control signal from the central processing unit 200.

FIG. 2 is a block diagram specifically illustrating the configuration of the package detection unit and the central processing unit shown in FIG. 1;

2 includes a stereoscopic image extracting unit 110 for picking up an image within a holding range of the picking robot 400 and outputting a 3D stereoscopic image signal in at least one frame unit, a picking robot 400 A distance measuring unit 130 for emitting a laser signal up to the grasping range of the picking robot 400 and outputting a distance signal, ), And a signal conversion storage unit (140) for digitally converting the 3D stereoscopic image signal, the image signal, and the distance signal of at least one frame unit and outputting 3D stereoscopic image data, image data, and distance data at least every one frame unit do.

The stereoscopic image extracting unit 110 includes at least one stereo camera. At least one stereo camera may be provided on the outer side of the picking robot 400, the case of the picking robot 400, or the like. At least one stereo camera captures an image within the holding range of the picking robot 400 at least one frame and outputs at least one frame of 3D stereoscopic video signal to the signal conversion storage unit 140. Here, the stereo camera shoots a 3D stereoscopic image through the RGV-D camera sensor, resulting from its ability to produce better quality and depth information. Accordingly, the stereo camera is preferably configured on the body or the front portion of the picking robot 400.

The image capturing unit 120 is configured to include at least one CCD sensor, an RGB camera, and the like. The CCD sensor or the RGB camera may be provided on the outer side of the picking robot 400 or the case of the picking robot 400. [ The CCD sensor, the RGB camera, or the like captures an image within the holding range of the picking robot 400 in units of frames, and supplies the image signal to the signal conversion storage unit 140 at least every one frame. Accordingly, the CCD sensor and the RGB camera are preferably formed on the body or the front portion of the picking robot 400.

The distance measuring unit 130 includes at least one laser sensor. At least one laser sensor is provided on the outer side of the picking robot 400, the case of the picking robot 400, and the like. The at least one laser sensor real-time measures the distance from the body of the picking robot 400 to the package to be delivered within the gripping range of the picking robot 400 in real time. And supplies the real-time distance signal to the signal conversion storage unit 140 on a frame-by-frame basis.

The signal conversion storage unit 140 includes at least one AD conversion device and a short-range communication module such as an RS-232 interface device. Accordingly, the signal conversion storage unit 140 converts the 3D stereoscopic image signal, the image signal, and the distance signal of at least one frame unit into digital data. And supplies the 3D stereoscopic image data, the image data, and the distance data to the central processing unit 200 in units of at least one frame through a short-range communication module such as an RS-232 interface device.

2, the central processing unit 100 includes a package detection unit 210, a grip position extraction unit 220, a grip position setting unit 230, and a grip control unit 240.

The package detection unit 210 analyzes the center point position of the package by receiving the 3D stereoscopic image data, the image data, and the distance data in units of at least one frame and performing real-time analysis by combining them. To this end, the package detection unit 210 maps the 3D stereoscopic image data to image data and a depth map based on a preset mapping function.

3 is an image diagram for explaining a stereoscopic image extraction and a package detection process in detail.

Referring to FIG. 3, the package detector 210 includes a SURF algorithm (Fast Up Robust Features algorithm) and a Fast Library for Approximate Nearest Neighbors algorithm (FLANN algorithm) to accurately detect target packages Apply sequentially.

Specifically, the package detector 210 maps the 3D stereoscopic image data to the image data and the depth map based on the mapping function, and calculates the Hessian matrix of the point X and the σ in the mapping image using Equation 1 and Equation As well.

[Equation 1]

Here, L _xx , L _yy , L _xy , and L _yx are constants of the Gaussian second derivatives such as the following Equation 2 for the image of the coordinate point (x, y).

&Quot; (2) "

FIG. 4 is an image diagram for explaining a package position detection process of the package detection unit shown in FIG. 2; FIG.

As shown in FIG. 4, the package detector 210 sets the position coordinates of the image of (x, y) using the approximate values of the discrete Gaussian derivative filter and the corresponding box filter of Equation (2).

The approximate Gaussian Hessian determinant for setting the positional coordinates of the image of (x, y) can be defined as Equation (3) below.

&Quot; (3) "

Here, the scale size of the Box filter starts from the Gaussian derivative having a default size of 9 × 9 having σ of 1.2, and can be gradually increased to 15 × 15, 21 × 21, 27 × 27, and the like.

Approximate Gaussian determinants of the Hessian matrix are computed on each scale, and non-maximum suppression in a 3 × 3 contiguous range is applied to find even more sharp edges by finding the maximum value. In this case, the scale value of the SURF points is obtained as a maximum value. Then, the direction of the obtained SURF point is obtained in both the x and y directions using the HWR (Harr-Wavelet Response), and the dominant orientation direction is determined. A magnitude normal angle centered at the SURF point in this direction is constructed. Then, by comparing the file descriptors obtained from the two images, a matching pair is obtained, and the FLANN algorithm is used for matching.

FIG. 5 is an image diagram for explaining the procedure of extracting a parcel position of a parcel by the parcel position extracting unit shown in FIG. 2; And FIG. 6 is an image diagram for explaining the procedure of setting a parcel position of a parcel by the parcel position setting unit shown in FIG.

5 and 6, the gripping position extracting unit 220 calculates distance data between the outer shape, the size, and the outer shape of the package based on the position of the center point of the package, and extracts possible positions of the package.

For this, the grip position extracting unit 220 extracts two points based on the position of the center point of the package in the data mapped to the depth map, using a Rectangle Representation algorithm. Then, distance data between the outer shape and size of the package and the outer shape are calculated based on the position of the central point of the package. Here, the rectangular representation algorithm is a well-known algorithm for obtaining a maximum rectangle of an orthogonal, monotone or block polygon, and is used in the present invention to arbitrarily extract two points that are preferred to obtain a rectangle. Basically, two points need to be found basically in order to get or get a rectangle. Specifically, the grip position extracting unit 220 extracts two points based on the center point position of the object to be gripped in the mapping data. Then, the distance data between the outer shape, the size, and the outer shape of the object to be gripped is calculated based on the center point position of the object to be gripped. At this time, the maximum depth is searched based on the two points, and the distance data between the outer shape, the size, and the outer shape of the package is calculated to extract the positions where the package can be gripped.

More specifically, the grip position extracting unit 220 first extracts two points based on the depth map data obtained from each image data by using a rectangular representation algorithm. Thus, the maximum depth can be searched based on two points to calculate the distance data between the outer shape, the size, and the outer shape of the package.

Then, the grip position extracting unit 220 determines two points based on the depth map data, and extracts the depth data sequentially from the two points to the peripheral portion, thereby distinguishing the appearance of the package from the background of the package. Here, the distance between the two points may be a distance within the gripper grip range of the picking robot 400. Thus, the range of package appearance at distances less than the distance between two points can be gripping positions.

6, in the case of the grip position setting unit 230, one of the grip positions that can be grasped through the distance data analysis between the outer shape, the size, and the outer shape of the package Setting.

Specifically, the grip position setting unit 230 randomly samples the appearance points within a narrower range than the point distance within the gripper grip range of the picking robot 400. [ Then, connect each sampled point as a parameter to set the connection points that best match the width of the gripper grip range and the angle of the gripper grip range. Accordingly, the gripping position is set to the outer surface of the package corresponding to the connecting line of the connecting points, which most closely matches the angle of the gripper grip range and the width of the gripper grip range.

Then, the grip control unit 240 generates and outputs a grip control signal so as to be grasped by the picking robot 400 according to one grip position setting result, And further outputs a signal.

FIG. 7 is a graph showing the results of the parcel detection accuracy and the grip position extraction accuracy according to the present invention.

As shown in FIG. 7, simulation and experiment are performed to recognize the target parcels a predetermined number of times and automatically set the grip position of the recognized parcels, and the accuracy thereof is higher than that of the conventional 2D image version Can be confirmed.

In the central processing unit 200, a plurality of amorphous packages having a variety of sizes, shapes, and packaging types may be set in advance for a product previously handled at a distribution center, and then an image of predetermined amorphous packages may be stored. Thus, the central processing unit 200 can quickly detect the grip position of the package by real-time comparison between a plurality of images previously set and the stereoscopic image of the amorphous package actually detected.

As described above, the control system for a robot that can pick up atypical digits according to an embodiment of the present invention automatically recognizes an irregular package using a 3D vision and a laser detection module. By accurately analyzing the grip position of the package, the package can be grasped and moved accurately.

Particularly, the picking tool and the package recognition technology, which can be universally applied to the picking operation for the multi-item packages, can enhance the utilization of the picking robot and minimize labor and time consumption. When a picking robot system capable of picking unmanned is developed, it will be possible to automate more than 90% of logistics processing in the field of automation of picking.

In addition, in the present invention, the distance between the picking robot and the package is accurately sensed in three dimensions using a stereoscopic image camera, an image photographing camera, and a laser sensor. By automatically analyzing the position of the package to the gripping position, it is possible to reduce the picking time but improve the accuracy. Accordingly, the present invention can anticipate economical effects such as an increase in the throughput of the logistics, a cost of the logistics, and a decrease in the peaking error rate.

The various embodiments disclosed in the present specification and drawings are only specific examples for the purpose of understanding and are not intended to limit the scope of various embodiments of the present invention. Accordingly, the scope of various embodiments of the present invention should not be limited by the above-described embodiments, and all changes or modifications derived from the technical ideas of various embodiments of the present invention are included in the scope of various embodiments of the present invention Should be interpreted.

100:
110: stereoscopic image extracting unit
120: Image shooting unit
130: distance measuring unit
140: Signal conversion storage section
200: central processing unit
210:
220: a grip position extracting unit
230: grip position setting unit
240:

Claims (9)

A parcel detection unit for extracting stereoscopic image data, image data, and distance measurement data;
A central processing unit for analyzing the stereoscopic image data, image data, and distance measurement data to detect a center point position of the parcel and a grip position of the parcel, and output a grip control signal according to the detection result; And
And a robot driving control unit for driving the picking robot to grasp and move the grip position of the package according to the grip control signal,
The central processing unit
Extracting two points having a distance within a gripper grip range of the picking robot based on a position of a center point of the package, analyzing the three-dimensional image data, the image data, and the distance measurement data, A grip position extracting unit for searching a maximum depth based on the points and calculating a distance from the external shape of the package; And
A plurality of outward points in a range narrower than the two point distances in the gripper grip range are sampled based on the depth map data and the sampled points are connected to each other to define a gripping position A control system for a logistics robot capable of picking up atypical digits including a setting unit.
The method according to claim 1,
The package detector
A stereoscopic image extracting unit for picking up an image within a grip range of the picking robot and outputting a 3D stereoscopic image signal in at least one frame unit;
An image photographing unit photographing an image within a holding range of the picking robot and outputting an image signal in at least one frame unit;
A distance measurement unit for emitting a laser signal to a grasping range of the picking robot and outputting a distance signal; And
A signal conversion storage unit for digitally converting the stereoscopic image signal, the image signal, and the distance signal of the at least one frame unit and outputting the stereoscopic image data, the image data, and the distance measurement data at least every one frame;
A robot control system capable of picking up atypical digits including at least one robot;
3. The method of claim 2,
The stereoscopic image extracting unit
And at least one stereo camera including an RGV-D camera sensor, wherein the at least one stereo camera photographs an image within a holding range of the picking robot in at least one frame unit, And a signal conversion storage unit for outputting the signal to the signal conversion storage unit.
3. The method of claim 2,
The distance measuring unit
Wherein the at least one laser sensor real-time measures the distance from the body of the picking robot to the package to be delivered within the gripping range of the picking robot, And the at least one amorphous packet to be supplied to the signal conversion storage unit in units of frames.
The method according to claim 3 or 4,
The signal conversion storage unit
Wherein the signal conversion storage unit converts at least one frame of the stereoscopic image signal, the image signal, and the distance signal into digital data,
Wherein the stereoscopic image data, the image data, and the distance measurement data of 3D are supplied to the central processing unit by at least one frame unit through the RS-232 interface device.
The method according to claim 1,
The central processing unit
A parcel detection unit for analyzing the center point position of the parcel by inputting the stereoscopic image data, the image data, and the distance measurement data in units of at least one frame, And
A grasping control unit for generating and outputting a grasping control signal so as to be grasped by the picking robot according to the grasping position setting result and further outputting a movement control signal for moving the grasped gripper to a preset position;
Wherein the at least one at least one of the at least two at least two of the plurality of at least two of the at least one at least two of the at least one at least two of the at least one robot is selected.
The method according to claim 6,
The package detection unit
In order to accurately detect the target package, the SURF algorithm (Fast Up Robust Features algorithm) and the FLANN algorithm (Fast Library for Approximate Nearest Neighbors algorithm)
A logistic robot control system capable of atypical picking which maps stereoscopic image data to image data and depth map based on mapping function.
delete 8. The method of claim 7,
The grip position setting unit
Randomly sampling the appearance points and connecting the sampled points as parameters to set connection points that best match an angle of the gripper grip range and a width of the gripper grip range,
Wherein a gripping position is set at a gripping position corresponding to an angle of the gripper grip range and a connection line of connection points most matching with a width of the gripper grip range.

Images