CN110135439B - Method and device for automatically identifying goods shelf and mobile robot - Google Patents

Method and device for automatically identifying goods shelf and mobile robot Download PDF

Info

Publication number
CN110135439B
CN110135439B CN201910386141.4A CN201910386141A CN110135439B CN 110135439 B CN110135439 B CN 110135439B CN 201910386141 A CN201910386141 A CN 201910386141A CN 110135439 B CN110135439 B CN 110135439B
Authority
CN
China
Prior art keywords
shelf
point set
point
center
leg
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910386141.4A
Other languages
Chinese (zh)
Other versions
CN110135439A (en
Inventor
张海涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Noah robot technology (Shanghai) Co.,Ltd.
Original Assignee
Shanghai Mumu Jucong Robot Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Mumu Jucong Robot Technology Co ltd filed Critical Shanghai Mumu Jucong Robot Technology Co ltd
Priority to CN201910386141.4A priority Critical patent/CN110135439B/en
Publication of CN110135439A publication Critical patent/CN110135439A/en
Application granted granted Critical
Publication of CN110135439B publication Critical patent/CN110135439B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Abstract

The invention provides a method and a device for automatically identifying a goods shelf and a mobile robot, wherein the method comprises the following steps: acquiring point cloud data reflecting surrounding environment information; preprocessing the point cloud data to obtain a center point set of suspected shelf legs; according to a pre-constructed shelf model point set, carrying out coarse registration on the center point set, and extracting a single shelf point set after the coarse registration; performing fine registration on the single shelf point set according to shelf leg characteristics; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated. The method for automatically identifying the goods shelf provided by the invention can support the goods shelf to be placed at will, and a guide device is not required to be installed at the bottom of the goods shelf, so that the operation is simplified.

Description

Method and device for automatically identifying goods shelf and mobile robot
Technical Field
The invention relates to the field of robots, in particular to a method and a device for automatically identifying a goods shelf and a mobile robot.
Background
The robot automatically enters the bottom of the goods shelf and carries the goods shelf away, which is a means for solving the problem of manual transportation. The space of the bottom of the goods shelf is narrow, and the robot is required to be capable of executing high-precision movement to move to the bottom of the goods shelf. At present, the existing method enables a robot to identify a goods shelf:
the first method is to require the goods shelf to be placed at a specific position, and then a path is designed to enable the robot to navigate and move to the position to load the goods shelf. In this method, the robot itself does not know that it has moved to the bottom of the rack, simply moves to the destination, and then performs the rack loading action. If the shelf is placed inaccurately or the robot is positioned with a deviation, the robot can not enter the bottom of the shelf or even collide with the shelf.
And secondly, installing a special device at the bottom of the goods shelf, for example, the device is special in shape, and guiding the robot into the bottom of the goods shelf through the special device. The method comprises the steps that a robot is navigated to a certain position in front of a goods shelf, then a special device installed at the bottom of the goods shelf is searched through laser scanning on the robot, and the robot is guided to enter the center of the bottom of the goods shelf according to the position of the special device at the bottom of the goods shelf; the devices are manually installed in advance for all the shelves, which is time-consuming and labor-consuming; in addition, if the robot stops at an inappropriate position in front of the shelf or the shelf is not placed at an appropriate angle, the robot cannot scan the special device, and the shelf loading fails.
Disclosure of Invention
The invention aims to provide a method and a device for automatically identifying a goods shelf and a mobile robot, which can support the goods shelf to be placed at will, do not need to install a guide device at the bottom of the goods shelf and simplify the operation.
The technical scheme provided by the invention is as follows:
a method of automatically identifying a shelf, comprising: acquiring point cloud data reflecting surrounding environment information; preprocessing the point cloud data to obtain a center point set of suspected shelf legs; according to a pre-constructed shelf model point set, carrying out coarse registration on the center point set, and extracting a single shelf point set after the coarse registration; performing fine registration on the single shelf point set according to shelf leg characteristics; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated.
Further preferably, the preprocessing the point cloud data to obtain a center point set of the suspected shelf leg specifically includes: clustering the point cloud data to obtain at least one cluster; filtering non-shelf leg clusters according to the number of point clouds in the clusters; for the remaining clusters, the center point of each cluster is calculated, and all the center points are formed into a center point set.
Further preferably, the coarsely registering the center point set according to the pre-constructed shelf model point set, and extracting a single shelf point set after coarse registration includes: matching the center point set with a pre-constructed shelf model point set; when the center point set has a candidate point set matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set; obtaining a shelf extraction frame under the same coordinate system according to the shelf model point set, wherein the shelf extraction frame comprises the shelf model point set and is superposed with the central point of the shelf model point set; and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
Further preferably, the matching the center point set and the pre-constructed shelf model point set includes: and matching the central point set with a pre-constructed shelf model point set by adopting an iterative closest point algorithm.
Further preferably, said fine registration of said single shelf point sets is performed according to shelf leg features; when the single-shelf point set is matched with the shelf leg features, identifying a shelf, and calculating the center position of the bottom of the shelf specifically comprises: performing fine registration on the single shelf point set according to the geometrical characteristics of the shelf leg; when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf; and obtaining the central position of the bottom of the goods shelf according to the position of the key point.
The invention also provides a device for automatically identifying the goods shelf, which comprises: the data acquisition module is used for acquiring point cloud data reflecting surrounding environment information; the data preprocessing module is used for preprocessing the point cloud data to obtain a center point set of the suspected goods shelf leg; the rough registration module is used for carrying out rough registration on the center point set according to a pre-constructed shelf model point set, and extracting a single shelf point set after the rough registration; the fine registration module is used for performing fine registration on the single shelf point set according to the shelf leg characteristics; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated.
Further preferably, the data preprocessing module includes: the clustering unit is used for clustering the point cloud data to obtain at least one cluster; the filtering unit is used for filtering the non-shelf leg clusters according to the quantity of the point clouds in the clusters; and the center calculating unit is used for calculating the center point of each cluster for the rest clusters and forming a center point set by all the center points.
Further preferably, the coarse registration module includes: the matching unit is used for matching the center point set with a pre-constructed shelf model point set; when one candidate point set exists in the center point set and is matched with the shelf model point set, a target transformation matrix is obtained according to the candidate point set and the shelf model point set; the goods shelf extracting unit is used for obtaining goods shelf extracting frames under the same coordinate system according to the goods shelf model point sets, and the goods shelf extracting frames comprise the goods shelf model point sets and are superposed with the center points of the goods shelf model point sets; and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
Further preferably, the fine registration module is further configured to perform fine registration on the single shelf point set according to a shelf leg geometric feature; when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf; and obtaining the central position of the bottom of the goods shelf according to the position of the key point.
The invention also provides a mobile robot, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of the method for automatically identifying the shelf when running the computer program.
Compared with the prior art, the method and the device for automatically identifying the goods shelf and the mobile robot have the advantages that:
1. the method for automatically identifying the goods shelf provided by the invention can support the goods shelf to be placed at will, and a guide device is not required to be installed at the bottom of the goods shelf, so that the operation is simplified.
2. The invention provides a method for accurately identifying a goods shelf according to the geometrical characteristics of the leg of the goods shelf, and the geometrical characteristics of the leg with the least number of points are adopted, so that the goods shelf can be accurately identified even if a robot cannot acquire the information of all the leg points of a single goods shelf according to the geometrical characteristics.
Drawings
The above features, technical features, advantages and implementations of a method and apparatus for automatically recognizing a shelf, a mobile robot, and a storage medium will be described in further detail in a clearly understandable manner with reference to the accompanying drawings.
FIG. 1 is a flow diagram of one embodiment of a method of automatically identifying a shelf of the present invention;
FIG. 2 is a flow diagram of another embodiment of a method of automatically identifying shelves of the present invention;
FIG. 3 is a flow diagram of another embodiment of a method of automatically identifying shelves of the present invention;
FIG. 4 is a schematic diagram of an embodiment of an apparatus for automatically identifying a shelf according to the present invention;
FIG. 5 is a schematic structural diagram of another embodiment of an apparatus for automatically identifying a shelf according to the present invention;
FIG. 6 is a schematic structural diagram of one embodiment of a mobile robot of the present invention;
fig. 7 is a schematic view of a shelf bottom structure.
The reference numbers illustrate:
100. the system comprises a data acquisition module, 200, a data preprocessing module, 300, a coarse registration module, 400, a fine registration module, 210, a clustering unit, 220, a filtering unit, 230, a central computing unit, 310, a matching unit, 320, a shelf extraction unit, 500, a mobile robot, 510, a memory, 520, a processor, 530, and a computer program.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "one" means not only "only one" but also a case of "more than one".
In one embodiment of the present invention, as shown in fig. 1, a method of automatically identifying a shelf includes:
step S100 acquires point cloud data reflecting the surrounding environment information.
Specifically, the robot knows the surrounding environment through laser radar scanning in the walking process, acquires the place ahead object information, and the place ahead object probably is goods shelves, wall, passerby etc..
Point Cloud Data (PCD) refers to a Data file obtained by scanning with a laser radar, where the Data file is recorded in the form of points, and each Point may contain intensity information in addition to a geometric position, where the intensity information is obtained according to the intensity of an echo acquired by the laser radar.
According to the embodiment, the two-dimensional laser radar is adopted to collect point cloud data, whether a front object is a shelf leg part is identified according to the collected point cloud data, and the shelf is further identified according to the shelf leg part. In order to accurately acquire data information of the leg of the shelf, the laser radar is required to be horizontally installed on one side of the robot entering the shelf, for example, the robot moves forwards to enter the shelf and is installed in the front, the installation height is proper and is not higher than the length of the leg of the shelf, so that the leg of the shelf can be scanned during laser horizontal scanning.
The robot has a certain range of laser scanning each time, and some robots can scan at 360 degrees and some robots can scan at a range less than 180 degrees. In order to avoid that the goods shelf is not in the laser detection range of the robot, for example, certain legs of the goods shelf are shielded and cannot be scanned by the laser due to improper relative positions of the robot and the goods shelf, it is preferable that point cloud data reflecting the surrounding environment information is acquired by 360-degree scanning each time, and the 360-degree scanning is completed by in-situ rotation for the robot with limited scanning range each time.
Step S200, the point cloud data are preprocessed to obtain a center point set of suspected shelf legs.
Specifically, the data volume of the point cloud data is related to the scanning distance, and the data volume obtained by long-distance scanning is small and the data volume obtained by short-distance scanning is large for the same object. For example, a round of 360 ° scanning emits 1000 laser beams in total, each round obtains a set of data, a shelf leg with a width of 10cm is scanned by the robot at a distance of 3 meters, and the number of the scanning points corresponds to about 5 points of the shelf leg; if the robot scans at a distance of 50 cm, about 35 points correspond to the shelf leg. However, the 35 points do not bring more information than the 5 points, so that the acquired point cloud data needs to be preprocessed.
Firstly, clustering processing is carried out on point cloud data, and a plurality of dense points are gathered into a cluster. A number of clusters can be obtained using a variety of clustering methods, such as K-means clustering, mean shift clustering, density-based clustering, and the like.
Then, clusters of definite non-shelf legs are eliminated according to the number of the point clouds in the clusters, such as clusters with a small number of the point clouds in the clusters (such as only 1 and 2 points, and high probability is noise generated by laser errors), clusters with a large number of the point clouds in the clusters (such as 100 points, which are far thicker than the shelf legs), and clusters which are possible to be shelf legs (namely clusters of suspected shelf legs) are reserved.
For the rest clusters, one point at the center of each cluster is used for replacing the whole cluster, and the centers of all the clusters form a center point set of the suspected shelf legs, so that the data volume of the point cloud can be reduced, and the subsequent processing calculation volume can be effectively reduced.
Step S300, according to a pre-constructed shelf model point set, carrying out rough registration on the center point set, and extracting a single shelf point set after rough registration.
Specifically, the shelf model point set is created by modeling the top view of the shelf leg at the laser height, or simply understood, based on the bottom schematic view of the shelf. Two-dimensional plane coordinates can be established by taking the center of the shelf as the origin of coordinates, the coordinate position of the central point of each leg of the shelf is obtained according to the actual shelf size, and the coordinate positions of the central points of all the legs form the shelf model point set. Or the center of a certain shelf leg may be used as the origin of coordinates, and the coordinate position of the center point of another leg is established, so as to obtain the shelf model point set, which is not limited here.
And performing coarse registration when the central point set is not empty or meets a preset minimum data volume. The rough registration refers to matching a center point set of the suspected shelf leg with a shelf model point set, and identifying whether a sub-point set which can form an approximate shelf model exists in the center point set. The matching method can adopt a RANSAC (RANdom SAmple Consensus) algorithm, and a group of point sets with the matching degree reaching the preset requirement with the shelf model point set are found out from the central point set, so that the approximate position of a single shelf is obtained; and extracting the point set at the approximate position of the single shelf to obtain a single shelf point set.
The RANSAC algorithm can estimate parameters of a mathematical model from a group of observation data sets containing 'out-of-office points' in an iterative manner; outliers refer to data that cannot fit into the mathematical model, such as data noise. In this embodiment, the observation data set refers to a center point set of suspected shelf legs, and the mathematical model refers to a shelf model.
The coarse registration is only to find the approximate position of the shelf, and the accuracy requirement (such as 1cm error) cannot be met, so the fine registration is needed subsequently to reduce the error. And the fine registration is time-consuming, and in order to reduce the calculation amount of the fine registration, a single-shelf point set is extracted from the central point set for fine registration, so that the real-time performance of the system is improved.
Step S400, carrying out fine registration on the single shelf point set according to shelf leg characteristics;
step S500, when the single-shelf point set is matched with the shelf leg characteristics, identifying a shelf and calculating the center position of the bottom of the shelf.
Specifically, the shelf leg features include shelf leg geometric features, for example, as shown in fig. 7, 1-6 points indicate 6 legs, C point indicates a shelf center point, the distance between 1-2, 2-3, 4-5, 5-6 is t1, and the distance between 1-4, 3-6 is t 3; this goods shelves bottom respectively has 3 points about goods shelves central point, corresponds 3 legs respectively, and this goods shelves shank geometric characteristics is: there are six points that simultaneously satisfy the following characteristics:
1) the six points form a rectangle, one side of the rectangle is at a distance of 2 × t1, and the other side of the rectangle is at a distance of t 3;
2) one side has three collinear points, each separated by a distance t 1.
If there are 6 points that satisfy the above feature at the same time, a single shelf point set including these 6 points corresponds to one actual shelf.
And the fine registration is to perform fine matching on the single-shelf point set and the shelf leg characteristics and judge whether the single-shelf point set has point set data meeting the shelf leg characteristics. Following the above example, fine registration is to search the single-shelf point set, determine whether there are 6 points that satisfy the above geometric features, and if so, consider the single-shelf point set to correspond to an actual shelf.
The shelf leg features may include other features besides the geometrical features of the shelf legs, for example, the reflecting paper is attached to the 1 or 4 points in fig. 7, the reflecting strength of the leg attached with the reflecting paper is relatively high, the 1 or 4 points can be quickly identified according to the light intensity features, and the shelf is identified by combining the geometrical features of the leg.
When the single shelf point set is judged to be the actual shelf, the position of the central point of the bottom of the shelf can be calculated according to the positions of the points in the single shelf point set and the geometrical characteristics of the leg parts of the shelf. As shown in FIG. 7, the position of the center point C can be calculated from the positions of the points t1, t3, and 1-6.
Based on the center location of the bottom of the shelf, the path to the bottom of the shelf can be further determined. When the center position of the bottom of the shelf is reached, the robot is considered to reach the shelf loading position, and the robot can perform the shelf loading operation.
Of course, after a shelf is identified, the remaining center point set data is used to replace the center point set data, the step 300 is repeated, the shelf is continuously searched until all the original center point set data are processed, and then the shelf is transported according to the preset transportation rule, for example, 2 shelves are obtained from the original center point set data, and a shelf closest to the robot is transported according to the principle of proximity.
This embodiment passes through laser scanning discernment goods shelves leg support, through goods shelves model point set, the accurate goods shelves of discerning of goods shelves leg characteristic to the accurate goods shelves bottom central point that calculates puts. Therefore, the shelf can be randomly placed, a guide device is not required to be arranged at the bottom of the shelf, and the operation is simplified.
In another embodiment of the present invention, as shown in fig. 2, a method of automatically identifying a shelf includes:
step S100 acquires point cloud data reflecting the surrounding environment information.
Step S210, clustering the point cloud data to obtain at least one cluster;
step S220, filtering non-shelf leg clusters according to the quantity of point clouds in the clusters;
step S230 calculates the center point of each cluster for the remaining clusters, and forms all the center points into a center point set.
Specifically, the clustering process is a clustering process. The point cloud data can be clustered by adopting various clustering methods, such as a K-means clustering method, and can also be simply processed: and taking the point cloud data within the preset radius r as a cluster, and if the distance between the point at the periphery of the cluster and one point in the cluster is less than or equal to r, adding the point into the cluster, wherein the r is set according to experience (such as 2 cm).
After cluster S is obtained, performing preliminary filtering, removing clusters of definite non-shelf legs, and removing all clusters with the number of point clouds smaller than m and larger than n from cluster S to obtain cluster S1, where m and n are obtained empirically (for example, m is 2 and n is 50, which can be set in combination with the length and width of a shelf leg), where m is smaller than m represents that the cluster may be noise generated by laser error, and n is larger than n represents that the cluster is much thicker than the shelf leg to be detected and is not a shelf. And calculating the cluster center coordinates of each cluster for the rest clusters, and replacing the whole cluster by using one cluster center point, wherein all cluster centers form a center point set.
Step S310, matching the center point set with a pre-constructed shelf model point set;
step S330, judging whether the center point set has a candidate point set matched with the shelf model point set;
step S340, when a candidate point set exists in the center point set and is matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set.
Specifically, the center point set and the shelf model point set are data in different two-dimensional coordinate systems, and the coordinate system in which the center point set is located is based on a robot, for example, a coordinate system with a laser scanning point as an origin; the coordinate system in which the shelf model point sets are located is based on the shelf itself, for example, a coordinate system with the center of the bottom of the shelf as the origin. The matching between the center point set and the shelf model point set is to register data under two different coordinate systems, namely, the registration from the point set to the model.
And matching the center point set with the shelf model point set by adopting an RANSAC algorithm and an ICP algorithm. And finding a sub-point set which is roughly matched with the shelf model point set from the central point set through an RANSAC algorithm, obtaining an initial rotation and translation matrix of the ICP algorithm according to the sub-point set and the shelf model point set, and further matching by utilizing the ICP algorithm. The matching speed of the ICP algorithm can be improved by adopting an initial rotation and translation matrix relative to the spectrum and a randomly generated initial rotation and translation matrix.
As shown in the following formula, an objective of an ICP (iterative closest point) algorithm is to find a rotational-translation matrix between a source point cloud and a target point cloud, so that optimal matching between two points of data meets a certain measurement criterion.
pt＝R*ps+T
Wherein p ist、psThe target point cloud and the source point cloud form a corresponding point, and the rotation parameter R and the translation parameter T form a rotation and translation matrix.
The registration steps are as follows:
two-dimensional point sets are assumed: a center point set C (equivalent to source point cloud data) and a shelf model point set K (equivalent to target point cloud data).
Step one, transforming a point set C by using an initial rotation translation matrix to obtain a transformation point set;
finding corresponding near points of each point in the point set K in the transformation point set, wherein each point in the point set K and the corresponding near points form corresponding point pairs; obtaining rigid body transformation which enables the average distance of the corresponding points to be minimum, and obtaining an optimized rotation translation matrix;
step three, transforming the point set C by adopting the optimized rotation translation matrix to obtain a new transformation point set;
and step four, if the minimum average distance between the new transformation point set and the point set K is smaller than a given threshold value, stopping iterative computation, otherwise, replacing the transformation point set with the new transformation point set, and repeating the step two, the step three and the step four to continue iteration.
And when the iteration is stopped, the finally adopted rotation and translation matrix is the target transformation matrix.
And S350, obtaining a shelf extraction frame under the same coordinate system according to the shelf model point set, wherein the shelf extraction frame comprises the shelf model point set and is superposed with the central point of the shelf model point set.
Specifically, the shelf model point set only contains the coordinate positions of the shelf leg points, and the frame of the actual shelf is often wider than the shelf leg portions, so that a shelf extraction frame not smaller than the contour of the shelf is established on the basis of the shelf model point set, and effective data forming the shelf model is prevented from being eliminated.
Step S360, obtaining a corresponding frame of the shelf extracting frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
Specifically, according to the target transformation matrix, the shelf extraction frame is converted from the coordinate system of the shelf model point set to the coordinate system of the center point set to obtain a corresponding frame, and the corresponding frame is used for extracting a single shelf point set from the center point set. The single shelf point set not only comprises data of shelf leg points, but also comprises noise data under the shelf frame, so that the interference of the data analysis of the remaining center point set can be reduced by arranging the shelf extraction frame.
Step S400, carrying out fine registration on the single shelf point set according to shelf leg characteristics;
step S500, when the single-shelf point set is matched with the shelf leg characteristics, identifying a shelf and calculating the center position of the bottom of the shelf.
The embodiment provides a specific method for extracting a single shelf matched with a shelf model point set from the center point set, accurately identifying the single shelf according to the leg characteristics of the shelf, and accurately calculating the center position of the bottom of the shelf. Therefore, the shelf can be randomly placed, and a guide device is not required to be arranged at the bottom of the shelf.
In another embodiment of the present invention, as shown in fig. 3, a method of automatically identifying a shelf includes:
step S100, point cloud data reflecting surrounding environment information is obtained;
step S210, clustering the point cloud data to obtain at least one cluster;
step S220, filtering non-shelf leg clusters according to the quantity of point clouds in the clusters;
step S230 calculates the center point of each cluster for the remaining clusters, and forms all the center points into a center point set.
And step S320, matching the central point set with a pre-constructed shelf model point set by adopting an iterative closest point algorithm.
Specifically, in view of the fact that the present invention is a 2-dimensional point cloud and the calculated amount of a 3-dimensional point cloud is not large, the present embodiment directly adopts the ICP algorithm to match the center point set with the shelf model point set.
Step S330, judging whether the center point set has a candidate point set matched with the shelf model point set;
step S340, when a candidate point set exists in the center point set and is matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set;
step S350, obtaining a shelf extraction frame under the same coordinate system according to the shelf model point set, wherein the shelf extraction frame comprises the shelf model point set and is superposed with the center point of the shelf model point set;
step S360, obtaining a corresponding frame of the shelf extracting frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
Step S410, carrying out fine registration on the single shelf point set according to the geometrical characteristics of the shelf leg;
step S510, when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf;
and step S520, obtaining the center position of the bottom of the goods shelf according to the position of the key point.
In particular, shelf leg geometry may be pre-extracted for a particular shelf. As shown in fig. 7, the bottom of the shelf has 3 points on the left and right of the center point of the shelf, which correspond to 3 legs respectively; the geometrical characteristics of the shelf leg with the least points are as follows: there are four points that simultaneously satisfy the following characteristics: 1) if three points are collinear and the distance between each point satisfies the threshold t1, the feature is said to be collinear. 2) A specific included angle formed between the line sections connected with one point and the other two points is 90 degrees, and the length of the two line sections respectively meets the threshold values t1 and t3, so that the characteristic is called an included angle relationship.
If four points satisfying the above characteristics exist in the single shelf point set, that is, the collinear relationship and the included angle relationship are simultaneously satisfied, the single shelf point set is considered to be matched with the leg characteristics of the shelf, the single shelf point set corresponds to an actual shelf, and the four points satisfying the above characteristics are also called key points.
Following the above example, the method of finding keypoints is as follows: the distance between all the points in the single shelf point set is calculated, for example, the obtained distance values are t1, t2, t3.. From the above shelf leg geometry (which concerns distances t1, t3), it is extracted that it is fullAll the point pairs of the feet t1 and t3 are respectively marked as candidate point pair 1 and candidate point pair 2. Randomly selecting a point pair [ p ] from the candidate point pairs 1i,pj](indicates piPoint and pjThe distance of the points is equivalent to t1), it is detected whether there is a point pair [ p ] satisfying the collinear relationship among the candidate point pairs 1j,ps]Or point pair [ pi,ps]If yes, then searching whether a point pair [ p ] meeting the included angle relation exists in the candidate point pair 2i,pw]If yes, finding out the key point pi、pj、ps、pw. From the positions of these four key points, the position of the center point at the bottom of the shelf can be calculated.
As shown in fig. 7, there are multiple sets of key points at the bottom of the shelf, such as point (1,2,3,4), point (1,2,3,6), point (1,4,5,6), point (2,4,5,6), etc. Any set of keypoints is identified, and it can be confirmed that the single shelf point set matches the shelf leg features. The position of the center point of the bottom of the shelf can be calculated according to any group of key points.
Depending on the size of the robot and the relationship between t1 and t3, a suitable direction may be further selected to enter the bottom of the rack, for example, when the size of the robot is larger than t1, or the size of the robot is smaller than t1 but closer to t1, and t3 is larger than t1, then it is preferable to enter from the side of t3, and the risk of collision with the rack from the side of t1 is higher.
The embodiment provides a method for accurately identifying a shelf according to the geometrical characteristics of the leg of the shelf, and the geometrical characteristics of the leg with the least number of points are adopted, so that the shelf can be accurately identified even if a robot cannot acquire the information of all the leg points of a single shelf according to the geometrical characteristics.
In one embodiment of the present invention, as shown in fig. 4, an apparatus for automatically identifying a shelf includes:
the data acquisition module 100 is configured to acquire point cloud data reflecting ambient environment information.
Specifically, the robot knows the surrounding environment through laser radar scanning in the walking process, acquires the place ahead object information, and the place ahead object probably is goods shelves, wall, passerby etc..
Point Cloud Data (PCD) refers to a Data file obtained by scanning with a laser radar, where the Data file is recorded in the form of points, and each Point may contain intensity information in addition to a geometric position, where the intensity information is obtained according to the intensity of an echo acquired by the laser radar.
According to the embodiment, the two-dimensional laser radar is adopted to collect point cloud data, whether a front object is a shelf leg part is identified according to the collected point cloud data, and the shelf is further identified according to the shelf leg part. In order to accurately acquire data information of the leg of the shelf, the laser radar is required to be horizontally installed on one side of the robot entering the shelf, for example, the robot moves forwards to enter the shelf and is installed in the front, the installation height is proper and is not higher than the length of the leg of the shelf, so that the leg of the shelf can be scanned during laser horizontal scanning.
The robot has a certain range of laser scanning each time, and some robots can scan 360 degrees or scan less than 180 degrees. In order to avoid that the goods shelf is not in the laser detection range of the robot, for example, certain legs of the goods shelf are shielded and cannot be scanned by the laser due to improper relative positions of the robot and the goods shelf, it is preferable that point cloud data reflecting the surrounding environment information is acquired by 360-degree scanning each time, and the 360-degree scanning is completed by in-situ rotation for the robot with limited scanning range each time.
And the data preprocessing module 200 is used for preprocessing the point cloud data to obtain a center point set of the suspected shelf leg.
Specifically, the data volume of the point cloud data is related to the scanning distance, and the data volume obtained by long-distance scanning is small and the data volume obtained by short-distance scanning is large for the same object. For example, a round of 360 ° scanning emits 1000 laser beams in total, each round obtains a set of data, a shelf leg with a width of 10cm is scanned by the robot at a distance of 3 meters, and the number of the scanning points corresponds to about 5 points of the shelf leg; if the robot scans at a distance of 50 cm, about 35 points correspond to the shelf leg. However, the 35 points do not bring more information than the 5 points, so that the acquired point cloud data needs to be preprocessed.
Firstly, clustering processing is carried out on point cloud data, and a plurality of dense points are gathered into a cluster. A number of clusters can be obtained using a variety of clustering methods, such as K-means clustering, mean shift clustering, density-based clustering, and the like.
Then, clusters of definite non-shelf legs are eliminated according to the number of the point clouds in the clusters, such as clusters with a small number of the point clouds in the clusters (such as only 1 and 2 points, and high probability is noise generated by laser errors), clusters with a large number of the point clouds in the clusters (such as 100 points, which are far thicker than the shelf legs), and clusters which are possible to be shelf legs (namely clusters of suspected shelf legs) are reserved.
For the rest clusters, one point at the center of each cluster is used for replacing the whole cluster, and the centers of all the clusters form a center point set of the suspected shelf legs, so that the data volume of the point cloud can be reduced, and the subsequent processing calculation volume can be effectively reduced.
And the rough registration module 300 is configured to perform rough registration on the center point set according to a pre-constructed shelf model point set, and extract a single shelf point set after the rough registration.
Specifically, the shelf model point set is created by modeling the top view of the shelf leg at the laser height, or simply understood, based on the bottom schematic view of the shelf. Two-dimensional plane coordinates can be established by taking the center of the shelf as the origin of coordinates, the coordinate position of the central point of each leg of the shelf is obtained according to the actual shelf size, and the coordinate positions of the central points of all the legs form the shelf model point set. Or the center of a certain shelf leg may be used as the origin of coordinates, and the coordinate position of the center point of another leg is established, so as to obtain the shelf model point set, which is not limited here.
And performing coarse registration when the central point set is not empty or meets a preset minimum data volume. The rough registration refers to matching a center point set of the suspected shelf leg with a shelf model point set, and identifying whether a sub-point set which can form an approximate shelf model exists in the center point set. The matching method can adopt a RANSAC (RANdom SAmple Consensus) algorithm, and a group of point sets with the matching degree reaching the preset requirement with the shelf model point set are found out from the central point set, so that the approximate position of a single shelf is obtained; and extracting the point set at the approximate position of the single shelf to obtain a single shelf point set.
The RANSAC algorithm can estimate parameters of a mathematical model from a group of observation data sets containing 'out-of-office points' in an iterative manner; outliers refer to data that cannot fit into the mathematical model, such as data noise. In this embodiment, the observation data set refers to a center point set of suspected shelf legs, and the mathematical model refers to a shelf model.
The coarse registration is only to find the approximate position of the shelf, and the accuracy requirement (such as 1cm error) cannot be met, so the fine registration is needed subsequently to reduce the error. And the fine registration is time-consuming, and in order to reduce the calculation amount of the fine registration, a single-shelf point set is extracted from the central point set for fine registration, so that the real-time performance of the system is improved.
A fine registration module 400, configured to perform fine registration on the single shelf point set according to shelf leg features; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated.
Specifically, the shelf leg features include shelf leg geometric features, for example, as shown in fig. 7, 1-6 points indicate 6 legs, C point indicates a shelf center point, the distance between 1-2, 2-3, 4-5, 5-6 is t1, and the distance between 1-4, 3-6 is t 3; this goods shelves bottom respectively has 3 points about goods shelves central point, corresponds 3 legs respectively, and this goods shelves shank geometric characteristics is: there are six points that simultaneously satisfy the following characteristics:
1) the six points form a rectangle, one side of the rectangle is at a distance of 2 × t1, and the other side of the rectangle is at a distance of t 3;
2) one side has three collinear points, each separated by a distance t 1.
If there are 6 points that satisfy the above feature at the same time, a single shelf point set including these 6 points corresponds to one actual shelf.
And the fine registration is to perform fine matching on the single-shelf point set and the shelf leg characteristics and judge whether the single-shelf point set has point set data meeting the shelf leg characteristics. Following the above example, fine registration is to search the single-shelf point set, determine whether there are 6 points that satisfy the above geometric features, and if so, consider the single-shelf point set to correspond to an actual shelf.
The shelf leg features may include other features besides the geometrical features of the shelf legs, for example, the reflecting paper is attached to the 1 or 4 points in fig. 7, the reflecting strength of the leg attached with the reflecting paper is relatively high, the 1 or 4 points can be quickly identified according to the light intensity features, and the shelf is identified by combining the geometrical features of the leg.
When the single shelf point set is judged to be the actual shelf, the position of the central point of the bottom of the shelf can be calculated according to the positions of the points in the single shelf point set and the geometrical characteristics of the leg parts of the shelf. As shown in FIG. 7, the position of the center point C can be calculated from the positions of the points t1, t3, and 1-6.
Based on the center location of the bottom of the shelf, the path to the bottom of the shelf can be further determined. When the center position of the bottom of the shelf is reached, the robot is considered to reach the shelf loading position, and the robot can perform the shelf loading operation.
Of course, after a shelf is identified, the remaining center point set data is used to replace the center point set data, the step 300 is repeated, the shelf is continuously searched until all the original center point set data are processed, and then the shelf is transported according to the preset transportation rule, for example, 2 shelves are obtained from the original center point set data, and a shelf closest to the robot is transported according to the principle of proximity.
This embodiment passes through laser scanning discernment goods shelves leg support, through goods shelves model point set, the accurate goods shelves of discerning of goods shelves leg characteristic to the accurate goods shelves bottom central point that calculates puts. Therefore, the shelf can be randomly placed, a guide device is not required to be arranged at the bottom of the shelf, and the operation is simplified.
In another embodiment of the present invention, as shown in fig. 5, an apparatus for automatically identifying a shelf includes:
the data acquisition module 100 is configured to acquire point cloud data reflecting ambient environment information.
And the data preprocessing module 200 is used for preprocessing the point cloud data to obtain a center point set of the suspected shelf leg. The data preprocessing module 200 includes:
and a clustering unit 210, configured to perform clustering on the point cloud data to obtain at least one cluster.
A filtering unit 220 for filtering non-shelf leg clusters according to the number of point clouds in the clusters of the clusters.
And a center calculating unit 230, configured to calculate a center point of each cluster for the remaining clusters, and form all the center points into a center point set.
Specifically, the clustering process is a clustering process. The point cloud data can be clustered by adopting various clustering methods, such as a K-means clustering method, and can also be simply processed: and taking the point cloud data within the preset radius r as a cluster, and if the distance between the point at the periphery of the cluster and one point in the cluster is less than or equal to r, adding the point into the cluster, wherein the r is set according to experience (such as 2 cm).
After cluster S is obtained, performing preliminary filtering, removing clusters of definite non-shelf legs, and removing all clusters with the number of point clouds smaller than m and larger than n from cluster S to obtain cluster S1, where m and n are obtained empirically (for example, m is 2 and n is 50, which can be set in combination with the length and width of a shelf leg), where m is smaller than m represents that the cluster may be noise generated by laser error, and n is larger than n represents that the cluster is much thicker than the shelf leg to be detected and is not a shelf. And calculating the cluster center coordinates of each cluster for the rest clusters, and replacing the whole cluster by using one cluster center point, wherein all cluster centers form a center point set.
And the rough registration module 300 is configured to perform rough registration on the center point set according to a pre-constructed shelf model point set, and extract a single shelf point set after the rough registration. The coarse registration module 300 includes:
a matching unit 310, configured to match the center point set with a pre-constructed shelf model point set; judging whether the center point set has a candidate point set matched with the shelf model point set or not; and when the center point set has a candidate point set matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set.
Specifically, the center point set and the shelf model point set are data in different two-dimensional coordinate systems, and the coordinate system in which the center point set is located is based on a robot, for example, a coordinate system with a laser scanning point as an origin; the coordinate system in which the shelf model point sets are located is based on the shelf itself, for example, a coordinate system with the center of the bottom of the shelf as the origin. The matching between the center point set and the shelf model point set is to register data under two different coordinate systems, namely, the registration from the point set to the model.
And matching the center point set with the shelf model point set by adopting an RANSAC algorithm and an ICP algorithm. And finding a sub-point set which is roughly matched with the shelf model point set from the central point set through an RANSAC algorithm, obtaining an initial rotation and translation matrix of the ICP algorithm according to the sub-point set and the shelf model point set, and further matching by utilizing the ICP algorithm. The matching speed of the ICP algorithm can be improved by adopting an initial rotation and translation matrix relative to the spectrum and a randomly generated initial rotation and translation matrix.
As shown in the following formula, an objective of an ICP (iterative closest point) algorithm is to find a rotational-translation matrix between a source point cloud and a target point cloud, so that optimal matching between two points of data meets a certain measurement criterion.
pt＝R*ps+T
Wherein p ist、psThe target point cloud and the source point cloud form a corresponding point, and the rotation parameter R and the translation parameter T form a rotation and translation matrix.
The registration steps are as follows:
two-dimensional point sets are assumed: a center point set C (equivalent to source point cloud data) and a shelf model point set K (equivalent to target point cloud data).
Step one, transforming a point set C by using an initial rotation translation matrix to obtain a transformation point set;
finding corresponding near points of each point in the point set K in the transformation point set, wherein each point in the point set K and the corresponding near points form corresponding point pairs; obtaining rigid body transformation which enables the average distance of the corresponding points to be minimum, and obtaining an optimized rotation translation matrix;
step three, transforming the point set C by adopting the optimized rotation translation matrix to obtain a new transformation point set;
and step four, if the minimum average distance between the new transformation point set and the point set K is smaller than a given threshold value, stopping iterative computation, otherwise, replacing the transformation point set with the new transformation point set, and repeating the step two, the step three and the step four to continue iteration.
And when the iteration is stopped, the finally adopted rotation and translation matrix is the target transformation matrix.
The shelf extracting unit 320 is configured to obtain a shelf extracting frame in the same coordinate system according to the shelf model point set, where the shelf extracting frame includes the shelf model point set and coincides with a center point of the shelf model point set; and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
Specifically, the shelf model point set only contains the coordinate positions of the shelf leg points, and the frame of the actual shelf is often wider than the shelf leg portions, so that a shelf extraction frame not smaller than the contour of the shelf is established on the basis of the shelf model point set, and effective data forming the shelf model is prevented from being eliminated.
And converting the shelf extracting frame from the coordinate system of the shelf model point set to the coordinate system of the center point set according to the target transformation matrix to obtain a corresponding frame, and extracting a single shelf point set from the center point set by using the corresponding frame. The single shelf point set not only comprises data of shelf leg points, but also comprises noise data under the shelf frame, so that the interference of the data analysis of the remaining center point set can be reduced by arranging the shelf extraction frame.
A fine registration module 400, configured to perform fine registration on the single shelf point set according to shelf leg features; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated.
The embodiment provides a specific method for extracting a single shelf matched with a shelf model point set from the center point set, accurately identifying the single shelf according to the leg characteristics of the shelf, and accurately calculating the center position of the bottom of the shelf. Therefore, the shelf can be randomly placed, and a guide device is not required to be arranged at the bottom of the shelf.
In another embodiment of the present invention, as shown in fig. 5, an apparatus for automatically identifying a shelf includes:
the data acquisition module 100 is configured to acquire point cloud data reflecting ambient environment information.
And the data preprocessing module 200 is used for preprocessing the point cloud data to obtain a center point set of the suspected shelf leg. The data preprocessing module 200 includes:
and a clustering unit 210, configured to perform clustering on the point cloud data to obtain at least one cluster.
A filtering unit 220 for filtering non-shelf leg clusters according to the number of point clouds in the clusters of the clusters.
And a center calculating unit 230, configured to calculate a center point of each cluster for the remaining clusters, and form all the center points into a center point set.
And the rough registration module 300 is configured to perform rough registration on the center point set according to a pre-constructed shelf model point set, and extract a single shelf point set after the rough registration. The coarse registration module 300 includes:
the matching unit 310 is used for matching the central point set with a pre-constructed shelf model point set by adopting an iterative closest point algorithm; judging whether the center point set has a candidate point set matched with the shelf model point set or not; and when the center point set has a candidate point set matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set.
Specifically, in view of the fact that the present invention is a 2-dimensional point cloud and the calculated amount of a 3-dimensional point cloud is not large, the present embodiment directly adopts the ICP algorithm to match the center point set with the shelf model point set.
The shelf extracting unit 320 is configured to obtain a shelf extracting frame in the same coordinate system according to the shelf model point set, where the shelf extracting frame includes the shelf model point set and coincides with a center point of the shelf model point set; and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
A fine registration module 400, configured to perform fine registration on the single shelf point set according to the shelf leg geometric features; when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf; and obtaining the central position of the bottom of the goods shelf according to the position of the key point.
In particular, shelf leg geometry may be pre-extracted for a particular shelf. As shown in fig. 7, the bottom of the shelf has 3 points on the left and right of the center point of the shelf, which correspond to 3 legs respectively; the geometrical characteristics of the shelf leg with the least points are as follows: there are four points that simultaneously satisfy the following characteristics: 1) if three points are collinear and the distance between each point satisfies the threshold t1, the feature is said to be collinear. 2) A specific included angle formed between the line sections connected with one point and the other two points is 90 degrees, and the length of the two line sections respectively meets the threshold values t1 and t3, so that the characteristic is called an included angle relationship.
If four points satisfying the above characteristics exist in the single shelf point set, that is, the collinear relationship and the included angle relationship are simultaneously satisfied, the single shelf point set is considered to be matched with the leg characteristics of the shelf, the single shelf point set corresponds to an actual shelf, and the four points satisfying the above characteristics are also called key points.
Following the above example, the method of finding keypoints is as follows: the distance between all the points in the single shelf point set is calculated, for example, the obtained distance values are t1, t2, t3.. From the shelf leg geometric features (the feature attention distances t1, t3), all point pairs satisfying t1 and t3 are extracted and are respectively designated as candidate point pair 1 and candidate point pair 2. Randomly selecting a point pair [ p ] from the candidate point pairs 1i,pj](indicates piPoint and pjThe distance of the points is equivalent to t1), whether the candidate point pair 1 is full is detectedPoint pair [ p ] of foot collinear relationj,ps]Or point pair [ pi,ps]If yes, then searching whether a point pair [ p ] meeting the included angle relation exists in the candidate point pair 2i,pw]If yes, finding out the key point pi、pj、ps、pw. From the positions of these four key points, the position of the center point at the bottom of the shelf can be calculated.
As shown in fig. 7, there are multiple sets of key points at the bottom of the shelf, such as point (1,2,3,4), point (1,2,3,6), point (1,4,5,6), point (2,4,5,6), etc. Any set of keypoints is identified, and it can be confirmed that the single shelf point set matches the shelf leg features. The position of the center point of the bottom of the shelf can be calculated according to any group of key points.
Depending on the size of the robot and the relationship between t1 and t3, a suitable direction may be further selected to enter the bottom of the rack, for example, when the size of the robot is larger than t1, or the size of the robot is smaller than t1 but closer to t1, and t3 is larger than t1, then it is preferable to enter from the side of t3, and the risk of collision with the rack from the side of t1 is higher.
The embodiment provides a method for accurately identifying a shelf according to the geometrical characteristics of the leg of the shelf, and the geometrical characteristics of the leg with the least number of points are adopted, so that the shelf can be accurately identified even if a robot cannot acquire the information of all the leg points of a single shelf according to the geometrical characteristics.
In another embodiment of the present invention, as shown in fig. 6, a mobile robot 500 includes:
a memory 510, a processor 520, and a computer program 530 stored in the memory and executable on the processor. The processor 520, when running the computer program 530, implements the steps of the method for automatically identifying a shelf according to any of the previous embodiments.
As an example, the processor realizes steps S100 to S500 according to the foregoing description when executing the computer program.
The processor also realizes the functions of the modules and units in the device for automatically identifying a shelf described above when the processor executes a computer program. As yet another example, the processor, when executing the computer program, implements the functionality of the data acquisition module 100, the data pre-processing module 200, the coarse registration module 300, and the fine registration module 400.
Alternatively, the computer program may be divided into one or more modules/units according to the particular needs to accomplish the invention. Each module/unit may be a series of computer program instruction segments capable of performing a particular function. The computer program instruction segment is used for describing the execution process of the computer program in the device for automatically identifying the shelf. As an example, the computer program may be divided into modules/units in a virtual device, such as a data acquisition module, a data pre-processing module, a coarse registration module and a fine registration module. Correspondingly, the data acquisition module is used for acquiring point cloud data reflecting the surrounding environment information; the data preprocessing module is used for preprocessing the point cloud data to obtain a center point set of the suspected goods shelf leg; the rough registration module is used for carrying out rough registration on the center point set according to a pre-constructed shelf model point set, and extracting a single shelf point set after the rough registration; the fine registration module is used for performing fine registration on the single shelf point set according to the shelf leg characteristics; when the single shelf point set matches the shelf leg feature, a shelf is identified and a shelf bottom center position is calculated.
The processor is used for realizing automatic shelf identification by executing the computer program. The processor may be a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), general purpose processor or other logic device, etc., as desired.
The memory may be any internal storage unit and/or external storage device capable of implementing data, program storage. For example, the memory may be a plug-in hard disk, a smart card (SMC), a Secure Digital (SD) card, or a flash card. The memory is used for storing computer programs, other programs and data of the device for automatically identifying the shelf. The memory may also be used to temporarily store data that has been output or is to be output.
It will be understood by those skilled in the art that the above-mentioned units and modules for implementing the corresponding functions are divided for the purpose of convenient illustration and description, and the above-mentioned units and modules are further divided or combined according to the application requirements, that is, the internal structures of the devices/apparatuses are divided and combined again to implement the above-mentioned functions. Each unit and module in the above embodiments may be separate physical units, or two or more units and modules may be integrated into one physical unit. The units and modules in the above embodiments may implement corresponding functions by using hardware and/or software functional units. Direct coupling, indirect coupling or communication connection among a plurality of units, components and modules in the above embodiments can be realized through a bus or an interface; the coupling, connection, etc. between the multiple units or devices may be electrical, mechanical, or the like. Accordingly, the specific names of the units and modules in the above embodiments are only for convenience of description and distinction, and are not intended to limit the scope of the present application.
It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (8)

1. A method of automatically identifying a shelf, comprising:
acquiring point cloud data reflecting surrounding environment information;
preprocessing the point cloud data to obtain a center point set of suspected shelf legs;
according to a pre-constructed shelf model point set, carrying out coarse registration on the center point set, and extracting a single shelf point set after the coarse registration;
performing fine registration on the single shelf point set according to shelf leg characteristics;
when the single-shelf point set is matched with the shelf leg characteristics, identifying a shelf and calculating the center position of the bottom of the shelf;
the preprocessing the point cloud data to obtain the center point set of the suspected shelf legs specifically comprises the following steps:
clustering the point cloud data to obtain at least one cluster;
filtering non-shelf leg clusters according to the number of point clouds in the clusters;
and calculating the center point of each cluster for the rest clusters, wherein each center point represents a suspected shelf leg, and all the center points form a center point set of the suspected shelf legs.
2. The method of claim 1, wherein the coarse registration of the set of center points is performed according to a pre-constructed set of shelf model points, and the extracting a single set of shelf points after the coarse registration comprises:
matching the center point set with a pre-constructed shelf model point set;
when the center point set has a candidate point set matched with the shelf model point set, obtaining a target transformation matrix according to the candidate point set and the shelf model point set;
obtaining a shelf extraction frame under the same coordinate system according to the shelf model point set, wherein the shelf extraction frame comprises the shelf model point set and is superposed with the central point of the shelf model point set;
and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
3. The method of automatically identifying shelves of claim 2, wherein said matching said set of center points to a set of pre-constructed shelf model points comprises:
and matching the central point set with a pre-constructed shelf model point set by adopting an iterative closest point algorithm.
4. The method of automatically identifying shelves of claim 3, wherein:
the fine registration of the single shelf point set according to shelf leg features comprises: performing fine registration on the single shelf point set according to the geometrical characteristics of the shelf leg;
identifying a shelf when the single shelf point set matches the shelf leg feature and calculating a shelf bottom center position comprises:
when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf;
and obtaining the central position of the bottom of the goods shelf according to the position of the key point.
5. An apparatus for automatically identifying a shelf, comprising:
the data acquisition module is used for acquiring point cloud data reflecting surrounding environment information;
the data preprocessing module is used for preprocessing the point cloud data to obtain a center point set of the suspected goods shelf leg;
the rough registration module is used for carrying out rough registration on the center point set according to a pre-constructed shelf model point set, and extracting a single shelf point set after the rough registration;
the fine registration module is used for performing fine registration on the single shelf point set according to the shelf leg characteristics; when the single-shelf point set is matched with the shelf leg characteristics, identifying a shelf and calculating the center position of the bottom of the shelf;
the data preprocessing module comprises:
the clustering unit is used for clustering the point cloud data to obtain at least one cluster;
the filtering unit is used for filtering the non-shelf leg clusters according to the quantity of the point clouds in the clusters;
and the center calculating unit is used for calculating the center point of each cluster for the rest clusters, each center point replaces one suspected shelf leg, and all the center points form a center point set of the suspected shelf legs.
6. The apparatus for automatically identifying shelves of claim 5, wherein said coarse registration module comprises:
the matching unit is used for matching the center point set with a pre-constructed shelf model point set; when one candidate point set exists in the center point set and is matched with the shelf model point set, a target transformation matrix is obtained according to the candidate point set and the shelf model point set;
the goods shelf extracting unit is used for obtaining goods shelf extracting frames under the same coordinate system according to the goods shelf model point sets, and the goods shelf extracting frames comprise the goods shelf model point sets and are superposed with the center points of the goods shelf model point sets; and obtaining a corresponding frame of the shelf extraction frame under the coordinate system of the central point set according to the target transformation matrix, and extracting a single shelf point set from the central point set by using the corresponding frame.
7. The apparatus for automatically identifying a shelf according to claim 6, wherein:
the fine registration module is further used for performing fine registration on the single shelf point set according to the geometrical characteristics of the shelf leg; when the single-shelf point set is matched with the geometrical characteristics of the shelf legs, identifying a shelf to obtain the position of a key point at the bottom of the shelf; and obtaining the central position of the bottom of the goods shelf according to the position of the key point.
8. A mobile robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor realizes the steps of the method of automatically identifying shelves according to any of claims 1-4 when running the computer program.
CN201910386141.4A 2019-05-09 2019-05-09 Method and device for automatically identifying goods shelf and mobile robot Active CN110135439B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910386141.4A CN110135439B (en) 2019-05-09 2019-05-09 Method and device for automatically identifying goods shelf and mobile robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910386141.4A CN110135439B (en) 2019-05-09 2019-05-09 Method and device for automatically identifying goods shelf and mobile robot

Publications (2)

Publication Number Publication Date
CN110135439A CN110135439A (en) 2019-08-16
CN110135439B true CN110135439B (en) 2021-05-04

Family

ID=67577058

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910386141.4A Active CN110135439B (en) 2019-05-09 2019-05-09 Method and device for automatically identifying goods shelf and mobile robot

Country Status (1)

Country Link
CN (1) CN110135439B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112630786A (en) * 2020-12-07 2021-04-09 兰剑智能科技股份有限公司 AGV buffer area inventory method, device and equipment based on 2D laser
CN112731445A (en) * 2021-01-04 2021-04-30 上海木蚁机器人科技有限公司 Goods shelf identification and positioning method, system and device and readable storage medium
CN112960319B (en) * 2021-01-29 2024-02-02 苏州寻迹智行机器人技术有限公司 Method for assisting submerged jacking AGV to drill into goods shelf by utilizing light reflecting unit
CN113213054B (en) * 2021-05-12 2023-05-30 深圳市海柔创新科技有限公司 Method, device, equipment, robot and warehousing system for adjusting pick-and-place device
CN113253737B (en) * 2021-06-21 2021-10-15 浙江华睿科技股份有限公司 Shelf detection method and device, electronic equipment and storage medium
CN113268066B (en) * 2021-07-19 2021-11-12 福勤智能科技（昆山）有限公司 Method and device for detecting target object, computer equipment and storage medium
CN113311844B (en) * 2021-07-28 2021-11-12 福勤智能科技（昆山）有限公司 Servo control method and device, computer equipment and storage medium
CN114234853A (en) * 2021-11-19 2022-03-25 广东嘉腾机器人自动化有限公司 Shelf inclination angle measuring method, system, equipment and medium based on laser scanning

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1753830A (en) * 2002-12-31 2006-03-29 Abb股份有限公司 A system and a method for container identification
WO2016052379A1 (en) * 2014-09-30 2016-04-07 日本電気株式会社 Information processing device, shelf label management system, control method and program
CN105479461A (en) * 2016-01-26 2016-04-13 先驱智能机械（深圳）有限公司 Control method, control device and manipulator system
CN106379684A (en) * 2016-11-11 2017-02-08 杭州南江机器人股份有限公司 Submersible AGV abut-joint method and system and submersible AGV
JP2017157216A (en) * 2016-02-29 2017-09-07 サインポスト株式会社 Information processing system
CN107843220A (en) * 2017-10-30 2018-03-27 江苏博睿通智能装备有限公司 Tiered warehouse facility shelf detecting system and detection method
CN107922119A (en) * 2015-11-25 2018-04-17 株式会社日立制作所 Shelf configuration system, transfer robot and shelf collocation method
CN109573439A (en) * 2018-12-27 2019-04-05 北京艾瑞思机器人技术有限公司 The method of transfer robot, shelf, warehousing system and Transport cargo rack
CN109650292A (en) * 2019-02-02 2019-04-19 北京极智嘉科技有限公司 The location regulation method and medium of a kind of intelligent forklift and intelligent forklift

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105674991A (en) * 2016-03-29 2016-06-15 深圳市华讯方舟科技有限公司 Robot positioning method and device
CN108830902A (en) * 2018-04-19 2018-11-16 江南大学 A kind of workpiece identification at random and localization method based on points cloud processing

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1753830A (en) * 2002-12-31 2006-03-29 Abb股份有限公司 A system and a method for container identification
WO2016052379A1 (en) * 2014-09-30 2016-04-07 日本電気株式会社 Information processing device, shelf label management system, control method and program
CN107922119A (en) * 2015-11-25 2018-04-17 株式会社日立制作所 Shelf configuration system, transfer robot and shelf collocation method
CN105479461A (en) * 2016-01-26 2016-04-13 先驱智能机械（深圳）有限公司 Control method, control device and manipulator system
JP2017157216A (en) * 2016-02-29 2017-09-07 サインポスト株式会社 Information processing system
CN106379684A (en) * 2016-11-11 2017-02-08 杭州南江机器人股份有限公司 Submersible AGV abut-joint method and system and submersible AGV
CN107843220A (en) * 2017-10-30 2018-03-27 江苏博睿通智能装备有限公司 Tiered warehouse facility shelf detecting system and detection method
CN109573439A (en) * 2018-12-27 2019-04-05 北京艾瑞思机器人技术有限公司 The method of transfer robot, shelf, warehousing system and Transport cargo rack
CN109650292A (en) * 2019-02-02 2019-04-19 北京极智嘉科技有限公司 The location regulation method and medium of a kind of intelligent forklift and intelligent forklift

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Model-based furniture recognition for building semantic object maps;Martin Günther 等;《Artificial Intelligence》;20150105;336-351 *
自动化立体仓库搬运机器人精定位视觉系统;刘存 等;《机器人》;19920930;第14卷(第5期);54-56页 *

Also Published As

Publication number Publication date
CN110135439A (en) 2019-08-16

Similar Documents

Publication Publication Date Title
CN110135439B (en) Method and device for automatically identifying goods shelf and mobile robot
US10366310B2 (en) Enhanced camera object detection for automated vehicles
EP3809370A1 (en) Apparatus, method, and system for alignment of 3d datasets
CN112847343B (en) Dynamic target tracking and positioning method, device, equipment and storage medium
JP5328979B2 (en) Object recognition method, object recognition device, autonomous mobile robot
US20180074203A1 (en) Lidar Object Detection System for Automated Vehicles
Kang et al. Automatic targetless camera–lidar calibration by aligning edge with gaussian mixture model
JP4046835B2 (en) High-speed surface segmentation method of distance data for mobile robot
CN111612728B (en) 3D point cloud densification method and device based on binocular RGB image
CN110793512A (en) Pose recognition method and device, electronic equipment and storage medium
CN111624622A (en) Obstacle detection method and device
CN104732514A (en) Apparatus, systems, and methods for processing a height map
US11822340B2 (en) Method and system for obstacle avoidance in robot path planning using depth sensors
US11506755B2 (en) Recording medium recording information processing program, information processing apparatus, and information processing method
CN114663526A (en) Obstacle detection method, obstacle detection device, robot and computer-readable storage medium
Al-Durgham et al. RANSAC approach for automated registration of terrestrial laser scans using linear features
CN112269386A (en) Method and device for repositioning symmetric environment and robot
JP5210396B2 (en) A method for calculating the motion of an object from sensor data with the aid of a computer
JP7323061B2 (en) Elevator 3D data processor
WO2021220346A1 (en) Elevator 3-d data processing device
CN114972495A (en) Grabbing method and device for object with pure plane structure and computing equipment
CN112068547A (en) Robot positioning method and device based on AMCL and robot
CN110609561A (en) Pedestrian tracking method and device, computer readable storage medium and robot
Xian et al. A review of fine registration for 3D point clouds
US20220327729A1 (en) Object positioning method and object positioning system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20210806

Address after: 200335 Room 401, floor 4, building 2, No. 33, Guangshun Road, Changning District, Shanghai

Patentee after: Noah robot technology (Shanghai) Co.,Ltd.

Address before: 201400 room 2340, building 2, Lane 1800, Xinyang highway, Fengxian District, Shanghai

Patentee before: SHANGHAI MUMU JUCONG ROBOT TECHNOLOGY Co.,Ltd.

TR01 Transfer of patent right