JP2017146708A - Object recognition device and three-dimensional point group modeling device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow an object to be accurately recognized from a three-dimensional point group.SOLUTION: A histogram generation unit 46 calculates a histogram representing a recognition target object. A box size determination unit 48 calculates a box representing a size of a three-dimensional point group representing the recognition target object, from the three-dimensional point group. A box histogram calculation unit 50 calculates a histogram representing the box in an arrangement pattern. A histogram comparison unit 52 compares a histogram calculated for each arrangement pattern and the histogram calculated by the histogram generation unit 46 to calculate a score for each arrangement pattern. An object recognition processing unit 54 determines an arrangement pattern in accordance with scores for respective arrangement patterns of the box. A coordinate restoration unit 56 restores three-dimensional coordinates where the recognition target object exists, from the arrangement pattern.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an object recognition device, a three-dimensional point cloud modeling device, a method, and a program, and more particularly to an object recognition device and a three-dimensional point for recognizing a recognition object included in a three-dimensional point cloud representing an environment. The present invention relates to a group modeling apparatus, method, and program.

  When an object is recognized from a three-dimensional point group that is a measurement result measured by a laser range finder, a large noise often occurs in the three-dimensional measurement result depending on the reflectance of the surface and the orientation of the surface. In order to recognize an object from the data of the three-dimensional point group measured in this way, an algorithm that is resistant to noise is required.

  Among object detection algorithms in a two-dimensional image, there is Bag of Visual Words (hereinafter referred to as BoVW) as a characteristic expression that is resistant to noise (see Non-Patent Document 1).

  In general, a BoVW vector is extracted from a number of two-dimensional images of an object to be recognized and stored in a storage unit, and the BoVW vector extracted from the input image is compared with a vector in the storage unit. By recognizing the object.

J. Sivic et al .: "Video Google: A Text Retrieval Approach to Object Matching in Videos", in Proc. ICCV, 2003. F. Tombari, S. Salti, and L. Di Stefano, "SHOT: Unique signatures of histograms for surface and texture description", in 11th European Conference on Computer Vision (ECCV)

  The object recognition method in the above two-dimensional image is robust, but cannot be used for object recognition by a three-dimensional point group as it is.

  The present invention has been made to solve the above-described problems, and is an object recognition device that can accurately recognize an object from a three-dimensional point cloud, a three-dimensional point cloud modeling device, a method, and a program. The purpose is to provide.

  In order to achieve the above object, an object recognition apparatus according to a first invention uses the three-dimensional point group representing a recognition target object and the three-dimensional point group representing an environment in which the recognition target object exists, to An object recognition apparatus for recognizing a recognition target object included in a three-dimensional point group representing a normal vector calculation unit for calculating a normal vector for each point in the three-dimensional point group of the recognition target object; , A local feature amount calculation unit for calculating a local feature amount representing information around a point for each point in the three-dimensional point group of the recognition target object, and a three-dimensional point group representing the environment in advance. A local feature amount assigning unit that assigns a local feature amount calculated from a three-dimensional point group representing a recognition target object by the local feature amount calculating unit to a cluster center of the local feature amount, and a local feature amount assigning unit Is it a result of assigning features? The number of local features assigned to each cluster is calculated, a histogram creation unit that calculates a histogram representing the recognition target object, and the size of the three-dimensional point group from the three-dimensional point group representing the recognition target object. A box size determination unit for calculating a box to be expressed, and a box in the arrangement pattern for each arrangement pattern in which the box calculated by the box size determination unit is arranged in a three-dimensional point group representing the environment. A box histogram calculation unit that calculates a histogram to be compared, the histogram calculated for each arrangement pattern by the box histogram calculation unit, and a histogram calculated by the histogram creation unit are compared, and for each arrangement pattern A histogram comparison unit for calculating a score, and the histogram An object recognition processing unit that determines a box arrangement pattern such that the recognition target object is included in the box from the score for each box arrangement pattern calculated by the comparison unit, and the arrangement obtained by the object recognition processing unit A coordinate restoration unit that restores the three-dimensional coordinates where the recognition target object exists from the pattern.

  In the object recognition device according to the first aspect of the present invention, the three-dimensional space in which the point cloud representing the environment exists is discretized by voxels having a predetermined size, and each of the three-dimensional point cloud representing the environment in advance is discretized. A predetermined containing voxel for each of the discretized voxels in the three-dimensional space based on a local feature calculated for a point and a cluster center assigned to the local feature A three-dimensional integral image creation unit that calculates a three-dimensional integral image storing a histogram calculated from local feature amounts included in the range of the box histogram calculation unit, the box histogram calculation unit for each arrangement pattern of the box Hiss representing the box of the arrangement pattern by calculating the sum or difference at most eight times using the histogram stored in each voxel of the three-dimensional integral image. It is also possible to calculate the grams.

  In the object recognition apparatus according to the first aspect of the invention, the local feature amount calculation unit may perform SHOT (Signature of Histograms of Orientations) using the normal vector calculated for each point by the normal vector calculation unit. ) May be calculated.

  A three-dimensional point cloud modeling apparatus according to a second invention includes a normal vector calculation unit that calculates a normal vector for each point in the three-dimensional point group of the recognition target object, and the three-dimensional of the recognition target object. A local feature quantity calculation unit that calculates local feature quantities that express information around a point for each point in the point cloud, and a local feature quantity that is created in advance from a three-dimensional point cloud that represents the environment in which the recognition target object exists A local feature amount assigning unit for assigning a local feature amount calculated from a three-dimensional point group representing a recognition target object by the local feature amount calculating unit, and a local feature amount by the local feature amount assigning unit. The number of local features assigned to each cluster is calculated from the assigned result, a histogram creating unit for calculating a histogram representing the recognition target object, and the three-dimensional point from the three-dimensional point group representing the recognition target object A box size determining unit for calculating a box expressing the size of the box; and for each arrangement pattern for arranging the box calculated by the box size determining unit in a three-dimensional point group representing the environment, the arrangement pattern And a box histogram calculation unit for calculating a histogram representing a box in FIG.

  An object recognition method according to a third invention includes a normal vector calculation unit, a local feature amount calculation unit, a local feature amount allocation unit, a histogram creation unit, a box size determination unit, a box histogram calculation unit, a histogram comparison unit, and an object recognition process. Recognition using the point cloud representing the recognition target object and the point cloud representing the environment where the recognition target object exists to recognize the recognition target object included in the point cloud representing the environment A method in which the normal vector calculation unit calculates a normal vector for each point in a three-dimensional point group of the recognition target object; and the local feature value calculation unit includes the recognition target object. Calculating a local feature amount expressing information around the point for each point in the three-dimensional point group, and the local feature amount assigning unit previously created from the three-dimensional point group representing the environment Feature value A step of assigning a local feature amount calculated from a three-dimensional point group representing a recognition target object by the local feature amount calculation unit to the cluster center; and the histogram generation unit includes a local feature amount by the local feature amount assignment unit Calculating the number of local feature values assigned to each cluster from the result of assigning, and calculating a histogram representing a recognition target object; and the box size determination unit includes a three-dimensional point group representing the recognition target object Calculating a box expressing the size of the three-dimensional point group from the box, and the box histogram calculating unit arranges the box calculated by the box size determining unit in the three-dimensional point group representing the environment. For each arrangement pattern, calculating a histogram representing a box in the arrangement pattern; The histogram comparison unit compares the histogram calculated for each arrangement pattern by the box histogram calculation unit with the histogram calculated by the histogram creation unit, and calculates a score for each arrangement pattern The object recognition processing unit determines a box arrangement pattern such that the recognition target object is included in the box from the score for each box arrangement pattern calculated by the histogram comparison unit; and the coordinates And a restoring unit that executes the three-dimensional coordinates in which the recognition target object exists from the arrangement pattern obtained by the object recognition processing unit.

  In the object recognition method according to the third aspect of the invention, the three-dimensional integral image creation unit discretizes a three-dimensional space in which a point cloud representing the environment exists with voxels having a predetermined size, Each of the discretized voxels in the three-dimensional space is based on the local feature amount calculated for each point in the three-dimensional point group to be represented and the cluster center assigned to the local feature amount. On the other hand, the method further includes a step of calculating a three-dimensional integral image storing a histogram calculated from a local feature included in a predetermined range including the voxel, and the step of calculating the box histogram calculation unit includes: For the arrangement pattern, the arrangement pattern is calculated by calculating the sum or difference at most eight times using the histogram stored in each voxel of the three-dimensional integral image. It may be calculated a histogram representing the over down box.

  A program according to a fourth invention is a program for causing a computer to function as each unit constituting the object recognition device according to the first invention.

  According to the object recognition device, the three-dimensional point cloud modeling device, the method, and the program of the present invention, it is possible to obtain an effect that an object can be recognized from the three-dimensional point cloud with high accuracy.

It is a block diagram which shows the structure of the object recognition apparatus which concerns on embodiment of this invention. 3 is a diagram illustrating an example of a basic data storage unit 21. FIG. 3 is a diagram illustrating an example of a normal vector storage unit 30. FIG. 4 is a diagram illustrating an example of a local feature amount storage unit 32. FIG. 3 is a diagram illustrating an example of a cluster storage unit 34. FIG. 4 is a diagram illustrating an example of an integral image storage unit 36. FIG. 4 is a diagram illustrating an example of a recognition result storage unit 38. FIG. It is a flowchart which shows the pre-processing routine in the object recognition apparatus which concerns on embodiment of this invention. It is a flowchart which shows the object recognition process routine in the object recognition apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>

  In the present embodiment, “SHOT (Signature of Histograms of Orientations)” that can be used even in a three-dimensional point cloud is used as a feature amount, thereby recognizing an object while covering various shapes. In addition, by representing an object using BoVW, it is possible to cover an unstable surface of SHOT that is easily affected by a minute change in shape due to the influence of noise such as a measurement error.

<Configuration of Object Recognition Device According to Embodiment of the Present Invention>

  First, the configuration of the object recognition device according to the embodiment of the present invention will be described. As shown in FIG. 1, an object recognition apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program for executing an object recognition processing routine to be described later and various data. Can be configured with a computer. Functionally, the object recognition apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 60 as shown in FIG.

  The input unit 10 receives a three-dimensional point group including a plurality of three-dimensional points in an environment where an object exists. The three-dimensional point group is a three-dimensional point group representing a plurality of positions that are measurement results obtained by measuring positions on the surface of an object, and is a three-dimensional point group such as a laser range finder, an infrared sensor, or an ultrasonic sensor. It was obtained by a device that measures

  Further, the input unit 10 receives a three-dimensional point group representing a recognition target object. The received three-dimensional point group is stored in the basic data storage unit 21.

  The calculation unit 20 includes a preprocessing unit 22, a basic data storage unit 21, a normal vector storage unit 30, a local feature storage unit 32, a cluster storage unit 34, an integrated image storage unit 36, and a recognition result storage. Unit 38, normal vector calculation unit 40, local feature quantity calculation unit 42, local feature quantity allocation unit 44, histogram creation unit 46, box size determination unit 48, box histogram calculation unit 50, and histogram comparison A unit 52, an object recognition processing unit 54, and a coordinate restoration unit 56 are configured.

  The preprocessing unit 22 receives a plurality of three-dimensional point groups in the environment where the object exists by processing of each unit described below, and normal vectors, local feature amounts, clusters, and three-dimensional integration necessary for object recognition Learn images.

  The preprocessing unit 22 includes a normal vector calculation unit 23, a local feature amount calculation unit 24, a clustering unit 25, and a three-dimensional integral image creation unit 26.

  The basic data storage unit 21 stores a three-dimensional point group in an environment where an object exists, which is received by the input unit 10 and used for preprocessing. In addition, a three-dimensional point group representing a recognition target object received by the input unit 10 is stored. In addition, you may make it receive in real time via the input part 10 from an external laser range finder, without providing the basic data memory | storage part 21. FIG. An example of the basic data storage unit 21 is shown in FIG.

  The normal vector calculation unit 23 uses, for each point of the three-dimensional point group in the environment where the object stored in the basic data storage unit 21 exists, the X, Y, Z coordinates using the three-dimensional points around the point. A normal vector is calculated as the unit vector in, and stored in the normal vector storage unit 30. For the calculation of the normal vector, the method described in Non-Patent Document 3 is used.

[Non-Patent Document 3]: K. Klasing, D. Althoff, D. Wollherr, M. Buss, “Comparison of Surface Normal Estimation Methods for Range Sensing Applications”, Robotics and Automation, 2009.

  The normal vector storage unit 30 stores N normal vectors calculated by the normal vector calculation unit 23 for the point cloud in the environment where the object exists. Further, N normal vectors calculated for the recognition target object by the normal vector calculation unit 40 are stored. An example of the normal vector storage unit 30 is shown in FIG.

  The local feature amount calculation unit 24 calculates the local feature amount of the point based on the normal vector stored in the normal vector storage unit 30 for each point of the three-dimensional point group in the environment where the object exists, Stored in the local feature amount storage unit 32. Here, for each of the M voxels having a predetermined size for the local feature amount, SHOT is used by using a normal vector calculated for a predetermined number of three-dimensional points close to the center position of the voxel. A feature amount is calculated (see Non-Patent Document 2 above). The local feature amount may be calculated for each of M points sampled from the three-dimensional point group.

  The local feature quantity storage unit 32 stores M local feature quantities calculated by the local feature quantity calculation unit 24. Further, the cluster number of the cluster center assigned by the clustering unit 25 described later is assigned to the local feature amount. The local feature amount is expressed by a D-dimensional vector. In addition, the coordinate values of x, y, and z (the coordinate value of the center of the voxel for the local feature amount) are also stored in association with the local feature amount. An example of the local feature amount storage unit 32 is shown in FIG.

  Based on the M local feature amounts calculated by the local feature amount calculating unit 24, the clustering unit 25 uses the code book learning method described in Non-Patent Document 1 to determine the cluster centers for the K clusters. Calculate and assign each of the M local features to each cluster. The calculated cluster center is stored in the cluster storage unit 34. Further, as a result of assigning each local feature quantity to each cluster, the cluster number for each local feature quantity is stored in the local feature quantity storage unit 32. The number of clusters K is determined in advance by setting. Here, a significant cluster center can be obtained by clustering using a large amount of three-dimensional point cloud data.

  The cluster storage unit 34 stores the cluster centers of K clusters calculated by the clustering unit 25. An example of the cluster storage unit 34 is shown in FIG.

  The three-dimensional integral image creation unit 26 discretizes a three-dimensional space including a three-dimensional point group representing an environment in which an object exists with voxels having a predetermined size for the three-dimensional integral image. It is assumed that the maximum coordinate values including the point group among the discretized coordinate values of x, y, and z are Xmax, Ymax, and Zmax, respectively. Next, the discretization in the three-dimensional space is performed based on the M local feature amounts calculated by the local feature amount calculation unit 24 and the clusters assigned to the M local feature amounts by the clustering unit 25. For each voxel, calculate a three-dimensional integral image that contains a histogram representing the frequency of each cluster, calculated from local features included in the rectangular parallelepiped range defined by the voxel and the origin of the three-dimensional space. To do. Extend the idea of 2D integral images to 3D and use the sum and difference of histograms stored for voxels around the box to be calculated so that the frequency of each cluster in the box to be calculated can be determined. To do. The unit for integration is determined in advance by setting. The local feature quantity voxel size may be specified independently.

  The integral image storage unit 36 stores a three-dimensional integral image calculated by the three-dimensional integral image creation unit 26. Specifically, as shown in FIG. 6, the voxel index number and the frequency of each cluster are stored for each voxel.

  The normal vector calculation unit 40 calculates a normal vector for each point in the three-dimensional point group of the recognition target object stored in the basic data storage unit 21 in the same manner as the normal vector calculation unit 23, It is stored in the normal vector storage unit 30.

  The local feature amount calculation unit 42 calculates a local feature amount for each point sampled from the three-dimensional point group of the recognition target object based on the three-dimensional point group of the recognition target object stored in the basic data storage unit 21. And stored in the local feature quantity storage unit 32. Here, as with the local feature value calculation unit 24, a feature value called SHOT is calculated using a normal vector calculated for each point (see Non-Patent Document 2 above).

  The local feature amount assigning unit 44 includes the cluster center of each cluster stored in the cluster storage unit 34 and the local feature amount calculated for each point of the three-dimensional point group representing the recognition target object by the local feature amount calculating unit 42. Based on the above, each of the local feature amounts calculated for each point of the three-dimensional point group representing the recognition target object is assigned to each cluster.

The histogram creation unit 46 expresses the recognition target object by calculating the number of local feature values assigned to each cluster from the result of assigning the local feature value for each point to each cluster by the local feature value assigning unit 44. Then, a histogram representing the appearance frequency of each cluster is calculated.

  The box size determination unit 48 calculates the size of a box expressing the size of the three-dimensional point group from the three-dimensional point group representing the recognition target object. The size in the x direction can be calculated by the following equation (1) using the measurement data in the x direction of the three-dimensional point group.

  The maximum value and the minimum value are the maximum value and the minimum value in the x direction of the three-dimensional point group. r is the length of one side of the voxel for the three-dimensional integral image, and the size of how many voxels the box will have is determined. The above can be similarly calculated for y and z as in the case of the x direction.

  Further, the box size determination unit 48 scans a three-dimensional space including a three-dimensional point group representing the environment stored in the basic data storage unit 21 and discretized by voxels. The boxes of the calculated size are arranged, and each arrangement pattern is determined. In addition, you may make it determine further the arrangement pattern which arrange | positioned what changed the magnitude | size of the box in three-dimensional space.

  The box histogram calculation unit 50 uses the three-dimensional integrated image stored in the integrated image storage unit 36 to set a box in the arrangement pattern for each arrangement pattern in which the box determined by the box size determination unit 48 is arranged. Calculate the histogram to represent. The calculated histogram represents the appearance frequency of each cluster assigned to the local feature amount existing in the box. A histogram representing a box in the arrangement pattern is calculated by calculating eight sums or differences at most using the histogram stored in each voxel of the three-dimensional integral image.

  The histogram comparison unit 52 compares the histogram calculated for each arrangement pattern by the box histogram calculation unit 50 with the histogram calculated by the histogram creation unit 46, and calculates a score for each arrangement pattern. For example, the vector similarity of a vector representing a histogram is calculated as a score.

  The recognition result storage unit 38 stores a score calculated for each arrangement pattern by the histogram comparison unit 52. An example of the recognition result storage unit 38 is shown in FIG. An index and a score corresponding to the arrangement pattern are stored.

  The object recognition processing unit 54 determines a box arrangement pattern in which the recognition target object is included in the box from the score for each box arrangement pattern calculated by the histogram comparison unit 52. Here, if the score value is equal to or greater than the threshold value and the score of the index of the arrangement pattern of interest is maximal (highest score) compared to the score for the index of the surrounding arrangement pattern, the index It is recognized that an object exists at a position corresponding to. Then, the recognition result is added to the recognition result storage unit 38 for the index recognized that the object exists.

  The coordinate restoration unit 56 is discretized by performing an inverse calculation to the above equation (1) from the arrangement pattern index obtained by the object recognition processing unit 54 and the box size obtained by the box size determination unit 48. The three-dimensional coordinates where the recognition target object exists are restored from the obtained coordinate values and are output to the output unit 60.

<Operation of Object Recognition Device According to Embodiment of Present Invention>

  Next, the operation of the object recognition apparatus 100 according to the embodiment of the present invention will be described. When the input unit 10 receives a three-dimensional point cloud in an environment where an object exists and stores it in the basic data storage unit 21, the object recognition apparatus 100 executes a preprocessing routine shown in FIG.

  First, in step S100, a normal vector is calculated as a unit vector in the X, Y, and Z coordinates for each point of the three-dimensional point group in the environment where the object stored in the basic data storage unit 21 exists, and the normal line is calculated. Store in the vector storage unit 30.

  Next, in step S102, based on the three-dimensional point cloud in the environment where the object stored in the basic data storage unit 21 exists and the normal vector of each point stored in the normal vector storage unit 30, the local data A local feature amount is calculated for each of M voxels having a predetermined size for the feature amount, and stored in the local feature amount storage unit 32.

  In step S104, the cluster center is calculated for K clusters using the codebook learning method described in Non-Patent Document 1 based on the M local feature amounts calculated by the local feature amount calculation unit 24. Then, each of the M local feature values is assigned to each cluster, the cluster number for each of the M local feature values is stored in the local feature value storage unit 32, and the cluster center of each cluster is stored in the cluster storage unit 34. To do.

  In step S106, the three-dimensional space including the three-dimensional point group representing the environment stored in the basic data storage unit 21 is discretized by voxels having a predetermined size.

  In step S108, based on the M local feature values calculated in step S104 and the clusters assigned to the M local feature values in step S104, each of the discretized voxels in the three-dimensional space is assigned. On the other hand, a three-dimensional integrated image storing a histogram calculated from local feature amounts included in a rectangular parallelepiped range defined by the voxel and the origin of the three-dimensional space is calculated and stored in the integrated image storage unit 36. .

  Next, the object recognition processing routine will be described. When the input unit 10 receives a three-dimensional point cloud representing a recognition target object and stores it in the basic data storage unit 21, the object recognition device 100 executes an object recognition processing routine shown in FIG.

  In step S <b> 200, a normal vector is calculated for each point in the three-dimensional point group of the recognition target object stored in the basic data storage unit 21 and stored in the normal vector storage unit 30.

  In step S202, a local feature amount is calculated for each point sampled from the three-dimensional point group based on the three-dimensional point group of the recognition target object stored in the basic data storage unit 21, and the local feature amount storage unit 32 is calculated. To store.

  In step S204, the local feature amount calculated for each point of the three-dimensional point group representing the recognition target object in step S202 based on the cluster center of each cluster stored in the cluster storage unit 34 is assigned to each cluster.

  In step S206, the number of local feature values assigned to each cluster is calculated from the result of assigning the local feature values to the clusters in step S204, and a histogram representing the recognition target object is calculated.

  In step S208, the size of the box expressing the size of the three-dimensional point group is calculated from the three-dimensional point group representing the recognition target object by the above equation (1).

  In step S210, the box whose size is calculated in step S208 is added to the three-dimensional space including the three-dimensional point cloud representing the environment stored in the basic data storage unit 21 and discretized by the voxels. Arrange and determine each placement pattern of the box.

  In step S211, using the three-dimensional integrated image stored in the integrated image storage unit 36, for each arrangement pattern in which the box determined in step S210 is arranged, a histogram expressing the box in the arrangement pattern is calculated. To do. A histogram representing a box in the arrangement pattern is calculated by calculating eight sums or differences at most using the histogram stored in each voxel of the three-dimensional integral image.

  In step S212, the histogram calculated for each arrangement pattern in step S210 is compared with the histogram calculated in step S206, and a score is calculated for each arrangement pattern.

  In step S214, an arrangement pattern having the highest score is determined as a box arrangement pattern in which a recognition target object is included in the box from the scores for the respective arrangement patterns of boxes calculated in step S212.

  In step S216, the three-dimensional coordinates where the recognition target object is present are restored from the arrangement pattern obtained by the object recognition processing unit 54, output to the output unit 60, and the process ends.

  As described above, according to the object recognition apparatus according to the embodiment of the present invention, the histogram representing the recognition target object is calculated, and the size of the three-dimensional point group is calculated from the three-dimensional point group representing the recognition target object. Calculating a box representing a box in the arrangement pattern, determining a placement pattern in which the box is placed in a three-dimensional space including a three-dimensional point cloud representing an environment in which the recognition target object exists, Comparing the histogram calculated for the placement pattern with the histogram representing the recognition target object, calculating the score for each placement pattern, determining the placement pattern from the score for each placement pattern in the box, and placing By reconstructing the 3D coordinates where the recognition target object exists from the pattern, it is possible to recognize the object from the 3D point cloud with high accuracy. Kill.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, although the object recognition apparatus 100 has been described in the above-described embodiment, the present invention is not limited to this apparatus and configuration. For example, in order to model a recognition target object, a basic data storage unit 21, a normal vector storage unit 30, a local feature storage unit 32, a cluster storage unit 34, an integrated image storage unit 36, a normal line Three-dimensional point cloud modeling including a vector calculation unit 40, a local feature value calculation unit 42, a local feature value allocation unit 44, a histogram creation unit 46, a box size determination unit 48, and a box histogram calculation unit 50 You may make it comprise as an apparatus.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 22 Pre-processing part 21 Basic data storage part 23 Normal vector calculation part 24 Local feature quantity calculation part 25 Clustering part 26 Three-dimensional integral image creation part 30 Normal vector storage part 32 Local feature quantity storage part 34 Cluster storage unit 36 Integrated image storage unit 38 Recognition result storage unit 40 Normal vector calculation unit 42 Local feature amount calculation unit 44 Local feature amount allocation unit 46 Histogram creation unit 48 Box size determination unit 50 Box histogram calculation unit 52 Histogram comparison unit 54 Object recognition processing unit 56 Coordinate restoration unit 60 Output unit 100 Object recognition device

Claims (7)

An object recognition apparatus for recognizing a recognition target object included in a three-dimensional point group representing an environment using a three-dimensional point group representing a recognition target object and a three-dimensional point group representing an environment in which the recognition target object exists. There,
A normal vector calculator that calculates a normal vector for each point in the three-dimensional point cloud of the recognition target object;
A local feature amount calculation unit that calculates a local feature amount that represents information around a point for each point in the three-dimensional point group of the recognition target object;
A local feature in which a local feature amount calculated from a three-dimensional point group representing a recognition target object by the local feature amount calculation unit is assigned to a cluster center of the local feature amount previously created from the three-dimensional point group representing the environment. A quantity allocation unit;
Calculating the number of local feature values assigned to each cluster from the result of assigning local feature values by the local feature value assigning unit, and calculating a histogram expressing a recognition target object;
A box size determining unit for calculating a box expressing the size of the three-dimensional point group from the three-dimensional point group representing the recognition target object;
A box histogram calculation unit for calculating a histogram representing a box in the arrangement pattern for each arrangement pattern in which the box calculated by the box size determination unit is arranged in a three-dimensional point group representing the environment;
A histogram comparison unit that compares the histogram calculated for each arrangement pattern by the box histogram calculation unit and a histogram calculated by the histogram creation unit, and calculates a score for each arrangement pattern;
An object recognition processing unit that determines a layout pattern of a box such that the recognition target object is included in the box from a score for each layout pattern of the box calculated by the histogram comparison unit;
A coordinate restoration unit that restores the three-dimensional coordinates where the recognition target object exists from the arrangement pattern obtained by the object recognition processing unit;
An object recognition apparatus including: A three-dimensional space in which a point cloud representing the environment exists is discretized by a voxel having a predetermined size, and a local feature amount calculated in advance for each point in the three-dimensional point cloud representing the environment; A histogram calculated from local features included in a predetermined range including the voxels for each of the discretized voxels in the three-dimensional space based on the cluster center assigned to the local features. A three-dimensional integral image creating unit for calculating a three-dimensional integral image storing
The box histogram calculation unit represents the box of the arrangement pattern by calculating the sum or difference at most eight times using the histogram stored in each voxel of the three-dimensional integral image for each arrangement pattern of the box. The object recognition apparatus according to claim 1, wherein a histogram to be calculated is calculated.   The object according to claim 1, wherein the local feature amount calculation unit calculates SHOT (Signature of Histograms of Orientations) using the normal vector calculated for each point by the normal vector calculation unit. Recognition device. A normal vector calculator that calculates a normal vector for each point in the 3D point cloud of the recognition target object;
A local feature amount calculation unit that calculates a local feature amount that represents information around a point for each point in the three-dimensional point group of the recognition target object;
The local feature calculated from the three-dimensional point group representing the recognition target object by the local feature amount calculation unit with respect to the cluster center of the local feature amount previously created from the three-dimensional point group representing the environment where the recognition target object exists A local feature amount assigning unit for assigning an amount;
Calculating the number of local feature values assigned to each cluster from the result of assigning local feature values by the local feature value assigning unit, and calculating a histogram expressing a recognition target object;
A box size determining unit for calculating a box expressing the size of the three-dimensional point group from the three-dimensional point group representing the recognition target object;
A box histogram calculation unit for calculating a histogram representing a box in the arrangement pattern for each arrangement pattern in which the box calculated by the box size determination unit is arranged in a three-dimensional point group representing the environment;
3D point cloud modeling device. It includes a normal vector calculation unit, a local feature amount calculation unit, a local feature amount allocation unit, a histogram creation unit, a box size determination unit, a box histogram calculation unit, a histogram comparison unit, an object recognition processing unit, and a coordinate restoration unit, and is a recognition target An object recognition method for recognizing a recognition target object included in a point group representing an environment using a point group representing an object and a point group representing an environment where the recognition target object exists,
The normal vector calculation unit calculating a normal vector for each point in the three-dimensional point group of the recognition target object;
The local feature amount calculation unit calculating a local feature amount expressing information around the point for each point in the three-dimensional point group of the recognition target object;
The local feature amount assigning unit is calculated from the three-dimensional point group representing the recognition target object by the local feature amount calculating unit with respect to the cluster center of the local feature amount previously created from the three-dimensional point group representing the environment. Assigning local feature quantities,
Calculating the number of local feature values assigned to each cluster from the result of assigning local feature values by the local feature value assigning unit, and calculating a histogram representing a recognition target object;
The box size determining unit calculating a box expressing the size of the three-dimensional point group from the three-dimensional point group representing the recognition target object;
The box histogram calculation unit calculates a histogram representing a box in the arrangement pattern for each arrangement pattern in which the box calculated by the box size determination unit is arranged in a three-dimensional point group representing the environment. Steps,
The histogram comparison unit compares the histogram calculated for each arrangement pattern by the box histogram calculation unit with the histogram calculated by the histogram creation unit, and calculates a score for each arrangement pattern When,
The object recognition processing unit determines a box arrangement pattern such that the recognition target object is included in a box from a score for each box arrangement pattern calculated by the histogram comparison unit;
The coordinate restoration unit restoring the three-dimensional coordinates where the recognition target object exists from the arrangement pattern obtained by the object recognition processing unit;
An object recognition method including: The three-dimensional integral image creation unit discretizes a three-dimensional space in which the point cloud representing the environment exists with voxels having a predetermined size, and calculates in advance for each point in the three-dimensional point cloud representing the environment. And each of the discretized voxels in the three-dimensional space is included in a predetermined range including the voxel based on the determined local feature and the cluster center assigned to the local feature. A step of calculating a three-dimensional integral image storing a histogram calculated from the local features;
The step of calculating by the box histogram calculation unit includes calculating the arrangement pattern by performing at most eight sums or differences using the histogram stored in each voxel of the three-dimensional integral image for each arrangement pattern of the box. The object recognition method according to claim 5, wherein a histogram representing a box is calculated.   The program for functioning a computer as each part which comprises the object recognition apparatus of any one of Claims 1-3.