JP6851295B2 - Object detection system, object detection device, object detection program, and object detection method - Google Patents

Description

The present invention relates to an object detection system, an object detection device, an object detection program, and an object detection method that detect an object by sensing the real space.

Conventionally, there has been known a three-dimensional object detection (recognition) technique that senses a real space to obtain 3D data and detects an object object based on the 3D data. This three-dimensional object detection technology is applied to AR (Augmented Reality), VR (Virtual Reality), and MR (Mixed Reality).

In model-based 3D object detection, 3D data (CAD data) of an object to be detected is prepared in advance as reference 3D data, and 3D data in real space is acquired by sensing with a 3D data acquisition means such as a depth sensor. , The object is detected by matching the acquired 3D data with the reference 3D data.

The following documents describe the techniques related to the present invention.

Special Table 2013-513191

Yu Aida, Keiji Yanai, Kazutomo Shibahara, Koji Fujimoto, "Extracting Color Information from Clothing Areas for Clothing Image Mining", IEICE Technical Report, vol. 111, no. 478, IE2011-173, pp. 235-240, 2012. Shuichi Akizuki, Manabu Hashimoto, “Position and Pose Recognition of Randomly Stacked Objects using Highly Observable 3D Vector Pairs”, Proc. The 40th Annual Conference of the IEEE Industrial Electronics Society, pp.5266-5271, Oct. 2014. Asako Kanasaki, Emanuele Rodola, Tatsuya Harada, "Object Detection from RGB-D Images Using Graph Matching Learning", 20th Robotics Symposia, pp.432-437, 2015. Tao Wang, Haibin Ling, Congyan Lang, Jun Wu, “Branching Path Following for Graph Matching”, Computer Vision --ECV 2016, pp.508-523. Fayao Liu, Chunhua Shen, Guosheng Lin, “Deep Convolutional Neural Fields for Depth Estimation from a Single Image”, ArXive-prints 1411, 6387. Shu Liu, Xiaojuan Qi, Jianping Shi, Hong Zhang, Jiaya Jia, “Multi-scale Patch Aggregation (MPA) for Simultaneous Detection and Segmentation”, CVPR, 2016.

As described above, in model-based 3D object detection, it is necessary to sense the real space to acquire 3D data of the object, but in reality, 3D data of a part of the object may not be obtained. For example, in the case of optically acquiring 3D data, when a shield exists between the 3D data acquisition means and the object, the 3D data of the object is missing for the shield portion. Further, even when a part of the object is made of metal, a material having high gloss, or is transparent, effective 3D data may not be obtained for the part. Furthermore, in principle, 3D data of an object may be obtained only in a limited range due to the installation position of the 3D data acquisition means.

As described above, if the 3D data of the object in the real space is missing (insufficient point cloud), the matching with the reference 3D data may not be established, or the matching accuracy may be lowered.

An object of the present invention is to improve the possibility of object detection by matching with reference data even when the 3D data of the object obtained by sensing is insufficient in the model-based 3D object detection. To do.

The object detection system of one aspect of the present invention includes a storage means for storing 3D data of an object object as reference 3D data, a 3D data acquisition means for sensing real space to acquire measurement 3D data, and the measurement 3D data. The area specifying means for specifying the detection target area in the above, the pseudo 3D data adding means for adding pseudo 3D data to the missing area where the measurement 3D data for the target object is insufficient, the measurement 3D data, and the pseudo 3D. A detection processing means for detecting the target object by matching the query 3D data based on the data with the reference 3D data is provided , and the area identification unit is included in the measurement 3D data. With respect to a connecting region in which segments that are close to each other and adjacent to each other are connected, there is a difference in distance between the other connecting regions adjacent to the connecting region and a predetermined threshold or more at the boundary with the connecting region, and the distance is larger than that of the connecting region. When there is the other connecting region at a short distance, the connecting region is specified as a detection target region, and the other region is specified as the missing region .

With this configuration, even if the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then the matching with the reference 3D data is performed. It is possible to improve the accuracy of matching when the measurement 3D data is insufficient, and thus improve the possibility of detecting the target object.

The object detection system may further include a graph generation means for generating a graph having 3D data as a node, and the 3D data acquisition means acquires 3D point group data in the real space by sensing. The data acquisition means and the feature point extraction means for extracting the feature points as the measurement 3D data from the 3D point group data may be included, and the storage means may include the feature points of the target object as the reference 3D data. The graph may be stored, the pseudo 3D data adding means may give a pseudo feature point as the pseudo 3D data, and the graph generating means may use the measured 3D data and the pseudo 3D data as a node for a query. A graph may be generated, and the detection processing means may use the query graph as the query 3D data and perform graph matching with the reference 3D data.

With this configuration, the target object can be detected by graph matching using a graph of feature points. As a 3D point cloud data acquisition means for acquiring 3D point cloud data in real space by sensing, for example, a depth sensor, stereo ranging using a stereo camera image, and a deep convolutional neural network are used. A method of estimating a depth image from a monocular camera image, a method of acquiring dense depth data by combining a LiDAR (Light Detection and Ranging) and a camera, and the like can be adopted.

The object detection system captures the missing area based on an image acquisition means that captures the real space and acquires an image, an object detection means that detects a foreground object from the image, and a detection result of the foreground object. The estimation means for estimating may be further provided, and the pseudo 3D data addition means may add the pseudo 3D data to the missing area estimated by the estimation means.

With this configuration, an object that can be detected by object detection on an image (RGB data) can be detected from the image, and an object that cannot be detected from the image can be detected by object detection using 3D data.

The object detection system may further include a contour calculation means for cutting out the foreground object by a graph cut process and calculating a contour in which the edge of the cut out portion is expanded, and the estimation means is the measurement 3D data. A rectangular parallelepiped including the feature points corresponding to the contour, which is the feature points extracted from the above, may be estimated as the missing region.

In the above object detection system, the pseudo 3D data adding means sets a plurality of virtual planes arranged in the depth direction in the missing area, and adds the pseudo 3D data on a plurality of concentric ellipses in the virtual plane. Good.

The object detection system may further include an image acquisition means for capturing the real space and acquiring an image, and the 3D data acquisition means acquires 3D point group data in the real space by sensing. The group data acquisition means may be provided, and the area specifying means may specify the detection target area based on the segmentation information using the image synchronized with the 3D point group data.

With this configuration, the detection target area can be specified based on the image.

In the above-mentioned object detection system, the area specifying means may specify the detection target area by excluding the area where the distance is within a predetermined range from the 3D point cloud data.

With this configuration, the target object can be detected without searching for a useless area, and the processing time required for detection can be reduced. For example, a region where the distance is equal to or greater than a predetermined threshold value may be excluded.

In the object detection system, the 3D data acquisition means may include a 3D point cloud data acquisition means for acquiring 3D point cloud data in the real space by sensing, and the area identification means may include the 3D point cloud data. The shielded area that shields the target object may be specified as the missing area based on the above.

With this configuration, pseudo 3D data can be added to the shielded area.

In the above-mentioned object detection system, the area specifying means is a region adjacent to the detection target region, and the distance of the 3D point cloud data is shorter than the distance of the 3D point cloud data of the detection target region. The missing region may be specified by using a region where the difference between the distance of the 3D point cloud data and the distance of the detection target region from the 3D point cloud data is equal to or greater than a threshold value as the shielding region.

With this configuration, the shielding area can be preferably specified.

In the above-mentioned object detection system, the pseudo 3D data adding means may add the pseudo 3D data on a spherical surface centered on a starting point in the detection target region.

With this configuration, pseudo 3D data can be added on a spherical surface centered on the starting point. The starting point may be a feature point extracted from the detection target area. The starting point can be a point close to the shielded area in the detection target area. Further, a plurality of starting points may be set, and a plurality of predetermined distances may be set. Instead of the above, the pseudo 3D data addition means may add pseudo 3D data using an average distribution obtained by aligning a plurality of target objects in which reference 3D data is stored in the storage unit with the center of gravity. ..

In the object detection system, the query 3D data may be weighted so that the weight of the pseudo 3D data is lighter than that of the measurement 3D data.

With this configuration, it is possible to perform matching with more emphasis on the information of the measurement 3D data obtained by the actual sensing. More specifically, weighting may be performed so that the weight becomes lighter as the pseudo 3D data is farther from the starting point (or farther from the detection target area).

The object detection device of one aspect of the present invention includes a storage means for storing 3D data of an object object as reference 3D data, a 3D data acquisition means for sensing real space to acquire measurement 3D data, and the measurement 3D data. The area specifying means for specifying the detection target area in the above, the pseudo 3D data adding means for adding pseudo 3D data to the missing area where the measurement 3D data for the target object is insufficient, the measurement 3D data, and the pseudo 3D. A detection processing means for detecting the target object by matching the query 3D data based on the data with the reference 3D data is provided , and the area identification unit is included in the measurement 3D data. With respect to a connecting region in which segments that are close to each other and adjacent to each other are connected, there is a difference in distance between the other connecting regions adjacent to the connecting region and a predetermined threshold or more at the boundary with the connecting region, and the distance is larger than that of the connecting region. When there is the other connecting region at a short distance, the connecting region is specified as a detection target region, and the other region is specified as the missing region .

Even with this configuration, even if the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then matching with the reference 3D data is performed. Therefore, occlusion, etc. Therefore, the accuracy of matching can be improved when the measured 3D data is insufficient, and thus the possibility of detecting the target object can be improved.

The object detection program of one aspect of the present invention is a 3D data acquisition step of sensing the real space and acquiring the measured 3D data in an information processing device provided with a storage means for storing the 3D data of the target object as reference 3D data. And the area identification step for specifying the detection target area in the measurement 3D data, the pseudo 3D data addition step for adding the pseudo 3D data to the missing area where the measurement 3D data for the target object is insufficient, and the measurement. and query 3D data based on the 3D data and the pseudo 3D data, by performing matching between the reference 3D data, a object detection program for executing the detection processing step of performing detection of the target object, wherein In the area identification step, in the measurement 3D data, the difference in distance at the boundary between the connection area and the connection area, which is another connection area adjacent to the connection area, with respect to the connection area in which the segments that are close to each other and are adjacent to each other are connected. Is equal to or greater than a predetermined threshold value, and when there is the other connecting region that is closer than the connecting region, the connecting region is specified as a detection target region, and the other region is specified as the missing region.

Even with this configuration, even if the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then matching with the reference 3D data is performed. Therefore, occlusion, etc. Therefore, the accuracy of matching can be improved when the measured 3D data is insufficient, and thus the possibility of detecting the target object can be improved.

The object detection method of one aspect of the present invention is an object detection method in an information processing apparatus provided with a storage means for storing 3D data of an object object as reference 3D data, and the information processing apparatus senses a real space. Pseudo 3D data in the 3D data acquisition step of acquiring the measurement 3D data, the area identification step of specifying the detection target area in the measurement 3D data, and the missing area where the measurement 3D data of the target object is insufficient. A detection processing step for detecting the target object by matching the measurement 3D data and the query 3D data based on the pseudo 3D data with the reference 3D data. , Is executed, and in the measurement 3D data, the connection region in which segments adjacent to each other are close to each other in distance is connected to another connection region adjacent to the connection region and with the connection region. When the difference in distance is equal to or greater than a predetermined threshold value and there is the other connection region closer than the connection region, the connection region is specified as a detection target region, and the other region is the missing region. Specify as an area .

Even with this configuration, even if the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then matching with the reference 3D data is performed. Therefore, occlusion, etc. Therefore, the accuracy of matching can be improved when the measured 3D data is insufficient, and thus the possibility of detecting the target object can be improved.

According to the present invention, even if the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then the matching with the reference 3D data is performed. It is possible to improve the accuracy of matching when the measured 3D data is insufficient due to the above, and thus the possibility of detecting the target object can be improved.

The figure which shows the usage mode of the object detection system of 1st Embodiment of this invention. A block diagram showing a configuration of an object detection system according to the first embodiment of the present invention. The figure which shows the example of the state which the object of 1st Embodiment of this invention is shielded in a shielding region. The figure which shows the example of acquisition of the 3D point cloud data of 1st Embodiment of this invention. The figure which shows the example of giving the pseudo-feature point of the 1st Embodiment of this invention. The figure which shows the example of the query 3D graph generated by the graph generation part of 1st Embodiment of this invention. The figure which shows the example of the generation of the average distribution of a plurality of object objects of the 1st Embodiment of this invention. Conceptual diagram of graph matching according to the first embodiment of the present invention A flow chart showing the operation of the object detection system according to the first embodiment of the present invention. A block diagram showing the configuration of the object detection system 100 according to the second embodiment of the present invention. An example of image data obtained by the RGB-D camera according to the second embodiment of the present invention. An example of visualizing 3D point cloud data together with the image data of the second embodiment of the present invention. The figure which shows the result of the detection of the front object of the 2nd Embodiment of this invention. An image of an object (person) cut out by performing graph cut processing on the detection frame obtained by the front object detection unit of the second embodiment of the present invention. Binarized image obtained by binarization of the second embodiment of the present invention Contour image generated from the binarized image of the second embodiment of the present invention The figure which shows the 3D key point extracted by 3DSIFT of the 2nd Embodiment of this invention. The figure which shows the bounding box estimated as the shielding space of the 2nd Embodiment of this invention. The figure which shows the example of the estimated shield space OS of the 2nd Embodiment of this invention. The figure explaining the addition of pseudo 3D data of the 2nd Embodiment of this invention. A flow chart showing the operation of the object detection system according to the second embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below show an example of the case where the present invention is carried out, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted.

(First Embodiment)
FIG. 1 is a diagram showing a usage mode of the object detection system according to the first embodiment of the present invention. In this embodiment, the object detection system is applied to a head mounted display (HMD). The HMD 100 is attached to the head of the user U and displays a hologram image in front of the user U. User U can also see the real space through the image.

The HMD 100 includes a camera as an image acquisition unit that captures an image of the real space and acquires image data, and a depth sensor as a distance measuring unit that measures the distance to the real space and acquires 3D point cloud data. .. The HMD 100 detects (recognizes or identifies) an object in the real space using image data and 3D point cloud data, generates a hologram image in association with the detected object, and displays the generated hologram image in front of the user U. indicate.

In the example of FIG. 1, objects O1 to O5 are present in the table in front of the user U. The user U wearing the HMD100 can see the real space including the objects O1 to O5 through the HDM100, and can see the image displayed in association with (a part or all of) the detected objects O1 to O5. it can. As such an HMD100, for example, HoloLens (registered trademark) manufactured by Microsoft Corporation can be used.

As shown in FIG. 1, when the user U extends the hand Uh, the hand Uh acts as a shield against the objects O1 to O5, and the back side of the hand Uh as seen from the camera or depth sensor of the HMD 100 is a shield region S. Become. In the example of FIG. 1, a part of the object O3 is shielded by the hand Uh, and the camera or depth sensor of the HMD 100 cannot obtain the image data or the 3D point group data of the object O3 for the shielded portion. The HMD 100 of the present embodiment solves the problem that an object cannot be detected or the detection accuracy is lowered due to such occlusion.

FIG. 2 is a block diagram showing the configuration of the object detection system 100 of the present embodiment configured as HDM. As described above, the object detection system 100 includes an image acquisition unit 1 that photographs the real space and acquires image data, and a distance measuring unit 2 that measures the distance to the real space and acquires 3D point group data. It has. The image acquisition unit 1 and the distance measuring unit 2 are provided adjacent to each other in the HMD and have substantially the same field of view.

The image acquisition unit 1 and the distance measurement unit 2 perform shooting and distance measurement at a predetermined frame rate (for example, 1/30 second), and output the acquired image data and 3D point cloud data to the synchronization unit 3. Each point data of this 3D point cloud data includes a plane coordinate value and a depth value (distance). That is, in the object detection system 100, as input data, a pixel value (RGB) is obtained by the image acquisition unit 1 and a depth value (D) is obtained by the distance measuring unit 2 for each coordinate in the field of view.

The synchronization unit 3 synchronizes the image data with the 3D point cloud data, sets the synchronized (acquired at the same timing) image data and the 3D point cloud data as a set, and sends the information processing unit 4 as input data. Output. The information processing unit 4 includes an area identification unit 41, a feature point extraction unit 42, a graph generation unit 43, a pseudo 3D data addition unit 44, and a detection processing unit 45.

The area identification unit 41 identifies an area (detection target area) for which object detection should be performed based on the image data and the 3D point cloud data. Specifically, the area specifying unit 41 performs segmentation (division into small areas) by a superpixel image using image data (see Non-Patent Document 1). More specifically, the area specifying unit 41 divides the tree into segments by dividing the tree in a timely manner in the process of constructing the minimum spanning tree in which the pixels of the image data are regarded as nodes.

At this time, the area specifying unit 41 excludes a small area whose distance is equal to or greater than a predetermined threshold value (for example, a distance beyond the reach of the user U) from the target based on the 3D point cloud data. As a result, the target object can be searched without searching for a useless area, and the processing time can be reduced. The region identification unit 41 further connects segments adjacent to each other whose depth values indicated by the 3D point cloud data obtained by the distance measuring unit 2 are close to each other to form a connected region. Generally, a plurality of connecting regions are specified from one frame.

As a result, when the certain object is not shielded, the area corresponding to the object is regarded as one connecting area. When a part of an object is shielded, the visible part is regarded as one connecting area, and the shielding part is regarded as another connecting area (shielding area). The visible region and the shielded region are in contact with each other.

The feature point extraction unit 42 extracts feature points from the 3D point cloud data for the region specified as the detection target region in the region identification unit 41. Specifically, the feature point extraction unit 42 extracts feature points by using a key point extraction method using observability (see Non-Patent Document 2), which is said to be robust to shielding. As a result, useful points in the 3D point cloud data of the target object can be treated as feature points. However, if there is a large shield that covers a large proportion of the target object, only a few feature points may be obtained.

Therefore, if there is a shield, the pseudo 3D data addition unit 44 scatters and arranges the pseudo feature points in the shielded area, and the detection processing unit 45 also uses the pseudo feature points to perform object detection. The identification of the shielded area S in the area specifying unit 41 and the addition of the pseudo feature points in the pseudo 3D data adding unit 44 will be described in detail below.

FIG. 3A is a diagram showing a state in which the object O3 is shielded in the shielding region S. In the example of FIG. 3A, the visible portion of the object O3 is specified as the detection target region K by the region identification unit 41. As shown in FIG. 3A, in this example, the object O3 is shielded in the shielding region S. In this case, as shown in FIG. 3B, the 3D point cloud data of the object O3 (Δ: invisible 3D point cloud data in FIG. 3B) that should have been originally obtained cannot be obtained in the shielded region S, and the object O3 Can obtain 3D point cloud data (◯ in FIG. 3B: visible 3D point cloud data) only from the unshielded detection target area K.

Therefore, first, the area specifying unit 41 specifies the shielding area S from the connecting area. The area specifying unit 41 is a connecting area A2 adjacent to the connecting area A1 with respect to a certain connecting area A1, and the difference in distance (depth value) at the boundary with the connecting area A1 is equal to or more than a predetermined threshold value. When there is a connection area A2 that is closer than the connection area A1 (has a small depth value), the connection area A1 is set as the detection target area K, and the connection area A2 is set as the shielding area S for the object in the connection area A1. Identify.

The pseudo 3D data addition unit 44 receives the identification result of the detection target area K and the shielding area S from the area identification unit 41, identifies the boundary B between the shielding area S and the detection target area K, and is a visible 3D close to the boundary B. The feature point of the point cloud data is selected as the starting feature point. In the example of FIG. 3C, two origin feature points are selected. The pseudo 3D data addition unit 44 arranges pseudo feature points on a plurality of spherical surfaces (scale spheres) having different radii centered on the origin feature point. The radius of the scale sphere may be set variably according to the size of the target object, and when there are a plurality of target objects, scale spheres of a plurality of sizes may be set.

For the detection target area K, which is the visible area, the visible 3D point cloud data of the object O3 is obtained, and if the pseudo feature points are arranged here, unnecessary information will be increased. Therefore, the pseudo 3D data addition unit The area where 44 places pseudo-feature points is a boundary where a point cloud having a similar depth value or a depth mass existing in the same superpixel has a remarkable depth change in which a shielding object is considered to be on the front side. The back side of the area. That is, if the cause is shielding, since there is some other object on the front side, the pseudo feature points are arranged only in the area on the back side of the object and on the front side of the sphere to be virtually arranged. The pseudo 3D data addition unit 44 arranges the pseudo feature points in the shielding area S, and does not arrange the pseudo feature points in the detection target area K where the visible 3D point cloud data for the object O3 is obtained.

FIG. 4 is a diagram showing an example of a query 3D graph generated by the graph generation unit 43. The graph generation unit 43 simulates using any of the feature points extracted from the detection target region, which is the visible region in the feature point extraction unit 42, in the region specified by the region identification unit 41, and any of those feature points as the starting feature point. A 3D graph (query 3D graph) is generated in which the pseudo feature points arranged by the 3D data addition unit 44 are set as query feature points and these query feature points are nodes. If there is no shielding, the graph generation unit 43 generates a query 3D graph using only the feature points extracted from the detection target area as query feature points.

The graph generation unit 43 sets a limit on the distance between the nodes according to the scale of the target object in the generation of the query 3D graph. This limit may be static (fixed) or dynamically set. For example, when the cup is the largest among the target objects, if the maximum length of the width, depth, and height of the cup is 15 cm, 30% of the width, 4.5 cm or less, should not be generated. And, do not generate an edge of 15 cm or more.

Further, the graph generation unit 43 weights each node according to the reliability (probability) in the generation of the query 3D graph. At this time, the graph generation unit 43 assigns a weight of 1.0 to the feature points in the detection target region and a weight of 1.0 or less to the pseudo feature points. Specifically, the graph generation unit 43 gives a smaller weight to the pseudo feature point as the distance from the starting feature point increases.

Instead of this, the graph generation unit 43 may give weights according to a Gaussian distribution centered on the origin feature point. Further, the graph generation unit 43 may use the average distribution of the target object instead. That is, when there are a plurality of target objects, the average field is calculated as shown in FIG. 5, the average distribution of the plurality of target objects is generated, and by applying this, each pseudo feature point is according to the average distribution. May be weighted.

The detection processing unit 45 uses the query 3D graph, the image data, and the 3D point group data to generate the query 3D graph generated by the graph generation unit 43 and the 3D graph of a plurality of target objects stored in the storage unit 5 (see). By performing graph matching with the 3D graph for reference), it is searched for which target object stored in the storage unit 5 the query 3D graph corresponds to the reference 3D graph, and the corresponding target object is detected.

FIG. 6 is a conceptual diagram of graph matching. As shown in FIG. 6, the storage unit 5 is labeled with a reference 3D graph of the target object generated in advance from CAD data which is a complete 3D model (in the example of FIG. 6, "computer monitor" and "notebook". , "Cup", "Brush stand", etc.).

Specifically, the detection processing unit 45 performs graph matching of a 3D graph by applying the graph matching technique described in Non-Patent Document 3 in a form of extending it to a 3D graph. That is, in the graph matching technique described in Non-Patent Document 3, the D information (depth information) of RGBD (image data and 3D point cloud data) is used only for coarse segmentation of the target object from the space. The detection processing unit 45 positively uses this depth information. Further, although a graph matching algorithm is described as a technique for 2D images in Non-Patent Document 4, it may be extended to 3D and applied to graph matching in the detection processing unit 45. Stable graph matching can be realized by combining the techniques of Non-Patent Documents 3 and 4.

The detection processing unit 45 detects a target object whose matching score (likelihood) is equal to or higher than a predetermined threshold value and has the maximum matching score, and outputs a label thereof. The detection processing unit 45 outputs information on the position and orientation of the detected target object together with this label. When there are a plurality of detected target objects, label, position, and posture information is output for each of the plurality of detected target objects.

The detection result image generation unit 6 generates a hologram image to be displayed on the display unit 7 by using the label, position, and orientation information of the target object detected by the detection processing unit 45. As described above, this image is a hologram image that the user U should superimpose and view in the real space, and is, for example, an image showing related information (for example, label information) about the detected target object. The display unit 7 displays the detection result image generated by the detection result image generation unit 6.

FIG. 7 is a flow chart showing the operation of the object detection system 100. The flow shown in FIG. 7 is repeated at a predetermined frame rate. First, the image acquisition unit 1 acquires image data by photographing the real space, and the distance measuring unit 2 acquires 3D point cloud data by measuring the distance in the real space (step S71).

The area specifying unit 41 identifies the detection target area using the image data and the 3D point cloud data (step S72). Specifically, as described above, the area identification unit 41 divides the image data into superpixels (small areas) and excludes superpixels whose distance is longer than a predetermined threshold value (depth value is larger than the threshold value). Then, adjacent superpixels with similar depth values are concatenated to form a concatenated area.

Then, when the difference between the depth values of the two adjacent connecting regions is equal to or greater than a predetermined threshold value, the region specifying unit 41 partially shields the connecting region on the back side (the side having the larger depth value). The detection target area K of the object is specified, and the connecting area on the front side (the side having a small depth value) is specified as the shielding area S.

The feature point extraction unit 42 extracts feature points from the detection target area K (step S73). The pseudo 3D data addition unit 44 determines whether or not there is a shielding region S (step S74). If there is a shielded area S specified by the area specifying unit 41 (YES in step S74), the pseudo 3D data adding unit 44 sets a feature point close to the shielded area S in the detection target area K as a starting feature point. Pseudo-feature points are placed on the spherical surface of the scale sphere (step S75). The pseudo feature points are arranged in the shielding area S and not in the detection target area K.

When there is no shielding (NO in step S74), the 3D graph generation unit 43 uses the feature points extracted by the feature point extraction unit 42 as the query feature points, and sets the query feature points as nodes in the query 3D graph. Generate (step S76). When the 3D graph generation unit 43 has a shield (YES in step S74), the pseudo feature points assigned by the pseudo 3D data addition unit 44 in step S75 and the detection target area K by the feature point extraction unit 42 A query 3D graph having the extracted feature points as a node is generated as a query feature points (step S76).

The detection processing unit 45 performs graph matching between the query 3D graph generated by the graph generation unit 43 and the reference 3D graph stored in the storage unit 5, and obtains a reference 3D graph corresponding to the query 3D graph. Search (step S77). The detection processing unit 45 identifies the label of the reference 3D graph having the maximum matching score (likelihood), and records the position and orientation thereof (step S78).

The detection result image generation unit 6 generates a hologram image related to the label recorded in step S78, determines a display position and a display angle according to the position and orientation, and generates a detection result image (step S79). .. The display unit 7 displays the detection result image generated in step S79 (step S80).

As described above, according to the object detection system 100 of the present embodiment, the detection processing unit 45 performs 3D graph matching based on the 3D point group data obtained by the distance measuring unit 2 measuring the distance in the real space. When the corresponding object is detected from the target objects stored in the storage unit 5, if 3D point group data of a part of the target object cannot be obtained even by distance measurement, pseudo 3D is performed. Pseudo 3D point group data (feature points) is added to the area by the data addition unit 44. Then, the graph generation unit 43 treats the pseudo 3D point group data in the same manner as the 3D point group data obtained by the distance measurement, and includes the 3D point group data obtained by the distance measurement and the pseudo 3D point group data. A query 3D graph is generated from the query 3D point group data, and the detection processing unit 45 matches the query 3D graph generated in this way with the reference 3D graph stored in the storage unit 5.

As a result, even when the image acquisition unit 1 or the distance measuring unit 2 cannot obtain all the image data or the 3D point cloud data of the target object due to shielding, the target object can be detected by graph matching.

In the above embodiment, the pseudo 3D data addition unit 44 arranges the pseudo feature points on the surface of the scale sphere centered on the starting point feature point in the shielding region S, but the method of arranging the pseudo feature points is this. Not limited to. The pseudo 3D data addition unit 44 may, for example, use the average distribution shown in FIG. 5 to probabilistically determine the amount and whether or not to disperse the pseudo feature points.

Further, in the above embodiment, an example in which the depth sensor is used in the distance measuring unit 2 to acquire the 3D point cloud data has been described, but instead, a stereo camera may be used, which is deeper than the image data. A method of generating a depth estimation image using learning (see Non-Patent Document 5) may be used, or a semantic segmentation technique (see Non-Patent Document 6) may be applied to limit the area to be searched and a processing system. And the processing time may be reduced.

Further, in the above embodiment, graph matching is used in the detection of the model-based object in the detection processing unit 45, but the object may be detected by matching other than graph matching. For example, in the above embodiment, the graph generation unit 43 generates a 3D graph having feature points as nodes, but instead of this, as query 3D data, SIFT (Scale-Invariant Feature Transform) for feature points, Local features such as SURF (Speeded Up Robust Features) are calculated, and the local features of the target object are stored in the storage unit 5 as reference 3D data, and the detection processing unit 45 uses the SVM (Support Vector Machine). The target object may be detected by performing matching.

Further, in the above embodiment, when a shielded area in which a part of the target object is shielded by the shielded object occurs as a missing area in which the 3D point cloud data of the target object is insufficient, such a shielded area is pseudo An example of assigning feature points has been described, but as described above, the missing area where the 3D point cloud data is insufficient is not limited to the shielded area. For example, even when a part of the target object has high gloss, sufficient 3D point cloud data may not be obtained for such a part. In this case, such a high-gloss region can be specified based on the brightness value of the image data, and pseudo feature points can be arranged.

Further, in the above embodiment, the object detection system 100 is configured as an HMD, and the result of the object detection in the detection processing unit 45 is used to generate the detection result image. However, the object detection system 100 of the embodiment of the present invention is used. Can be applied to other than HMD. For example, it is also possible to mount the object detection system 100 on a vehicle, detect a pedestrian, another vehicle, a sign, or the like as a target object, and configure the system to reflect the detection result in driving control.

Further, in the above embodiment, in the object detection system 100, all the components are mounted on one device called the HMD to form the object detection device, but some or all the components are dispersed. In addition, a communication network may be interposed between the components arranged in a distributed manner. Further, each component of the object detection system 100 may operate according to a computer program, and an object detection program that realizes and operates each component of the object detection system 100 by being executed by the CPU may be provided. That is, the HMD as the object detection device of the above embodiment may operate according to the object detection program.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described. The object detection system of the first embodiment and the object detection system of the second embodiment sense the real space to acquire the measurement 3D data, specify the detection target area in the measurement 3D data, and detect the detection target. Pseudo 3D data is added to the missing area adjacent to the area, and the query 3D data based on the measurement 3D data and the pseudo 3D data is matched with the reference 3D data stored in the storage means to perform the target object. It is common in that it detects.

In the following, the same configuration as that of the first embodiment will not be described in detail, and the applicable contents of the first embodiment will be applied to the second embodiment.

FIG. 8 is a block diagram showing the configuration of the object detection system 100 of the second embodiment. The object detection system 100 includes an RGB-D camera 11, an information processing unit 12, a storage unit 13, a detection result image generation unit 14, and a display unit 15. The information processing unit 12 includes a front object detection unit 21, a contour calculation unit 22, a feature point extraction unit 23, a shielding space estimation unit 24, a pseudo 3D data addition unit 25, and a detection processing unit 26.

The RGB-D camera 11 has a function as an image acquisition unit that captures an image in the real space and acquires image data, and a function as a distance measuring unit that measures the distance in the real space and acquires the distance measurement data. ing. The RGB-D camera 11 acquires image data and distance measurement data synchronized with each other by shooting. The information processing unit 12 is based on the input from the RGB-D camera 11, that is, the image data (RBG data) in the real space and the distance measurement data in the real space (also referred to as 3D point cloud data, depth data, or D data). Then, the object is detected by matching with the reference 3D data stored in the storage unit 13.

The storage unit 13 stores reference 3D data for a plurality of types of objects. As will be described later, in the present embodiment, since the information processing unit 12 performs FPFH matching between the query 3D data and the reference 3D data, the reference 3D data is also stored in the storage unit 13 in the form of the FPFH feature. ing. The detection result image generation unit 14 generates a detection result image by superimposing the detection result of the object in the information processing unit 12 on the image data obtained by the RBG-D camera 11. The display unit 15 displays the detection result image.

FIG. 9 is an example of image data obtained by the RGB-D camera 11, and FIG. 10 shows 3D point group data (also referred to as measurement 3D data) obtained by the RGB-D camera 11 together with the image data of FIG. Is an example of visualization.

In the examples of FIGS. 9 and 10, the cup is placed behind the human hand, and a part of the cup is hidden by the human hand. Similar to the first embodiment, the present embodiment is not detected by the object detection by the image such as CNN because the image data is not completely obtained like this cup, and the measurement 3D data. However, the object is detected by targeting an object including a missing area in which the measured 3D data is missing, particularly an object having a relatively large missing area.

The front object detection unit 21 of the information processing unit 12 detects an object by acquiring image data (RGB data) obtained by the RGB-D camera 11 and performing image recognition on the image data. For this object detection, for example, a technique based on a CNN (Convolutional Neural Network) such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector) can be used.

FIG. 11 is a diagram showing the result of object detection by the front object detection unit 21. In the example of FIG. 11, the front object detection unit 21 obtains identification results such as “tv (69%)”, “person (51%)”, and “keyboard (51%)” together with the detection frame. At this time, it is considered that the detection frame includes most of the object. In other words, when most of the object is visible in the image data, the object is detected by the front object detection unit 21, and the object that is mostly shielded (shielded object) such as a cup is the front object. It is not detected by the detection unit 21. Therefore, the foreground object detection unit 21 detects an object that is not shielded by other objects and most of which appears in the image data, that is, the foreground object.

The contour calculation unit 22 calculates the contour of the object detected by the front object detection unit 21. FIG. 12 is an image of an object (person) cut out by performing a graph cut process on the detection frame obtained by the front object detection unit 21. The contour calculation unit 22 distinguishes between the cut-out object area and the background area by setting the area of the object cut out by the graph cut as the first value and the background area as the second value. Perform digitization processing. FIG. 13 is a binarized image obtained by binarization. In the example of FIG. 13, the region of the cut-out object is white and the background is black.

The contour calculation unit 22 further generates a contour image by detecting the edge of the binarized image. FIG. 14 is a contour image generated from the binarized image. The contour calculation unit 22 of the present embodiment expands the edge portion of the binarized image to obtain a contour image, as shown in FIG. 14, in consideration of the fact that there is a shielded object around the contour. The expanded contour in this contour image is a region in which an object partially hidden by a foreground object exists, corresponds to a detection target region of the present invention, and is a contour calculation unit that generates such a contour image. Reference numeral 22 corresponds to the area specifying means of the present invention.

The feature point extraction unit 23 extracts 3D key points corresponding to the contour obtained by the contour calculation unit 22 from the 3D point cloud data (measurement 3D data) obtained by the RGB-D camera 11. The feature point extraction unit 23 of the present embodiment extracts 3DSIFT feature points by using 3DSIFT (Scale-Invariant Feature Transform) features as 3D key points.

FIG. 15 is a diagram showing 3D key points extracted by 3DSIFT. The feature point extraction unit 23 extracts the 3D key points corresponding to the expanded contours calculated by the contour calculation unit 22 from the 3D key points shown in FIG. As described above, in the contour calculation unit 22, the peripheral region of the foreground object is obtained as the contour image, and the image data obtained by the RGB-D camera 11 and the 3D point group data correspond to each other. By extracting the 3D key points corresponding to this contour image, the feature point extraction unit 23 can extract the 3D key points around the foreground object.

The shielded space estimation unit 24 estimates the shielded space for adding pseudo 3D data. The shielded space estimation unit 24 estimates the space including the 3D key points extracted from the periphery of the foreground object as the shielded space. Specifically, the shield space estimation unit 24 estimates a rectangular parallelepiped (bounding box) including all 3D key points around the foreground object extracted by the feature point extraction unit 23 as the shield space.

FIG. 16 is a diagram showing a bounding box estimated as a shielded space. In this example, the RGB-D camera 11 is in a posture in which the optical axis is horizontal, and it is assumed that the shielded object is placed on a horizontal plane. With the RGB-D camera 11 as the center, the optical axis direction is the X3 direction, the X2 direction is defined vertically downward, and the direction perpendicular to the X3 direction and the X2 direction is X1.

Of the 3D key points extracted by the feature point extraction unit 22, the maximum and minimum values for each axis of X1 to X3 are maxX1, maxX2, maxX3, minX1, minX2, and minX3, respectively, and these maxX1 and maxX2. , MaxX3, minX1, minX2, minX3 define a rectangular parallelepiped as a shielded space OS. FIG. 17 is a diagram showing an example of an estimated shielded space OS.

By setting the rectangular parallelepiped including the 3D key points extracted from the periphery of the foreground object as the shielding space in this way, the shielding space OS is located on the back side of the foreground object, that is, the area where the 3D data of the shielded object is missing. (Missing area) will be included. Therefore, by adding pseudo 3D data to this shielded space OS, it is possible to add pseudo 3D data to the missing area. The 3D key points around the foreground object extracted by the feature point extraction unit 23 are used only for estimating the shield space OS in the shield space estimation unit 24, and the FPFH feature is used at the time of matching as described later. ..

The pseudo 3D data addition unit 25 adds pseudo 3D data to the shielded space. This pseudo 3D data is artificially given as a point on the shielded object. The pseudo 3D data addition unit 25 sets a plurality of virtual planes parallel to the X1X2 plane and arranged at predetermined intervals in the X3 direction in the shielded space OS, and distributes the pseudo 3D data on the virtual planes. ..

A shielded object that is largely shielded is likely to have its center of gravity also shielded. Therefore, the pseudo 3D data addition unit 25 distributes the pseudo 3D data on the ellipse on each virtual plane. The ellipse is adopted because the existence probability of the surface of the shielded object is considered to decrease radially from the center of the shielded space.

FIG. 18 is a diagram illustrating the addition of pseudo 3D data. In the example of FIG. 18, as shown in the upper part, it is assumed that most of the right side of the cup, which is a shielded object, is shielded. The lower left of FIG. 18 is the true 3D data of the cup, a part of which is measured and the other part is shielded by the foreground object. The lower center of FIG. 18 shows that the pseudo 3D data is distributed on a plurality of concentric ellipses centered on the center of the shielded space. As shown on the lower right of FIG. 18, the pseudo 3D data addition unit 25 is placed on a plurality of concentric ellipses on a plurality of concentric ellipses in each of the plurality of virtual planes P1 to PH set at intervals in the X3 direction in the shielded space. Give data.

ここで、

である。 The pseudo 3D data addition unit 25 assigns pseudo 3D data by the following equation (1).

here,

Is.

Further, k is a position in the radial direction, m is the density of pseudo 3D data, and h is the interval of virtual planes arranged at equal intervals. Further, as shown in FIG. 18, K is an interval in the circumferential direction of the ellipse, M is an interval in the radial direction, and H is the number of virtual planes in the depth direction (X3 direction). By adjusting M and K, the number of pseudo 3D data can be adapted to the number of measured 3D data after downsampling. In this way, the pseudo 3D data addition unit 25 assigns H × K × M pseudo 3D data.

The detection processing unit 26 matches the measurement 3D data measured in the visible region with the reference 3D data stored in the storage unit 13 by using the pseudo 3D data given to the shield region. For this matching, a matching method such as PFH (Point Feature Histogram), FPFH (Fast Point Feature Histogram), SHOT (Signature of Histograms of OrienTations) can be used.

Specifically, the detection processing unit 26 extracts the FPFH feature (pseudo FPFH feature) from the pseudo 3D data given by the pseudo 3D data addition unit 25 and the measurement 3D data in the vicinity of the pseudo 3D data, and queries this. 3D data is used, and this query 3D data is matched by FPFH. As a result, the likelihood lowered due to the missing area is raised, and normal detection becomes easier. In particular, the feature description can be improved in the subsequent stage of the matching process.

The detection result image generation unit 14 obtains the center of gravity of the target object detected by the detection processing unit 26, and superimposes an annotation indicating the position of the center of gravity on the RGB-D data acquired by the RGB-D camera 11. Generate a screen. The display unit 15 displays the detection result image generated by the detection result image generation unit 14.

FIG. 19 is a flow chart showing the operation of the object detection system 100. The flow shown in FIG. 19 is repeated at a predetermined frame rate. First, the RGB-D camera 11 acquires RGB-D data, that is, image data and 3D point cloud data by photographing the real space (step S91).

The front object detection unit 21 detects the front object from the image data by YOLO (step S92). Then, the contour calculation unit 22 performs graph cutting processing on the detected foreground object, cuts out the foreground object, binarizes the cut out foreground object, and detects the edge of the binarized image to obtain the contour image. Generate (step S93). At this time, as described above, the contour calculation unit 22 calculates a contour having a certain width by expanding the edge of the binary image.

Next, the feature point extraction unit 23 extracts 3D key points corresponding to the contour positions from the 3D point cloud data obtained by the RGB-D camera 11 (step S94). The shielded space estimation unit 24 obtains a rectangular parallelepiped including all the extracted 3D key points, and estimates this rectangular parallelepiped as the shielded space OS (step S95).

The pseudo 3D data addition unit 25 sets a plurality of virtual planes in the shielded space, and adds pseudo 3D data in each virtual plane (step S96). At this time, in each virtual plane, pseudo 3D data is added on a plurality of concentric ellipses.

When the pseudo 3D data is added to the shielded space OS in this way, the detection processing unit 26 extracts the FPFH feature from the measurement 3D data in the visible region obtained by the RGB-D camera 11, and also extracts the pseudo 3D data. Is used as a pseudo FPFH feature, and these FPFH features and pseudo FPFH features are used to perform matching by FPFH to detect a shielded object (step S97).

The detection result screen generation unit 14 calculates the position of the center of gravity of the detected object to be shielded (step S98) and generates a detection result screen (step S99).

As described above, according to the object detection system 100 of the second embodiment of the present invention, the object detected from the image data is set as the front object, and the shielding area is set on the back side thereof to add the pseudo 3D data. Therefore, even for a shielded object that is not detected from the image data, the object can be detected by the model-based three-dimensional object detection.

In the above embodiment, the foreground object is cut out and binarized, the edge is detected to generate an expanded contour, the 3D key point corresponding to the contour is extracted, and the extracted 3D key point is used. The rectangular parallelepiped to be included is used as a shielding space, but the method of setting a shielding space for adding pseudo 3D data to the back side of the front object is not limited to this. For example, a rectangular parallelepiped extending from the detection frame when the foreground object is detected in the optical axis direction (X3 direction) of the RGB-D camera 11 by a predetermined length may be used as the shielding space. In this case, it is not necessary to calculate the expanded contour (that is, the detection target area). Further, the shielded space may have a shape other than a rectangular parallelepiped.

Further, in the above embodiment, the result of object detection is used for superimposing annotations, but the application example of the result of object detection is not limited to this, and the result of object detection is, for example, a starting point when tracking. Various applications such as recognition of and picking in robots are possible.

In the present invention, even when the measurement 3D data of the target object is insufficient, the pseudo 3D data is added to the measurement 3D data, and then the matching with the reference 3D data is performed. It is possible to improve the accuracy of matching when the measurement 3D data is insufficient, thereby improving the possibility of detecting the target object, and it is useful as an object detection system or the like that detects the target object by sensing the real space.

100 Object Detection System (HMD)
1 Image acquisition unit 2 Distance measurement unit 3 Synchronization unit 4 Information processing unit 41 Area identification unit 42 Feature point extraction unit 43 Graph generation unit 44 Pseudo 3D data addition unit 45 Detection processing unit 5 Storage unit 6 Detection result image generation unit 7 Display unit 11 RGB-D camera 12 Information processing unit 13 Storage unit 14 Detection result image generation unit 15 Display unit 21 Front object detection unit 22 Contour calculation unit
23 Feature point extraction unit 24 Shielding space estimation unit 25 Pseudo 3D data addition unit 26 Detection processing unit K Detection target area S Shielding area OS Shielding space

Claims (14)

A storage means for storing 3D data of the target object as reference 3D data,
A 3D data acquisition means that senses the real space and acquires measurement 3D data,
An area specifying means for specifying a detection target area in the measurement 3D data, and
Pseudo 3D data adding means for adding pseudo 3D data to a missing area where measurement 3D data for the target object is insufficient, and
It is provided with a detection processing means for detecting the target object by matching the query 3D data based on the measurement 3D data and the pseudo 3D data with the reference 3D data.
The area identification part is
In the measurement 3D data, with respect to a connecting region in which segments adjacent to each other are close to each other, the difference in distance is equal to or greater than a predetermined threshold value in another connecting region adjacent to the connecting region and at the boundary with the connecting region. When there is the other connecting region that is closer than the connecting region, the connecting region is specified as the detection target region, and the other region is specified as the missing region.
An object detection system characterized by this. Further equipped with a graph generation means for generating a graph using 3D data as a node,
The 3D data acquisition means includes a 3D point cloud data acquisition means that acquires 3D point cloud data in the real space by sensing, and a feature point extraction means that extracts feature points as measurement 3D data from the 3D point cloud data. Including
The storage means stores a graph of feature points of the target object as the reference 3D data, and stores the graph.
The pseudo 3D data adding means adds pseudo feature points as the pseudo 3D data.
The graph generation means generates a query graph having the measurement 3D data and the pseudo 3D data as nodes.
The detection processing means uses the query graph as the query 3D data and performs graph matching with the reference 3D data.
The object detection system according to claim 1. An image acquisition means for photographing the real space and acquiring an image,
An object detection means for detecting a foreground object from the image,
An estimation means for estimating the missing region based on the detection result of the foreground object is further provided.
The pseudo 3D data adding means adds the pseudo 3D data to the missing area estimated by the estimating means.
The object detection system according to claim 1. Further provided with a contour calculation means for cutting out the foreground object by a graph cut process and calculating a contour in which the edge of the cut out portion is expanded.
The estimation means estimates the feature points extracted from the measurement 3D data, and a rectangular parallelepiped including the feature points corresponding to the contour as the missing region.
The object detection system according to claim 3. The pseudo 3D data adding means sets a plurality of virtual planes arranged in the depth direction in the missing area, and imparts the pseudo 3D data on a plurality of concentric ellipses in the virtual plane.
The object detection system according to claim 4. Further provided with an image acquisition means for photographing the real space and acquiring an image,
The 3D data acquisition means includes a 3D point cloud data acquisition unit that acquires 3D point cloud data in the real space by sensing.
The area specifying means identifies the detection target area based on the segmentation information using the image synchronized with the 3D point cloud data.
The object detection system according to claim 1. The object detection system according to claim 5 , wherein the area specifying means specifies the detection target area by excluding a area having a distance within a predetermined range from the 3D point cloud data. The 3D data acquisition means includes a 3D point cloud data acquisition unit that acquires 3D point cloud data in the real space by sensing.
The region specifying means identifies a shielded region that shields the target object as the missing region based on the 3D point cloud data.
The object detection system according to claim 1. The area specifying means is a region adjacent to the detection target region, the distance of the 3D point group data is shorter than the distance of the 3D point group data of the detection target region, and the distance of the 3D point group data. The object detection system according to claim 8 , wherein the missing area is specified by using a region in which the difference between the detection target region and the distance from the 3D point group data is equal to or greater than a threshold value as the shielding region. The object detection system according to claim 8 or 9 , wherein the pseudo 3D data adding means applies the pseudo 3D data on a spherical surface centered on a starting point in the detection target region. The object detection system according to any one of claims 1 to 10 , wherein the query 3D data is weighted so that the weight of the pseudo 3D data is lighter than that of the measured 3D data. A storage means for storing 3D data of the target object as reference 3D data,
A 3D data acquisition means that senses the real space and acquires measurement 3D data,
An area specifying means for specifying a detection target area in the measurement 3D data, and
Pseudo 3D data adding means for adding pseudo 3D data to a missing area where measurement 3D data for the target object is insufficient, and
It is provided with a detection processing means for detecting the target object by matching the query 3D data based on the measurement 3D data and the pseudo 3D data with the reference 3D data.
The area identification part is
In the measurement 3D data, with respect to a connecting region in which segments adjacent to each other are close to each other, the difference in distance is equal to or greater than a predetermined threshold value in another connecting region adjacent to the connecting region and at the boundary with the connecting region. When there is the other connecting region that is closer than the connecting region, the connecting region is specified as the detection target region, and the other region is specified as the missing region.
An object detection device characterized by the fact that. An information processing device equipped with a storage means for storing 3D data of a target object as reference 3D data.
A 3D data acquisition step that senses the real space and acquires measured 3D data,
An area identification step for specifying a detection target area in the measurement 3D data, and
A pseudo 3D data addition step of adding pseudo 3D data to a missing area where measurement 3D data of the target object is insufficient, and
A detection processing step for detecting the target object by matching the query 3D data based on the measurement 3D data and the pseudo 3D data with the reference 3D data.
A object detection program for executing,
The area identification step
In the measurement 3D data, with respect to a connecting region in which segments adjacent to each other are close to each other, the difference in distance is equal to or greater than a predetermined threshold value in another connecting region adjacent to the connecting region and at the boundary with the connecting region. When there is the other connecting region that is closer than the connecting region, the connecting region is specified as the detection target region, and the other region is specified as the missing region.
An object detection program characterized by this. It is an object detection method in an information processing device provided with a storage means for storing 3D data of a target object as reference 3D data.
The information processing device
A 3D data acquisition step that senses the real space and acquires measured 3D data,
An area identification step for specifying a detection target area in the measurement 3D data, and
A pseudo 3D data addition step of adding pseudo 3D data to a missing area where measurement 3D data of the target object is insufficient, and
The detection processing step of detecting the target object by matching the query 3D data based on the measurement 3D data and the pseudo 3D data with the reference 3D data is executed.
The area identification step
In the measurement 3D data, with respect to a connecting region in which segments adjacent to each other are close to each other, the difference in distance is equal to or greater than a predetermined threshold value in another connecting region adjacent to the connecting region and at the boundary with the connecting region. When there is the other connecting region that is closer than the connecting region, the connecting region is specified as the detection target region, and the other region is specified as the missing region.
Object detection method.

Images