JP6742157B2 - Object detection device, object detection system, object detection method, and program - Google Patents

Description

The present invention relates to an object detection device that detects an object, an object detection system, an object detection method, and a program.

A sensor called a lidar (LIDAR), which detects surrounding objects in real time with high accuracy, has been attracting attention. The lidar is one of the remote sensing techniques using light, and determines the distance to the object by the time until the measurement light is emitted and the light reflected by the object is received. By scanning the light emission direction, it is possible to detect the surrounding objects. Such sensors are used to detect monitored objects such as vehicles and people.

For example, when an object that is different from the monitored object, such as a small insect or a fallen leaf, blocks the measurement light, the reflected light from the monitored object cannot be obtained due to the obstacle, and the monitored object cannot be detected properly. was there.

Patent Document 1 discloses a distance measuring device that solves the above problems. The distance measuring device described in Patent Document 1 generates a group in which a plurality of distance data are collected when distance data indicating that the area is within the monitoring target area exists in a plurality of adjacent scanning angle directions, and the group is formed. The size of the object along the scanning direction is defined by the number of distance data included in or the maximum scanning angle. Then, if the size of the object is equal to or larger than the number indicating the object detection minimum width or the scanning angle, it is determined that there is an object to be monitored, and if not, it is determined to be noise.

JP2012-242189A

It is an object of the present invention to provide an object detection device capable of appropriately detecting an object by reducing the influence of noise sources such as minute insects and fallen leaves.

The object detection system of the present invention includes a distance measuring device and an object detection device. The object detection device, a distance image acquisition unit that acquires a distance image having distance measurement data in each distance measurement direction of the distance measurement device as a pixel value, a non-background pixel extraction unit that extracts a non-background pixel from the distance image, The relative distance between two points in the space corresponding to two adjacent non-background pixels is calculated based on the distance measurement direction of the non-background pixel and the distance measurement data, and the adjacent non-pixels are calculated based on the relative distance. A first clustering unit that generates an object candidate cluster that is a set of the non-background pixels by repeatedly performing a process of determining whether or not the background pixels belong to the same cluster, and a distance image whose acquisition times are adjacent to each other. A set of the object candidate clusters is obtained by repeatedly performing a process of determining whether or not two object candidate clusters to be determined belong to the same cluster based on the relative distance in space of the determined object candidate clusters. A second clustering unit that generates a multi-layered cluster, and an object that detects one or more object candidate clusters in each range image included in the multi-layered cluster as an object when the multi-layered cluster satisfies a predetermined condition. And a detector.

As described above, the present invention clusters the non-background pixels extracted from the distance image to generate an object candidate cluster that is a set of object candidate pixels. An object candidate cluster is a group of pixels whose relative distances between corresponding points in space are short, and each pixel forming the object candidate cluster is considered to correspond to an object or a part of an object. In this specification, the distance between corresponding points in space with respect to the distance from the distance measuring device to the object is referred to as “relative distance”.

Then, the present invention generates a multi-layered cluster by clustering in the time direction a plurality of object candidate clusters generated from a plurality of range images having different acquisition times. The object candidate clusters included in the multi-layered cluster are considered to be a group of object candidate clusters that move over time.

When the multi-layered cluster satisfies a predetermined condition, one or a plurality of object candidate clusters in each range image included in the multi-layered cluster are detected as an object, so that the range image at a certain time contains noise. However, the object is not easily affected by the noise, and the object can be appropriately detected. For example, when the object candidate cluster B1 generated from the distance image at time t−1 and the object candidate clusters B2 and B3 generated from the distance image at time t form a multi-layered cluster and satisfy a predetermined condition, the object The candidate cluster B1 is detected as an object, and the object candidate clusters B2 and B3 are detected as the same object as the object candidate cluster B1. Thus, at time t, even if the object candidate clusters B2 and B3 are divided by noise, the object candidate clusters B2 and B3 can be appropriately determined as one object.

In addition, the first clustering unit of the present invention calculates the relative distance between two points in the space corresponding to two adjacent non-background pixels as a pixel value (ranging distance) without converting the non-background pixels into three-dimensional space coordinates. Direction and its distance measurement data), and clustering is performed based on the relative distance. In this way, by calculating the relative distance from the pixel value instead of converting it into the three-dimensional space coordinates corresponding to the pixel value, it is possible to perform appropriate clustering regardless of the perspective to the point in the space corresponding to the non-background pixel. It can be performed.

In addition, since clustering is performed by repeatedly performing the process of determining whether adjacent non-background pixels belong to the same cluster, the amount of calculation of the clustering process can be suppressed. Note that the second clustering unit can also perform appropriate clustering while suppressing the amount of calculation, similarly to the first clustering unit.

In the object detection device of the present invention, the second clustering unit weights the relative distance in space of the object candidate cluster to be determined according to the time interval of the acquisition time of the distance image that generated the object candidate cluster. The object candidate clusters may be clustered based on the weighted distance.

When the time intervals of the acquisition times of the range images are different, the distances that the object moves during the time intervals are different, but according to the configuration of the present invention, even when the time intervals of the acquisition times of the range images are different. , It is possible to appropriately generate a multi-layered cluster according to the interval of acquisition time.

In the object detection device of the present invention, the distance measuring device is a distance measuring device that performs two-dimensional scanning in a main direction and a sub direction, and the first clustering unit includes a plurality of distance images corresponding to the main direction. And a main direction clustering unit that generates a line segment by performing clustering of the non-background pixels in each divided region, and a relative distance in space of the line segment generated in an adjacent region. And a sub-direction clustering unit that clusters the line segments to generate the object candidate clusters.

By performing clustering in each of the main direction and the sub-direction in this way, it is possible to generate a three-dimensional object candidate cluster while suppressing the amount of clustering calculation.

In the second clustering unit, the process of obtaining the relative distance between the two object candidate clusters to be determined is performed by calculating a relative distance between pixels having a common sub-direction in the two object candidate clusters. A first process for obtaining the minimum relative distance among combinations of pixels having a common sub-direction, and a minimum relative distance in each direction obtained by performing the first process while changing the sub-direction. A second process of obtaining the smallest minimum relative distance among the relative distances as the relative distance between the two object candidate clusters may be performed. As described above, the calculation amount can be significantly reduced by not considering the relative distance due to the combination of pixels in different sub-directions.

In the object detection device of the present invention, the object detection unit may detect an object based on the size of the object candidate cluster included in the multilayer cluster and the number of pixels included in the multilayer cluster.

With this configuration, an object can be detected by separating the object to be detected from noise. For example, even if a noise source such as a flying insect is used as an object candidate cluster and a multi-layered cluster is generated, it can be excluded depending on the size of the object candidate cluster and the number of pixels included in the multi-layered cluster.

The object detection device of the present invention may include an object movement detection unit that obtains a movement direction and a movement speed of the object based on the transition of the position of the object in each distance image detected from the multilayer cluster. In addition, the object movement detection unit converts the pixel value of the detected object into position data of a three-dimensional space coordinate system to specify a space occupied by the object in the three-dimensional space, and determines a moving direction and a moving speed of the object. You may ask.

With this configuration, the moving direction and the moving speed of the object can be accurately obtained based on the information of the object detected by reducing the influence of noise.

The object detection device of the present invention may include an object tracking unit that detects the same object from the objects detected at different times based on the moving direction and the moving speed of the object and tracks the object. ..

Since the moving direction and moving speed of an object often do not change significantly in a short time, the same object as the previously detected object can be detected and the object can be tracked by the configuration of the present invention.

The object detection device of the present invention, based on the moving direction and the moving speed of the object detected by the object detection unit, estimates the range in which the object exists after a lapse of a predetermined time, the range as the search range of the object A search range setting unit for setting is provided, and the object tracking unit determines that the object detected from the search range after the predetermined time has passed is the same object as the object detected at the previous time. Good.

By setting the search range estimated to be the moving destination of the object in this way, it is possible to improve the tracking accuracy of the same object.

An object detection method of the present invention is a method for detecting an object by an object detection device based on a distance image having distance measurement data in each distance measurement direction of a distance measurement device as a pixel value, wherein the object detection device comprises: Obtaining the range image, the object detecting device extracting non-background pixels from the range image, and the object detecting device having two points in a space corresponding to two adjacent non-background pixels. Repeat the process of calculating the relative distance between the non-background pixels based on the distance measurement direction and the distance measurement data, and determining whether the adjacent non-background pixels belong to the same cluster based on the relative distance. By performing the step of generating an object candidate cluster that is a set of non-background pixels, the object detection apparatus, the acquisition time to the relative distance in space of the object candidate cluster respectively generated from the adjacent distance image. A step of generating a multi-layered cluster that is a set of the object candidate clusters by repeatedly performing a process of determining whether or not two object candidate clusters to be determined belong to the same cluster based on the above; However, when the multi-layered cluster satisfies a predetermined condition, one or a plurality of object candidate clusters in each range image included in the multi-layered cluster are detected as an object.

A program of the present invention is a program for detecting an object based on a distance image having distance measurement data in each distance measurement direction of a distance measuring device as a pixel value, and a step of acquiring the distance image in a computer. A step of extracting a non-background pixel from the distance image, and a relative distance between two points in a space corresponding to two adjacent non-background pixels, based on a ranging direction of the non-background pixel and the ranging data. A step of generating an object candidate cluster, which is a set of the non-background pixels, by repeatedly performing a process of determining whether or not the adjacent non-background pixels belong to the same cluster based on the relative distance. , The process of determining whether two object candidate clusters to be determined belong to the same cluster is repeatedly performed based on the relative distance in space of the object candidate clusters that are respectively generated from the distance images whose acquisition times are adjacent to each other. Thereby generating a multi-layered cluster which is a set of the object candidate clusters, and when the multi-layered cluster satisfies a predetermined condition, one or more objects in each distance image included in the multi-layered cluster. Detecting the candidate cluster as an object.

According to the present invention, it is possible to appropriately detect an object by reducing the influence of noise from the range image acquired from the range finder.

It is a figure which shows the structure of the object detection system of 1st Embodiment. It is a figure for demonstrating the scanning angle of a range finder. It is a figure which shows the example which the person who is a detection target moves. It is a figure explaining the ranging data obtained by the ranging device. It is a figure which shows the example of an object candidate cluster and a multilayer cluster. It is a figure explaining the method of calculating the relative distance between pixels. It is a figure which shows the output example of the detection result of the object detection system of 1st Embodiment. It is a flow chart which shows operation which detects an object with an object detection device. It is a flow chart which shows processing by the 1st clustering part. It is a flow chart which shows processing by the 2nd clustering part. It is a figure which shows the process in which a multilayer cluster is produced|generated by the process of a 2nd clustering. It is a figure which shows the process in which a multilayer cluster is produced|generated by the process of a 2nd clustering. It is a figure which shows the structure of the object detection system of 2nd Embodiment. It is a figure which shows the example which divided the distance image into the main direction clustering unit, and the reading order of the pixel in a main direction clustering unit. It is a figure for demonstrating the method the 2nd clustering part of 2nd Embodiment calculates|requires the distance of object candidate clusters. It is a figure which shows the output example of the detection result of the object detection system of 2nd Embodiment. It is a figure which shows the structure of the object detection system of 3rd Embodiment. It is a figure which shows the example which calculates|requires the moving direction and moving speed of an object. It is a figure for explaining processing of a search range setting part. It is a figure which shows the method of calculating|requiring the object of a tracking target from the object detected in the search range. It is a figure for explaining processing when a plurality of search ranges overlap.

Hereinafter, an object detection system according to an embodiment of the present invention will be specifically described with reference to the drawings. In the present embodiment, an object detection device that detects a person who enters a predetermined object detection area will be described as an example.

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of an object detection system 1 according to the first embodiment of the present invention. The object detection system 1 includes a distance measuring device 10 and an object detection device 20.

[Distance measuring device]
The distance measuring device 10 irradiates the measurement light and calculates the distance to the object in the irradiation direction based on the reflected light from the object in the irradiation direction. The distance measuring device 10 is a two-dimensional scanning type distance measuring device, and scans the three-dimensional space in the two-dimensional direction in the distance measuring direction. The ranging device 10 calculates the distance to the object in each ranging direction. As a result, a distance image having the distance measurement data in each distance measurement direction as pixel values can be obtained.

The range image is a two-dimensional image, and one index (main direction) in the scanning direction is ims1 and the other index (sub direction) is ims2. When the number of pixels in the main direction and the number of pixels in the sub direction of the range image are Nms1 and Nms2, respectively, the index ims1 can take a value of 1 to Nms1 and the index ims2 can take a value of 1 to Nms2.

2A and 2B are diagrams for explaining the distance measuring device 10 in detail. It is desirable that the distance measuring device 10 be installed above the object detection area so as to avoid a shield in the object detection area as much as possible. Then, the distance measuring device 10 of the present embodiment has a function of performing scanning in the horizontal direction and the vertical direction as the two-dimensional direction. That is, the distance measuring device 10 further scans the scanning system that scans in the horizontal direction in the vertical direction. As shown in FIG. 2A, the horizontal distance measuring angle θms1 can be −95 to +95 degrees with the front being 0 degrees. As shown in FIG. 2B, the distance measurement angle θms2 in the vertical direction can be 0 to −90 degrees with 0° in the horizontal direction. The set of distance measurement angles (θms1, θms2) indicates the direction of distance measurement.

The distance measuring device 10 irradiates the measuring light in the direction indicated by a certain set of distance measuring angles (θms1, θms2). Then, the distance measuring device 10 calculates the distance R between the object in this direction and the distance measuring device 10 based on the reflected light. Then, the distance measuring device 10 sets the calculated distance R as the value of the pixel (ims1, ims2) in the distance image corresponding to the set of the distance measuring angles (θms1, θms2). Hereinafter, the distance (data) indicated by the pixels (ims1, ims2) in the distance image will be expressed as R(ims1, ims2). The distance measuring device 10 discretely scans the scanning range shown in FIG. 2 in order to obtain distance data in a set of distance measuring angles.

Note that the correspondence between a set of distance measurement angles (θms1, θms2) and the pixels (ims1, ims2) in the distance image is uniquely determined in advance. Conversely, a set of distance measurement angles (θms1, θms2) is determined for the pixels (ims1, ims2) in the distance image.

A known method can be applied as the distance measuring method. For example, the TOF (Time of Flight) method described in Japanese Patent No. 4837413 irradiates a pulsed measurement light toward an object detection area, and based on the time difference between the irradiation time and the reflected light detection time, The distance is calculated. Further, the AM (Amplitude Modulation) method described in Japanese Patent No. 4703830 irradiates an object detection area with measurement light whose amplitude is modulated, and based on the phase difference between the measurement light and the reflected light, the distance to the object. Is calculated. The configuration of the distance measuring device 10, the main and sub scanning patterns, and the distance measuring method are not particularly limited, and may be different from the examples described above.

The configuration of the distance measuring device 10 has been described above. The distance measuring device 10 is a device capable of scanning in two directions, that is, a horizontal direction and a vertical direction. However, in the first embodiment, the object detecting device 20 is scanned only in the horizontal direction and a one-dimensional distance image is displayed. An example of inputting data will be described, and an example of performing scanning in the horizontal direction and the vertical direction to input data of a two-dimensional range image to the object detection device 20 in the second embodiment will be described.

[Outline of object detection]
Before entering a detailed description of the object detection device 20, an outline of object detection by the object detection device 20 of the present embodiment will be described. Here, in order to understand the outline of object detection, an example in which only one-dimensional scanning in the horizontal direction is performed will be described, but it is of course applicable to the case of performing two-dimensional scanning.

FIG. 3A illustrates an example in which a person who is a detection target moves from time t-4 to time t. When the distance measuring device 10 is scanned in the horizontal direction to measure the distance, the distance R to the object is obtained for each angle θms1 as shown in FIG. 3B. The scanning by the distance measuring device 10 is sufficiently faster than the movement of the person, and the scanning in the horizontal direction is completed at each time.

The mark "x" described in FIG. 3B indicates that noise is included in the distance measurement at time t-2. That is, when a distance measurement image is acquired at time t-2, a noise source such as an insect or a fallen leaf is interposed between the distance measurement device 10 and the person, and it is impossible to measure the distance of a part of the person. Showing.

FIG. 4 is a diagram illustrating distance measurement data obtained by the distance measuring device 10. The distance measurement image acquired at each time from time t-4 to time t has distance data for each distance measurement direction. Specifically, the data of the distance R(ims1) is associated with the index ims1 indicating the distance measuring direction.

In FIG. 4, shaded pixels are non-background pixels, and non-shaded pixels are background pixels. By storing background distance data in the object detection area in advance, subtracting the background data from the observed distance measurement data, and extracting pixels with a large difference, non-background pixels can be obtained.

In FIG. 4, the shaded type indicates that the data is obtained from the same object. Here, for convenience of explanation, different objects are shaded, but in reality, it is not known whether the objects are the same or different unless the objects are detected. In FIG. 4, diagonal hatching represents the distance R to the person shown in FIGS. 3A and 3B. The pixel N1 at time t-2 is distance measurement data from an object different from the left and right pixels, but this is because a noise source is present between the person and the distance measuring device 10 (that is, in front of the person). It is caused by intervening and blocking the measuring light. The object detection according to the present embodiment appropriately detects a person even if there is such a noise source.

In the present embodiment, for each distance image acquired at each time, non-background pixels included in the distance image are clustered to generate an object candidate cluster. The clustering here is performed by sequentially determining whether or not adjacent non-background pixels are included in the same cluster. When the relative distance between two points in the space corresponding to two adjacent non-background pixels is less than or equal to a predetermined threshold value, it is determined that the two adjacent pixels are included in the same cluster, and the relative distance is greater than the predetermined threshold value. If they are larger, it is determined that they are not included in the same cluster.

FIG. 5A is a diagram showing an example of object candidate clusters. An object candidate cluster B4 is generated in the distance image at time t-4, an object candidate cluster B3 is generated in the distance image at time t-3, and an object candidate cluster B0 is generated in the distance image at time t. In the distance image at time t-2, it is determined that the pixel N1 and the left and right non-background pixels are not the same cluster, and two object candidate clusters B21 and B22 are generated. In the distance image at time t−1, since there are non-background pixels N2 and N3 in addition to the person, the object candidate cluster B11 composed of these two non-background pixels N2 and N3 was generated.

Next, the object candidate clusters are clustered in the time direction to generate a multi-layered cluster. The clustering here is performed by sequentially determining whether or not respective object candidate clusters obtained from two distance images whose acquisition times are adjacent to each other are included in the same cluster. When the relative distance between two object candidate clusters is less than or equal to a predetermined threshold value, it is determined that adjacent object candidate clusters are included in the same cluster, and when the relative distance is greater than the predetermined threshold value, they are not included in the same cluster. judge.

FIG. 5B is a diagram showing an example of the multilayer cluster Cm. It is determined that the object candidate cluster B4 and the object candidate cluster B3 are included in the same cluster, the object candidate cluster B3 and the object candidate clusters B21, B22 are included in the same cluster, and the object candidate clusters B21, B22 and the object candidate clusters are included. B12 is determined to be included in the same cluster, and it is determined that the object candidate cluster B12 and the object candidate cluster B0 are included in the same cluster, and as a result, a multi-layered cluster including object clusters B4, B3, B21, B22, B12, B0. Cm was produced.

Then, when the multilayer cluster Cm satisfies a predetermined condition, the object candidate cluster included in the multilayer cluster Cm is detected as an object. The predetermined condition relates to the size of the object candidate cluster or the total number of pixels of the object candidate cluster included in the multi-layered cluster Cm. For example, the size of the object candidate cluster included in the multi-layered cluster Cm is larger than the predetermined size. Further, when the total number of pixels of the object candidate clusters included in the multilayer cluster Cm is equal to or larger than a predetermined threshold value, the object candidate clusters included in the multilayer cluster Cm are determined to be objects.

When the multilayer cluster Cm satisfies a predetermined condition, an object is detected for each range image. For example, when one layer of the object candidate cluster B4 is included in the multi-layered cluster Cm as in the distance image at time t-4, the object candidate cluster B4 is detected as an object. When two object candidate clusters B21 and B22 are included as in the distance image at time t-2, the two object candidate clusters B21 and B22 are the same one object detected at time t-4. It is determined to indicate. As a result, even when the object is divided into two object candidate clusters due to the noise source as at time t-2, one object is obtained by using the information of the multi-layered cluster Cm generated including the range images acquired at other times. It can be detected as an object.

In addition, non-background pixels that appear for a moment, such as the pixels N2 and N3 at time t-1, can be determined to be noise because the object candidate clusters are not continuous at the times before and after that, and the noise source is the object. It will not be erroneously detected.

[Configuration of object detection device]
The configuration of the object detection device 20 will be described with reference to FIG. The object detection device 20 includes a distance image acquisition unit 21, a non-background pixel extraction unit 22, a first clustering unit 23, a second clustering unit 24, an object detection unit 25, and an output unit 26. .. Some or all of these units may be configured by hardware, or may be realized by a processor of a computer executing a predetermined program.

The distance image acquisition unit 21 acquires the distance image generated by the distance measuring device 10. In the first embodiment, it is assumed that the data of the range image at the time of scanning in the horizontal direction is acquired. In this case, the value of each pixel (ims1) of the distance image indicates the distance R(ims1) to the object in the direction corresponding to the pixel.

The non-background pixel extraction unit 22 extracts a non-background pixel (ims1) from the distance image. The “background” may be a distance measurement target of the distance measuring device 10, but does not need to be detected as an object, and is, for example, a floor, a ceiling, or fixed furniture. The non-background pixel extraction unit 22 compares the background image stored in the background image data storage unit 27 with the distance image, and extracts a pixel whose difference is equal to or larger than a predetermined threshold as a non-background pixel.

Here, the background image stored in the background image data storage unit 27 can be, for example, a distance image generated by the distance measuring apparatus 10 in a state where there is no object in the object detection area. Alternatively, it may be an average distance image obtained by averaging a number of distance images generated by the distance measuring device 10 for each pixel. In the latter case, the influence of noise of the distance measuring device 10 can be suppressed, and a background image can be generated even when it is difficult to realize a state where there is no object in the object detection area. In addition to the arithmetic mean, a geometric mean or a trimmed mean can be used as the mean.

The first clustering unit 23 clusters the non-background pixels extracted by the non-background pixel extraction unit 22 to generate an object candidate cluster. The first clustering unit 23 includes a point in the space indicated by a certain non-background pixel (referred to as pixel (ims1)) and a point in the space indicated by an adjacent non-background pixel (referred to as pixel (i'ms1)). A relative distance R _rltv with _respect to is calculated, and clustering is performed based on this relative distance R _rltv . Here, "adjacent non-background pixels" will be described. The distance image has the distance data for each distance measuring direction as a pixel, and adjacent pixels can be defined by the distance measuring direction. A pixel includes a background pixel and a non-background pixel, and a pixel adjacent to a non-background pixel is not necessarily a non-background pixel. However, "adjacent non-background pixel" means that a neighboring pixel is a background pixel. Is a non-background pixel which is the first non-background pixel that appears first by skipping the background pixel and looking at the adjacent pixel.

The relative distance between adjacent non-background pixels is the distance R (i'ms1) between the object indicated by the non-background pixel (ims1) and the distance measuring device 10, and the object indicated by the adjacent non-background pixel (i'ms1). Is calculated based on the distance R(i′ms1) from the distance measuring device 10. FIG. 6 is a diagram illustrating a method of calculating the relative distance R _rltv between pixels. As shown in the figure, the point P(ims1) indicated by the pixel (ims1) is specified by the distance R(ims1) and the vector D(ims1) representing the distance measuring direction with the distance measuring device 10 as the origin O. Similarly, the point P'(i'ms1) indicated by the pixel (i'ms1) is specified by the distance R(i'ms1) and the vector D(i'ms1) representing the distance measuring direction.

The first clustering unit 23 calculates the relative distance R _rltv by applying the cosine theorem to the triangle OPP′ based on the distances R and R′ and the vectors D and D′ representing the distance measuring direction. Specifically, the relative distance R _rltv ((ims1), (i'ms1)) between the pixel (ims1) and the pixel (i'ms1) is obtained by the following equation (1).

When the relative distance R _rltv between adjacent non-background pixels is equal to or smaller than a predetermined threshold value (for example, 0.20 [m]), the first clustering unit 23 determines that the two pixels belong to the same cluster, and determines the relative distance. When R _rltv is larger than the predetermined threshold value, it is determined that the two pixels do not belong to the same cluster. The non-background pixels included in the range image are divided into a plurality of object candidate clusters by sequentially performing the comparative evaluation of the relative distance R _rltv between two adjacent non-background pixels and the threshold value for all the pixels forming the range image. ..

Returning to FIG. 1, the second clustering unit 24 will be described. The second clustering unit 24 clusters the plurality of object candidate clusters generated by the first clustering in the time direction to generate a multi-layered cluster. The second clustering unit 24 calculates the relative distance R _B2rltv between the object candidate cluster obtained from the distance image at the predetermined time and the object candidate cluster obtained from the distance image at the adjacent time.

ただし、i_ms1はB(t,j_t)内の画素、i'_ms1,はB(t-1,j'_t-1)内の画素である。 As the relative distance R _B2rltv between the object candidate clusters, the relative distance between all the pixels included in each object candidate cluster is calculated, and the minimum relative distance is _obtained as the relative distance R _B2rltv of the object candidate cluster. Specifically, according to the following equation (2), the j _t- th object candidate cluster B(t,j _t ) obtained from the range image obtained at a predetermined time t and the adjacent time t−1 are obtained. The relative distance R _B2rltv to the j′ _t−1 th object candidate cluster B(t-1, j′ _t−1 ) obtained from the distance _image is obtained.

However, i _ms1 is a pixel in B(t,j _t ), and i′ _ms1 is a pixel in B(t-1,j′ _t−1 ).

The second clustering unit 24 also calculates the relative distance R _B2rltv between pixels based on the distance R indicated by each pixel of the distance image and the distance measurement scanning direction vector D without converting the point indicated by each pixel into the XY coordinate system. calculate. As a result, the amount of clustering calculation can be reduced.

Then, when the relative distance R _B2rltv is equal to or less than a predetermined threshold value (for example, 0.50 [m]), the second clustering unit 24 determines that the two object candidate clusters belong to the same cluster, and the relative distance R _{When B2rltv} is larger than a predetermined threshold value, it is determined that the two object candidate clusters belong to different clusters. One or a plurality of multi-layered clusters are formed by sequentially performing the comparative evaluation of the relative distance R _B2rltv between the object candidate clusters obtained from the distance images acquired at the adjacent times and the threshold value for all the object candidate clusters. To be done.

Returning to FIG. 1, the object detection unit 25 will be described. The object detection unit 25 detects an object based on whether or not the multilayer cluster generated by the second clustering unit 24 satisfies a predetermined condition. The condition used in the present embodiment is that the size of the largest object candidate cluster included in the multilayer cluster Cm is larger than a predetermined size (for example, 0.5 m) and the object candidate cluster included in the multilayer cluster Cm. When the total number of pixels of is equal to or larger than a predetermined threshold value (for example, 20 pixels), the object candidate cluster included in the multilayer cluster Cm is determined to be an object. When the multilayer cluster Cm satisfies the predetermined condition, the object detection unit 25 detects an object for each distance image. That is, one or a plurality of object candidate clusters obtained from the distance image are detected as one object.

The output unit 26 outputs the object detected by the object detection unit 25 according to the output coordinate system. The object detected by the object detection unit 25 is a set of three-dimensional observation points of θms1, R, and T. When outputting as it is, the output unit 26 outputs in the R-θms1-T coordinate system. When the time information is unnecessary, for example, it is output in the R-θms1 coordinate system similar to the distance image. Further, it may be converted into the XYT coordinate system, and also in this case, if the time information is unnecessary, it may be converted into the XY coordinate system. In the present embodiment, an example of converting to an XYT coordinate system and outputting will be described. As an output mode, for example, it may be displayed on a display means such as a monitor.

FIG. 7 is a diagram showing an example of the detection result of the object K output by the output unit 26. As shown in FIG. 7, by displaying the detection result of the object K in the XYT coordinate system, the movement of the object can be easily grasped.

[Operation of Object Detection Device]
Next, the operation of the object detection device 20 of the present exemplary embodiment will be described. FIG. 8 is a flowchart showing the operation of detecting an object by the object detection device 20. The object detection device 20 acquires the distance image data from the distance measuring device 10 (S10). The distance measuring device 10 periodically performs distance measurement to generate a distance image, and the object detection device 20 periodically acquires distance image data from the distance measuring device 10.

The object detection device 20 extracts a non-background pixel from the acquired range image (S11). Subsequently, the object detection device 20 performs the first clustering on the non-background pixels included in the distance image to generate an object candidate cluster that is a set of non-background pixels (S12). The first clustering process will be described later with reference to FIG. The object detection device 20 performs the second clustering on the object candidate clusters to generate a multi-layered cluster which is a set of object candidate clusters (S13). The second clustering process will be described later with reference to FIG.

The object detection device 20 determines whether or not the multilayer cluster satisfies a predetermined condition, and detects an object (S14). Although a plurality of multi-layered clusters are generated by the second clustering process, an object is detected from the multi-layered clusters that satisfy a predetermined condition, and a multi-layered cluster that does not satisfy the predetermined condition is determined as noise.

The object detection device 20 converts the pixel value indicating the detected object into a spatial coordinate system (S15). As described above, when outputting the position information of the detected object, the conversion may be performed so as to correspond to the output coordinate system, but in the present embodiment, the position information of the object is converted into the XYT coordinate system. Subsequently, the object detection device 20 outputs the detection result (S16).

FIG. 9 is a flowchart showing the process of the first clustering. First, the first clustering unit 23 reserves a temporary storage area for sequentially adding pixels belonging to the object candidate cluster (step S20).

Then, the first clustering unit 23 acquires a pixel to be processed (ims1) from the distance image (step S21). The order of acquisition of pixels to be processed is to first acquire the non-background pixels at the edge of the scanning range, and then sequentially acquire the non-background pixels adjacent to the previous non-background pixels. A storage unit that stores the acquisition order of the pixels to be processed may be provided.

The first clustering unit 23 determines whether pixels are stored in the temporary storage area (S22). If the pixel is not stored in the temporary storage area (NO in S22), the pixel to be processed is stored in the storage area (S28). When the non-background pixel is first acquired from the range image, the pixel is not stored in the temporary storage area, so the first processing target pixel is stored in the temporary storage area (S28).

If the pixel is stored in the temporary storage area (YES in S22), the immediately preceding processing target pixel adjacent to the processing target pixel is read from the temporary storage area (S23). As described above, since the first clustering unit 23 sequentially reads the adjacent non-background pixels, the non-background pixels read from the temporary storage area here are the non-background pixels adjacent to the processing target pixel. The first clustering unit 23 calculates the relative distance R _rltv between the pixel to be processed and the pixel to be processed _immediately before by the equation (1) described above (S24).

The first clustering unit 23 determines whether or not the _calculated relative distance R _rltv is less than or equal to a predetermined threshold value (S25). When the relative distance R _rltv is equal to or less than the predetermined threshold value (YES in S25), it is determined that the processing target pixel and the _immediately preceding processing target pixel belong to the same cluster, and the first clustering unit 23 determines the processing target. The pixel is stored in the temporary storage area (S26). When the relative distance R _rltv is not equal to or less than the predetermined threshold value (NO in S25), it is determined that the processing target pixel and the processing target pixel _immediately before belong to a different cluster, and the first clustering unit 23 stores the temporary storage area in the temporary storage area. It is determined that the stored processing target pixels form an object candidate cluster (S27).

The first clustering unit 23 moves the pixel data forming the confirmed object candidate cluster to another storage area and clears the data in the temporary storage area (S27). At this time, if only one pixel is stored in the temporary storage area, that pixel does not form a cluster with the adjacent pixels before and after, so it is cleared from the temporary storage area without moving to another storage area. You may. Subsequently, the first clustering unit 23 stores the processing target pixel in the temporary storage area (S28).

The first clustering unit 23 determines whether or not all the non-background pixels included in the distance image have been processed (S29), and if there is an unprocessed non-background pixel (NO in S29), the unprocessed non-background pixels have not been processed. Returning to the step of extracting the non-background pixel of the process as the process target pixel (S21), the above-described process is repeated.

FIG. 10 is a flowchart showing the process of the second clustering. 11 and 12 are diagrams showing a process in which a multilayer cluster is generated by the second clustering process.

The second clustering unit 24 stores the object candidate cluster generated from the distance image with the earliest acquisition time as a multilayer cluster candidate (S30). In the example shown in FIG. 11A, the two object candidate clusters at the earliest acquisition time t-3 are stored as the multilayer cluster candidates Cm1 and Cm2.

The second clustering unit 24 selects a distance image acquired at an adjacent time as a processing target (S31), and selects an object candidate cluster obtained from the processing target distance image as a processing target (S32). Then, the relative distance R _B2rltv between the selected object candidate cluster and the multilayer cluster candidate is calculated (S33). In the example illustrated in FIG. 11A, the distance image at the acquisition time t-2 adjacent to the acquisition time t-3 is selected as the processing target (S31), and the object candidate cluster B1 is selected as the processing target (S32). Then, the relative distance R _B2rltv between the object candidate cluster B1 and the multilayer cluster candidates Cm1 and _Cm2 is calculated (S33).

The second clustering unit 24, the relative distance R _B2rltv it is determined whether there is a following multilayered cluster candidates threshold (S34), the relative distance in the case where R _B2rltv is the following multilayered cluster candidates threshold (S34 YES), the object candidate cluster to be processed is added to the multilayer cluster candidate (S35). If there is no multilayer cluster candidate having the relative distance R _B2rltv equal to or less than the threshold value (NO in S34), the object candidate cluster to be processed is stored as a new multilayer cluster candidate (S36).

In the example shown in FIG. 11A, a solid arrow indicates that the relative distance R _B2rltv is less than or equal to the threshold, and a dotted arrow indicates that the relative distance R _B2rltv is not less than or equal to the threshold. Here, since the relative distance R _B2rltv between the object candidate cluster B1 and the multi-layered cluster candidate Cm1 is less than or equal to a predetermined threshold value, the object candidate cluster B1 is added to the multi-layered cluster candidate Cm1 as shown in FIG. 11B. With respect to the multi-layered cluster candidate Cm2, there is no object candidate cluster to be added, so the update of the multi-layered cluster candidate Cm2 ends at this point.

The second clustering unit 24 determines whether or not all the object candidate clusters of the distance image to be processed have been processed (S37), and when there is an unprocessed object candidate cluster (NO in S37), An unprocessed object candidate cluster is selected as a processing target (S32), and the above-described processing is performed.

When the processing has been performed on all the object candidate clusters of the distance image to be processed (YES in S37), the second clustering unit 24 determines whether or not all the distance images have been processed (S38), and unprocessed. If there is a distance image of (NO in S38), the distance image acquired at the time adjacent to the processed distance image is selected as a new processing target (S31), and the above-described processing is performed. When the processing has been performed for all distance images (YES in S38), the generated multilayer cluster candidate is confirmed as a multilayer cluster (S39), and the processing ends.

The process of generating the multi-layered cluster will be further described with reference to FIGS. 11 and 12. When the processing of the distance image at the acquisition time t-2 is completed, the distance image at the acquisition time t-1 is selected as a processing target. Object candidate clusters B2 to B4 to be processed are sequentially selected from the distance image and processed. As shown in FIG. 11C, the relative distance R _B2rltv between the object candidate cluster B2 and the multilayer cluster candidate Cm1 is calculated, and it is determined that the relative distance R _B2rltv is less than or equal to a predetermined threshold, and the object candidate cluster B2 is set as the multilayer cluster candidate Cm1. to add.

Note that, when the relative distance R _B2rltv between the object candidate cluster B2 and the multi-layered cluster candidate Cm1 is obtained, the distance at the time t−1 that is the processing target among the object candidate clusters included in the multi-layered cluster candidate Cm1 is used. It is the relative distance to the object candidate cluster B1 generated from the distance image at time t-2 adjacent to the image, and is not the relative distance to the object candidate cluster generated from the distance image at time t-3 that is not adjacent. ..

Similarly, the relative distance R _B2rltv between the object candidate cluster B3 and the multilayer cluster candidate Cm1 is calculated, and it is determined that the relative distance R _B2rltv is equal to or smaller than a predetermined threshold value, and the object candidate cluster B3 is added to the multilayer cluster candidate Cm1. Subsequently, the relative distance R _B2rltv between the object candidate cluster B4 and the multilayer cluster candidate Cm1 is calculated, it is determined that the relative distance R _B2rltv is not less than or equal to a predetermined threshold, and the object candidate cluster B4 is newly created as shown in FIG. The candidate is a multilayer cluster candidate Cm3.

With the above processing of the object candidate clusters B2 to B4 included in the distance image at time t−1 is completed, the distance image at time t is set as the processing target. The object candidate cluster B5 to be processed is selected from the distance image and processed. As shown in FIG. 12 (b), the relative distance R _B2rltv between object candidate clusters B5 and multilayered cluster candidates Cm1, Cm3 calculates the relative distance R _B2rltv the multilayered cluster candidates Cm1 is greater than a predetermined threshold value, determining the relative distance R _B2rltv the multilayered cluster candidate Cm3 is below a predetermined threshold value, adds an object candidate cluster B5 in multilayered cluster candidates Cm3. In the present embodiment, the multi-layered cluster candidates are added in order from the object candidate cluster of the distance image with the old acquisition time, but the multi-layered cluster candidates are added in order from the new distance image with the acquisition time. Good.

The multi-layered clusters Cm1, Cm2 and Cm3 as described above are generated. Then, the multi-layered cluster candidate is determined as the multi-layered cluster. The multi-layered cluster candidate Cm2 having only one layer when the update of the multi-layered cluster candidate is completed may not be determined as the multi-layered cluster.

The object detection system 1 of the present embodiment generates a multi-layered cluster by clustering object candidate clusters extracted from range images acquired at a plurality of times in the time direction, and determines whether the multi-layered cluster satisfies a predetermined condition. The object is detected depending on. Therefore, even if noise is included at a certain acquisition time, it is difficult to be affected by the noise, and the object can be appropriately detected. In the example illustrated in FIG. 12C, the two object candidate clusters B2 and B3 included in the distance image acquired at time t-1 perform object detection based on the multi-layered cluster Cm1, thereby resulting in time t-2. It is possible to detect that the object is the same object as the object candidate cluster B1 detected in step 1.

(Second embodiment)
FIG. 13 is a diagram showing the configuration of the object detection system 2 according to the second embodiment. The basic configuration of the object detection system 2 of the second embodiment is the same as that of the object detection system 1 of the first embodiment, but the object detection system 2 of the second embodiment is Data of a distance image when scanning in the horizontal direction and the vertical direction is acquired from the apparatus 10. In this case, the value of each pixel (ims1, ims2) of the distance image indicates the distance R (ims1, ims2) to the object in the direction corresponding to the pixel.

Since the object detection system 2 of the second embodiment processes a two-dimensional range image, the configuration of the first clustering unit 28 included in the object detection device 20 is different. The first clustering unit 28 performs main-direction clustering and sub-direction clustering when clustering non-background pixels included in a distance image to generate object candidate clusters.

The first clustering unit 28 has a main-direction clustering unit 29 and a sub-direction clustering unit 30. The main-direction clustering unit 29 divides the distance image to be processed into sub-directions to generate main-direction clustering units, and clusters non-background pixels in main-direction clustering units. In the present specification, the cluster generated by the principal direction clustering is referred to as "line segment".

FIG. 14A is a diagram showing an example in which the distance image is divided into main direction clustering units. As shown in the figure, for example, the distance image is divided into eight equal parts in the sub direction (actually, for example, about 60 equal parts) to set eight main direction clustering units 1 to 8. Each main-direction clustering unit can be made up of a plurality of lines (3 lines in the figure) of pixels. The main direction clustering unit 29 sequentially reads adjacent non-background pixels and performs clustering based on whether or not the relative distance between points in the three-dimensional space indicated by the adjacent non-background pixels is equal to or less than a predetermined threshold value. ..

FIG. 14B is a diagram schematically illustrating a main-direction clustering scan map that defines the order in which the main-direction clustering reads out pixels. The order of the pixels belonging to each main direction clustering unit may be the ascending order of the main direction index ims1, and the descending order of the sub direction index ims2 when there are a plurality of pixels having the same main direction index ims1. For example, it is assumed that the pixel (1,1) and the pixel (1,2) are adjacent to each other, and the pixel (1,1) and the pixel (2,3) are also adjacent to each other. The main direction clustering unit 29 has a main direction clustering scan map as shown in FIG. 14B, and reads out non-background pixels according to the main direction clustering scan map.

The sub-direction clustering unit 30 clusters the line segments generated by the main-direction clustering unit 29 to generate object candidate clusters. The sub-direction clustering unit 30 repeatedly performs the process of determining whether or not the line segments belong to the same cluster based on the relative distance between the line segments generated in the adjacent main-direction clustering units, performs clustering, Generate object candidate clusters. It is understood that this processing replaces the “distance image acquired at the adjacent time” with the “adjacent main direction clustering unit” and replaces the “object candidate cluster” with the “line segment” described in the second clustering unit 24. can do.

The second clustering unit 24 clusters the plurality of object candidate clusters generated by the first clustering in the time direction to generate a multi-layered cluster. The second clustering unit 24 calculates a relative distance between an object candidate cluster obtained from a distance image at a predetermined time and an object candidate cluster obtained from a distance image at an adjacent time.

In the present embodiment, the object candidate cluster is created by clustering the line segments in the sub direction and has a spread in the main direction and the sub direction. Therefore, if the distances are calculated by brute force for all the pixels included in the object candidate cluster, the calculation process of the distance between the object candidate clusters becomes very heavy, and the object may not be detected quickly. The present embodiment proposes a method of efficiently calculating the distance between object candidate clusters.

FIG. 15A and FIG. 15B are diagrams for explaining a method of the second clustering unit 24 of the second embodiment obtaining a distance between object candidate clusters. 15(a) and 15(b) are both object candidate clusters B _t-1,1 , B _t-1,2 , B _t-1,3 obtained from the range image acquired at time _t-1. , B _t-1,4 and object candidate clusters B _t,1 , B _t,2 , B _t,3 , B _t,4 obtained from the range image acquired at time t. In FIG. 15, the shaded area is the pixels forming the object candidate cluster. When calculating the distance between these two object candidate clusters, the second clustering unit 24 fixes the line in the sub direction and calculates the distance between pixels included in the corresponding sub direction.

In FIG. 15A, the sub-direction is fixed to the first line, and the relative distance between the pixels of the object candidate cluster on the first line is calculated. The relative distance between the pixels included in the first line at time t−1 and the pixels on the second, third,... Lines at time t is not calculated, and the first distance at time t is calculated. The relative distance between the pixels included in the line and the pixels in the second, third,... Lines at time t−1 is not calculated. By not considering the relative distance due to the combination of pixels in different sub-directions as described above, the amount of calculation can be significantly reduced as compared with the case where the relative distance between object candidate clusters is simply obtained.

It should be noted that here, although it has been described that the distance between the object candidate clusters is defined by the distance between the pixels in an easily understandable manner, in actual processing, the pixel data R (ims1, ims2) of the distance image is used. By setting only the pixels whose sub-direction index ims2 has a predetermined value as the calculation target, it is possible to perform the calculation with the line in the sub-direction fixed.

The state in which the line in the sub-direction is fixed is equivalent to a range image acquired by scanning in the horizontal direction, and the method described in the first embodiment is used to calculate this relative distance. When the calculation of the distance between the pixels on the first line is completed, the calculation target pixel is moved to the second line as shown in FIG. 15B, and the distance between the pixels is calculated. Then, the minimum distance among the distances between pixels included in the object candidate cluster is set as the distance of the object candidate cluster.

For example, in the example shown in FIG. 15, the relative distance between the object candidate cluster B _t-1,1 and the pixels included in the object candidate cluster B _t,1 is four (2 pixels×2 pixels) in the first line. ), 2 ways (2 pixels x 1 pixel) on the second line, and 4 ways (2 pixels x 2 pixels) on the third line, a total of 10 ways of relative distances are obtained. _Let be the relative distance between the object candidate cluster B _t-1,1 and the object candidate cluster B _t,1 .

As can be seen from the above description and FIGS. 15A and 15B, for example, the object candidate cluster B _t-1,1 and the object candidate cluster B _t,4 and the object candidate cluster B _t-1, Object candidate clusters such as ₁ and the object candidate cluster B _t,4 that do not exist in the sub-direction position do not correspond to the distance calculation, and the calculation can be omitted.

The contents described with reference to the schematic diagram of FIG. 15 will be described in more detail. B _(t, j t) time t j _t th object candidate cluster distance included in the image of, B (t-1, j 't-1) included in the time t-1 of the range image are j' _{t -1st} object candidate cluster. The relative distance R _B2rltv between two object candidate clusters B(t,j _t ) and B(t-1,j′ _t-1 ) respectively belonging to adjacent time intervals t,t−1 is the same in the clusters. The smallest value among the relative distances between pixels at the sub-direction position is selected. Specifically, with respect to each sub-direction of the pixels included in the object candidate cluster B(t,j _t ), the sub-direction ims2 is fixed, and the sub-direction ims2 of the pixels of B(t,j _t ) and B(t) are fixed. -1,j' _t-1 ), the smallest relative distance between pixels is obtained as R _{anyims1(ims2)} . Then, select the minimum value from among the object candidate cluster _{B (t, j t) R} anyims1 (ims2) determined for all sub-direction IMS2 contained in, this is the object candidate inter-cluster relative distance R _B2rltv.

ただし、
Ｒ(i_ms1,i_ms2,t)：時刻ｔにおける画素(i_ms1,i_ms2)の距離
Ｒ(i'_ms1,i_ms2,t-1)：時刻ｔ-1における画素(i'_ms1,i_ms2)の距離 According to the following equation (3), the object candidate cluster B(t,j _t ) obtained from the distance image acquired at the predetermined time t and the object candidate cluster obtained from the distance image acquired at the adjacent time t-1 The relative distance R _B2rltv with B(t-1,j' _t-1 ) is obtained.

However,
R(i _ms1 , i _ms2 , t): Distance of pixel (i _ms1 , i _ms2 ) at time t R(i' _ms1 , i _ms2 , t-1): Pixel (i' _ms1 , i at time t-1 _ms2 ) distance

The configuration of the object detection system 2 of the second embodiment different from that of the object detection system 1 of the first embodiment has been described above. Similar to the first embodiment of the object detection system 2 of the second embodiment, the multi-layered cluster can be used to perform accurate object detection.

Since the object detection system 2 according to the second embodiment clusters adjacent non-background pixels in two scanning directions when generating the object candidate clusters, it is possible to detect the shape of a three-dimensional object.

FIG. 16 is a diagram showing an example of the detection result of the object K output by the object detection system 2 according to the second embodiment. In the object detection system 2 of the second embodiment, the movement of the object K can be grasped in three dimensions.

(Third Embodiment)
FIG. 17 is a diagram showing the configuration of the object detection system 3 of the third exemplary embodiment. The third object detection system 3 has a function of detecting the moving direction and moving speed of the detected object, and tracking the object using data regarding this movement.

The basic configuration of the object detection system 3 of the third embodiment is the same as the basic configuration of the object detection system 1 of the first embodiment, but in the third embodiment, the object detection system 3 is used. The device 20 further includes an object movement detection unit 31, a search range setting unit 32, and an object tracking unit 33 in addition to the configuration of the object detection device 20 according to the first embodiment.

The object movement detection unit 31 converts the position of the object detected by the object detection unit 25 into an XY coordinate system, and obtains the movement direction and movement speed of the object from the positions and times of the objects at both ends of the multi-layer cluster. FIG. 18 is a diagram showing an example of obtaining the moving direction and moving speed of an object.

As shown in FIG. 18, the object movement detection unit 31 converts the pixel value of the detected object into the XYT coordinate system. Then, the average position of the area occupied by the object at each time is obtained. Then, from the average position p1(x,y) of the object at time t-4 and the average position p5(x,y) of the object at time t, the moving direction and moving speed of the object, that is, the moving vector v(X,Y). ). Here, the example in which the moving direction and the moving speed of the object are obtained in the XYT coordinate system has been described, but the moving direction and the moving speed may be obtained in the R-θ coordinate system.

The search range setting unit 32 has a function of setting a search range in which it is estimated that an object to be tracked is present. The object tracking unit 33 links (associates) the objects detected at different times and tracks the objects between different times. In addition, the different time referred to here is a time that is significantly different from the time range in which the multi-layered clustering is performed. This is because the object detection is performed on the assumption that the object is being tracked when the multi-layered cluster is generated. The search range setting unit 32 estimates the place where the object exists after a predetermined time elapses based on the moving direction and the moving speed of the object, and sets the search range around it. The search range is also referred to as “gate”, and the search range setting is also referred to as “open gate”.

FIG. 19 is a diagram for explaining the processing of the search range setting unit 32. The search range setting unit 32 obtains an estimated position P2a of the object after a predetermined time based on the current position P1 of the object and the movement vector v of the object, and sets a circular search range Sa centered on the estimated position of the object. To do. The radius of the search range set by the search range setting unit 32 may be determined according to the time interval for tracking the object. In other words, the radius of the search range may be increased when the time interval of tracking is long, and the radius of the search range may be decreased when the time interval is short.

Since the search range setting unit 32 of the present embodiment estimates the position of the object after a predetermined time using the movement vector v of the object, the search range can be set appropriately. As a result, it can be linked that the object is the same as the object detected in the search range, and the object can be tracked. In this example, the position of the object after the predetermined time is P2, and the object found in the search range after the elapse of the predetermined time can be correctly tracked as the currently detected object. If the movement vector v is not used, the estimated position P2b is obtained on the straight line connecting the position one before the object and the current position, and the circular search range Sb centered on the estimated position P2b is set. However, there is a possibility of making an association with the wrong object.

The object tracking unit 33 determines that an object having a moving direction and a moving speed that are similar to those of the tracking target object among the objects detected in the search range is the same as the tracking target object, and performs association. As a method of detecting an object in the search range, it is assumed that the detection method using the multi-layered cluster described above is adopted, and the current position of the object and its movement vector are obtained.

FIG. 20 is a diagram showing a method of obtaining the same object as the tracking target object when three objects O1 to O3 are detected within the search range. The object O1 detected at the observation point 1 has a movement vector Vo1, the object O2 detected at the observation point 2 has a movement vector Vo2, and the object O3 detected at the observation point 3 has a movement vector Vo3. The object to be tracked has a movement vector Vo. The movement vector Vo of the tracking target object is a movement vector used when setting the search range.

The object tracking unit 33 identifies an object having a movement vector close to the movement vector Vo of the tracking target object. For the closeness of the two movement vectors, a threshold is set for the difference between the direction and the magnitude of the two movement vectors, and when the difference is less than or equal to the threshold, it is determined that the two movement vectors are close. In the example shown in FIG. 20, the objects O1 and O3 are specified as the objects having similar movement vectors, and the object O2 is excluded. The object tracking unit 33 specifies that among the narrowed-down objects, an object having a short distance from the prediction point A1 is the same object as the tracking target object. In this example, since the distance L1<the distance L3, the object O1 is specified. In this way, the object tracking unit 33 can specify the same object by using the movement vector of the tracking target object.

FIG. 21 is a diagram for explaining processing when the search ranges of a plurality of objects overlap. In the example shown in FIG. 21, the search range S1 of the tracking object A1 and the search range S2 of the tracking object A2 overlap. Then, the object O2 is detected at the observation point 2 in the area where the object O2 overlaps the search range S1 and the search range S2. Here, it is assumed that the movement vector of the object O2 is close to that of the tracking object A1 and that of the tracking object A2. In this case, even if the identity of the object is determined based on the closeness of the movement vector of the tracking target object, the object O2 cannot be associated with either of the tracking objects A1 and A2. In such a case, the object tracking unit 33 associates with the tracking object in the search range having the shorter distance to the prediction point which is the center of the search range. In this example, since the distance L2_1 to the prediction point 1 <the distance L2_2 to the prediction point 2, the tracking object A1 in the search range S1 is associated.

The object detection system 3 according to the third embodiment has been described above. The object detection system 3 according to the third embodiment can obtain not only the information on the position of the object but also the moving direction and the moving speed (that is, the moving vector). Then, by utilizing the information of the movement vector for tracking the object, the object can be tracked efficiently and accurately.

The object detection system of the present invention has been described in detail above with reference to the embodiments, but the present invention is not limited to the above-mentioned embodiments. When generating a multi-layered cluster, it was explained that if the distance between object candidate clusters is less than or equal to a predetermined threshold, it is determined that they belong to the same cluster, but this threshold is determined according to the type of object to be detected. be able to. For example, depending on whether the object to be detected is a vehicle or a person, the threshold may be larger in the case of a vehicle. This is because the moving speed of the vehicle is higher, and thus the vehicle may move greatly in the same time interval.

Further, this threshold may be changed according to the time interval for acquiring the range image. That is, when the distance image acquisition interval is 1 second, the moving distance between the two distance images may be twice as large as when the distance image acquisition interval is 0.5 seconds.

INDUSTRIAL APPLICABILITY The present invention is not easily affected by noise, can appropriately detect an object, and is useful as an object detection device using a distance measuring device.

1-3 Object detection system 10 Distance measuring device 20 Object detection device 21 Distance image acquisition unit 22 Non-background pixel extraction unit 23 First clustering unit 24 Second clustering unit 25 Object detection unit 26 Output unit 27 Background data storage unit 28 First Clustering unit 29 Main direction clustering unit 30 Sub direction clustering unit 31 Object movement detection unit 32 Search range setting unit 33 Object tracking unit

Claims (12)

A distance image acquisition unit for acquiring a distance image having the distance measurement data in each distance measurement direction of the distance measurement device as a pixel value;
A non-background pixel extraction unit that extracts non-background pixels from the distance image,
The relative distance between two points in the space corresponding to two adjacent non-background pixels is calculated based on the distance measurement direction of the non-background pixel and the distance measurement data, and the adjacent non-pixels are calculated based on the relative distance. A first clustering unit that generates an object candidate cluster that is a set of the non-background pixels by repeatedly performing a process of determining whether background pixels belong to the same cluster;
Based on the second relative distance in space of the object candidate clusters whose acquisition times are respectively generated from the adjacent distance images , and which is defined by the relative distances of the pixels forming the object candidate clusters. Then, it is determined that adjacent object candidate clusters are included in the same cluster when the second relative distance is less than or equal to a predetermined threshold, and the adjacent object candidate clusters are included in the same cluster when the second relative distance is greater than a predetermined threshold. A second clustering unit that generates a multi-layered cluster that is a set of the object candidate clusters by repeatedly performing a process of determining that the object clusters are not included ;
An object detection unit that detects one or a plurality of object candidate clusters in each distance image included in the multi-layered cluster as an object when the multi-layered cluster satisfies a predetermined condition,
An object detection device comprising. The second clustering unit weights the second relative distance in the space of the object candidate cluster to be determined according to the time interval of the acquisition time of the distance image that generated the object candidate cluster, and after weighting. The object detection device according to claim 1, wherein clustering of the object candidate clusters is performed based on the distance. The distance measuring device is a distance measuring device that performs two-dimensional scanning in a main direction and a sub direction,
The first clustering unit is
A main direction clustering unit that divides the distance image into a plurality of regions corresponding to the main direction, and performs line clustering of the non-background pixels in each of the divided regions to generate a line segment,
Based on the distance in space of the line segment generated in the adjacent region, a sub-direction clustering unit that performs clustering of the line segment to generate the object candidate cluster,
The object detection device according to claim 1 or 2, further comprising: In the second clustering unit, the process of obtaining the second relative distance between the two object candidate clusters to be determined is
A first process of calculating a relative distance between pixels having a common sub-direction in the two object candidate clusters, and obtaining a minimum relative distance among combinations of pixels having a common sub-direction;
Determining the minimum relative distance to the first treatment in each direction performed by changing the sub-direction, the smallest minimum relative distance among the minimum relative distance seek, the second obtaining as the second relative distance Processing,
The object detection device according to claim 3, wherein 5. The object detection unit according to claim 1, wherein the object detection unit detects an object based on the size of the object candidate cluster included in the multilayer cluster and the number of pixels included in the multilayer cluster. Object detection device. The object detection according to claim 1, further comprising: an object movement detection unit that obtains a moving direction and a moving speed of the object based on a transition of the position of the object in each distance image detected from the multi-layered cluster. apparatus. The object movement detection unit converts a pixel value of the detected object into position data of a three-dimensional space coordinate system, specifies a space occupied by the object in the three-dimensional space, and obtains a moving direction and a moving speed of the object. Item 6. The object detection device according to item 6. The object detection device according to claim 6 or 7, further comprising: an object tracking unit that detects the same object from the objects detected at different times based on the moving direction and the moving speed of the object and tracks the object. .. Based on the moving direction and moving speed of the object detected by the object detection unit, a range in which the object exists is estimated after a lapse of a predetermined time, and a search range setting unit that sets the range as a search range of the object is provided. ,
The object detection device according to claim 8, wherein the object tracking unit determines that an object detected from the search range after the predetermined time has passed is the same object as an object detected at a previous time. Rangefinder,
An object detection device according to any one of claims 1 to 9,
An object detection system including. A method of detecting an object by an object detection device based on a distance image having pixel values of distance measurement data in each distance measurement direction of the distance measurement device,
The object detection device, a step of acquiring the distance image,
The object detection device extracting non-background pixels from the range image;
The object detection device calculates a relative distance between two points in the space corresponding to two adjacent non-background pixels based on the distance measurement direction of the non-background pixel and the distance measurement data, and calculates the relative distance. Repeating the process of determining whether the adjacent non-background pixels belong to the same cluster based on the step of generating an object candidate cluster that is a set of the non-background pixels,
In the object detection device, the acquisition time is a second relative distance in the space of the object candidate clusters generated from the distance images adjacent to each other and is defined by a relative distance of pixels forming the object candidate cluster. Based on the relative distance of 2, it is determined that adjacent object candidate clusters are included in the same cluster when the second relative distance is less than or equal to a predetermined threshold value, and the second relative distance is greater than the predetermined threshold value. In the case, by repeatedly performing the process of determining that they are not included in the same cluster, generating a multi-layered cluster that is a set of the object candidate clusters,
The object detecting device, when the multi-layered cluster satisfies a predetermined condition, detecting one or a plurality of object candidate clusters in each distance image included in the multi-layered cluster as an object;
An object detection method comprising: A program for detecting an object on the basis of a distance image in which distance measurement data in each distance measurement direction of a distance measurement device is used as a pixel value, the program comprising:
Acquiring the range image,
Extracting non-background pixels from the range image,
The relative distance between two points in the space corresponding to two adjacent non-background pixels is calculated based on the distance measurement direction of the non-background pixel and the distance measurement data, and the adjacent non-pixels are calculated based on the relative distance. Generating an object candidate cluster that is a set of the non-background pixels by repeatedly performing a process of determining whether or not the background pixels belong to the same cluster,
Based on the second relative distance in space of the object candidate clusters whose acquisition times are respectively generated from the adjacent distance images , and which is defined by the relative distances of the pixels forming the object candidate clusters. Then, it is determined that adjacent object candidate clusters are included in the same cluster when the second relative distance is less than or equal to a predetermined threshold, and the adjacent object candidate clusters are included in the same cluster when the second relative distance is greater than a predetermined threshold. By repeatedly performing the process of determining that it is not included , generating a multi-layered cluster that is a set of the object candidate clusters,
Detecting, as an object, one or more object candidate clusters in each range image included in the multi-layered cluster when the multi-layered cluster satisfies a predetermined condition,
A program to execute.

Images