JP2021111018A - Object detection device and object detection method - Google Patents

Abstract

To provide an object detection device and an object detection method capable of suppressing an increase in cost when detecting a moving object.SOLUTION: An object detection device 1 comprises: an edge extraction unit 12 which extracts an edge of an image of an area R acquired at a first time; an edge amount setting unit 13 which sets an edge amount based on an extraction result of the edge, for a plurality of cells S corresponding to each area of the image; a candidate point setting unit 14 which sets the cell S in which the edge amount is a certain amount or more, as a candidate point Sa of the cell S in which a moving object Q exists at the first time; an estimation unit 16 which calculates an estimated result P5 of the moving object Q at the first time, based on the candidate point Sa at the first time, and an estimated result P4 of the moving object Q at a second time before the first time; and a detection unit 17 which detects the moving object Q which exists in the area R at the first time, based on the estimated result P5 at the first time.SELECTED DRAWING: Figure 3

Description

The present invention relates to an object detection device and an object detection method.

Patent Document 1 describes a three-dimensional object detection device. This device inputs captured image data of a predetermined region obtained by imaging with a camera, and generates bird's-eye view image data of the input captured image data. In addition, this device generates a difference image between the bird's-eye view image at the current time and the bird's-eye view image one time ago. Then, this device detects a three-dimensional object based on the generated difference image.

International Publication No. 2013/129357

In the above-mentioned three-dimensional object detection device, an appropriate detection of an adjacent vehicle is attempted by using a difference image between the image at the current time and the image one time before. On the other hand, when such image recognition is used when detecting a moving object such as an adjacent vehicle, a high-performance processing device is required, which increases the cost.

An object of the present invention is to provide an object detection device capable of suppressing an increase in cost when detecting a moving object, and an object detection method.

The object detection device according to the present invention is an object detection device for detecting a moving object in the peripheral region of the vehicle while the vehicle is moving, and is an edge for extracting an edge of an image of the peripheral region acquired at the first time. The extraction unit, the edge amount setting unit that sets the edge amount based on the edge extraction result for a plurality of cells corresponding to each area of the image, and the cell having an edge amount of a certain amount or more are moved at the first time. At the first time, based on the candidate point setting unit that sets the candidate point of the cell in which the object exists, the candidate point at the first time, and the estimation result of the moving object at the second time before the first time. It includes an estimation unit that calculates an estimation result of a moving object, and a detection unit that detects a moving object existing in a peripheral region at the first time based on the estimation result at the first time.

The object detection method according to the present invention is an object detection method for detecting a moving object in a peripheral region of a vehicle while the vehicle is moving, and is an edge for extracting an edge of an image of the peripheral region acquired at the first time. The extraction step, the edge amount setting step of setting the edge amount for each of the cells based on the edge extraction result for a plurality of cells corresponding to each area of the image, and the cell having the edge amount of a certain amount or more are the first. Based on the candidate point setting step of setting as a candidate point of a cell in which a moving object exists at one time, the candidate point of the first time, and the estimation result of the moving object at the second time before the first time. It includes an estimation step of calculating an estimation result of a moving object at a first time, and a detection step of detecting a moving object existing in a peripheral region at the first time based on the estimation result at the first time.

In these devices and methods, the edges of the image in the peripheral region of the vehicle are extracted, and the edge amount is set for each of the plurality of cells corresponding to each region of the image. Further, a cell having an edge amount of a certain amount or more is set as a candidate point where a moving object exists at the first time. Further, the estimation result of the moving object at the first time is generated based on the candidate point at the first time and the estimation result of the moving object at the second time before the first time. Then, a moving object is detected based on this estimation result. As described above, in these devices and methods, the edge amount of the image having a lower calculation load is used when detecting the moving object, so that a high-performance processing device becomes unnecessary. Therefore, an increase in cost can be suppressed when detecting a moving object.

In the object detection device according to the present invention, the frequency of the normal distribution is voted for the cell that is the candidate point at the first time and the cells around the cell so that the cell that is the candidate point is the center. A distribution generation unit that generates a frequency distribution is provided, and the estimation unit calculates the estimation result of the moving object at the first time based on the multiplication result of the frequency distribution and the estimation result of the moving object at the second time. May be good. In this case, the influence of noise is reduced and the detection accuracy is improved.

In the object detection device according to the present invention, the candidate point setting unit sets a cell having a frequency equal to or higher than a certain frequency in the frequency distribution as a new candidate point for a cell in which a moving object exists at the first time, and the distribution generation unit sets the cell. , By voting for the cell that is the new candidate point at the first time and the cells around the cell, the frequency of the normal distribution is centered so that the cell that is the new candidate point is the center. The frequency distribution may be generated, and the estimation unit may calculate the estimation result of the moving object at the first time based on the multiplication result of the new frequency distribution and the estimation result of the moving object at the second time. In this case, the influence of noise is further reduced and the detection accuracy is further improved.

According to the present invention, it is possible to provide an object detection device and an object detection method capable of suppressing an increase in cost when detecting a moving object.

FIG. 1 is a schematic view showing a vehicle equipped with the object detection system according to the present embodiment. FIG. 2 is a schematic diagram showing the configuration of the object detection system shown in FIG. FIG. 3 is a diagram showing a functional configuration of the object detection device shown in FIGS. 1 and 2. FIG. 4 is a flowchart showing the operation of the object detection device. FIG. 5 is an example of an image of a region to be detected as an object and an image showing an edge extraction result. FIG. 6 is a diagram showing an example of the frequency distribution at the first time. FIG. 7 is a diagram showing an example of the estimation result.

Hereinafter, one embodiment will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding elements may be designated by the same reference numerals, and duplicate description may be omitted.

FIG. 1 is a schematic view showing a vehicle equipped with the object detection system according to the present embodiment. FIG. 2 is a schematic diagram showing the configuration of the object detection system shown in FIG. FIG. 1A is a plan view, and FIG. 1B is a side view. The vehicle C can be, for example, a commercial vehicle such as a bus or an industrial vehicle such as a truck (in the illustrated example, it is a truck). Further, the vehicle C can be a large vehicle, a medium-sized vehicle, or the like having a height higher than that of an ordinary passenger car. The vehicle C includes a vehicle body Cb and a pair of mirror stays Cs attached to the vehicle body Cb on both sides in the vehicle width direction (here, the Y-axis direction).

The object detection system 100 is mounted on the vehicle C. The object detection system 100 includes an object detection device 1, an image pickup device 50, and an output device 60. The object detection device 1 will be described later. The image pickup device 50 is a device capable of capturing an image of a predetermined area and acquiring an image, and is, for example, an electronic mirror, a surveillance camera, or the like. The image pickup apparatus 50 is provided on each of the pair of mirror stays Cs. Therefore, the image pickup devices 50 are provided on both sides of the vehicle C in the vehicle width direction.

Further, the image pickup apparatus 50 is provided in front of the vehicle C in the front-rear direction (here, the X-axis direction) of the vehicle C (depending on the position of the mirror stays Cs). Further, the image pickup device 50 may be provided above the center of the vehicle C in the height direction (here, the Z-axis direction). As a result, the imaging region 50r of the imaging device 50 includes a region (peripheral region) R (a region that becomes a blind spot from the driver of the vehicle C) of the vehicle C that is near the side surface of the vehicle body Cb and includes the road surface Rs. Has been made. The image pickup device 50 outputs the acquired image to the object detection device 1 and the like.

The output device 60 acquires and outputs information indicating the detection result from the object detection device 1. The output device 60 is, for example, a display, a speaker, a vibrator, or the like, and is capable of outputting the detection result of the object detection device 1 to the driver of the vehicle C by sight, voice, vibration, or the like. The vehicle C is provided with a rear side monitoring device 51 for each of both side portions of the vehicle C in the vehicle width direction at the rear portion of the vehicle C. The rear side monitoring device 51 is, for example, a back camera, a laser radar, or the like having a predetermined detection range 51r.

Subsequently, the object detection device 1 will be described. FIG. 3 is a diagram showing a functional configuration of the object detection device shown in FIGS. 1 and 2. The object detection device 1 shown in FIG. 3 is for detecting a moving object in the region R, which is a peripheral region of the vehicle C, while the vehicle C is moving. The object detection device 1 may be configured as an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. Each functional unit described later in the object detection device 1 can be realized by loading the program recorded in the ROM into the RAM and executing the program loaded in the RAM in the CPU.

First, an outline of the operation of each part of the object detection device 1 will be described. The object detection device 1 includes an image generation unit 11, an edge extraction unit 12, an edge amount setting unit 13, a candidate point setting unit 14, a distribution generation unit 15, an estimation unit 16, and a detection unit 17. The image generation unit 11 receives the input of the captured image of the region R from the imaging device 50, and generates an image suitable for the subsequent processing (hereinafter, may be referred to as “target image”) from the captured image. The edge extraction unit 12 extracts the edge of the target image. The edge amount setting unit 13 sets the edge amount for each of the plurality of cells corresponding to each area constituting the target image based on the edge extraction result.

The candidate point setting unit 14 sets a cell having an edge amount of a certain amount or more as a candidate point of a cell in which a moving object exists. The distribution generation unit 15 generates a frequency distribution by voting for a cell having a candidate point and cells surrounding the cell with a frequency having a normal distribution so that the cell having the candidate point is at the center. .. The estimation unit 16 estimates the moving object at the first time based on the current (first time) candidate point and the estimation result of the moving object in the past (second time) before the first time. To generate. The detection unit 17 detects the moving object existing in the region R at the first time based on the estimation result of the moving object at the first time. The detection unit 17 outputs information indicating the detection result to the output device 60.

Subsequently, the details of the operation of each part of the object detection device 1 will be described. FIG. 4 is a flowchart showing the operation of the object detection device. The flowchart shown in FIG. 4 shows an object detection method according to the present embodiment.

In this method, first, the image generation unit 11 receives the input of the captured image (the image of the region R acquired at the first time) from the imaging device 50. FIG. 5A is an example of an input image. The input image includes a moving object Q and a road surface Rs. Subsequently, the image generation unit 11 reduces the input image (step S1). Here, the image generation unit 11 reduces the size of the input image by, for example, thinning out a plurality of regions (pixels) constituting the input image. Subsequently, the image generation unit 11 performs a projective transformation of the reduced input image (step S2). As a result, an orthographic projection image is generated from the input image. Further, in this step S2, the image generation unit 11 cuts out only the lane P1 adjacent to the traveling lane of the vehicle C from the orthographic projection image (see (a) of FIG. 5).

Subsequently, the image generation unit 11 smoothes the short-distance portion of the orthographic projection image (S3). This is because when the orthographic projection image is generated from the input image in step S2, the long-distance portion far from the image pickup device 50 is stretched with respect to the short-distance portion close to the image pickup device 50. This is to avoid extracting many edges different from the moving object Q due to the roughness of the road surface Rs in the short distance portion at the time of extraction. As a result, a target image to be processed such as subsequent edge extraction is generated.

Subsequently, the edge extraction unit 12 extracts the edge of the target image (the image of the region R acquired at the first time) (step S4: edge extraction step). FIG. 5B is an example of the edge image P2 showing the extracted edges.

Subsequently, the edge amount of the target image is evaluated (step S5). More specifically, in step S5, first, the edge amount setting unit 13 sets the edge amount for a plurality of cells corresponding to each area of the target image based on the edge image P2 (edge extraction result). (Edge amount setting process).

The target image and the edge image P2 used for edge extraction correspond to each other. Therefore, after forming cells (cells arranged in a two-dimensional matrix) corresponding to each region when the target image is divided into, for example, a two-dimensional matrix, the target image exists in each corresponding region of the edge image P2. By counting the amount of edges (the number of dots that make up the edge), the amount of edges can be set for each cell. That is, in this step S5, a cell group including a plurality of cells arranged in a two-dimensional matrix (for example, a 15 × 4 matrix) and having an edge amount set is formed.

In step S5, the candidate point setting unit 14 subsequently evaluates the edge amount of each cell to select cells having an edge amount of a certain amount or more and cells in which a moving object exists at the first time (moving object in the target image). Is set as a candidate point (cell corresponding to the area where) exists (candidate point setting step). A plurality of candidate points may be set for the cell group configured in step S5.

Subsequently, the observation result is calculated (step S6). More specifically, in this step S6, the distribution generation unit 15 is normal so that the cell that is the candidate point at the first time is centered on the cell that is the candidate point and the cells around the cell. A frequency distribution (histogram) is generated by voting for the frequency of the distribution.

FIG. 6A is an example of a normal distribution frequency (digitized normal distribution) used for voting. FIG. 6B is an example of the frequency distribution generated as a result of voting. In the example of FIG. 6, the cell S at the coordinates (3, 3) of the frequency distribution P3 is set as the candidate point Sa, and the distribution V in a normal distribution of 3 × 3 is centered on the cell S of the candidate point Sa. Voting is done according to. The frequency distribution P3 generated in this way can be used as the observation result of the moving object at the first time.

On the other hand, when there are a plurality of candidate points Sa, if voting is performed as described above, cells S that receive duplicate votes may occur. In this case, even if the cell S that has received duplicate votes is not the cell S set as the initial candidate point Sa, the frequency is higher than that of the cell S set as the initial candidate point Sa. There is.

Therefore, in this step S6, the candidate point setting unit 14 may set the cell S whose frequency is equal to or higher than a certain value in the frequency distribution as a new candidate point Sa of the cell S in which the moving object exists at the first time. can. That is, in this step S6, the candidate point Sa can be reset using the threshold value. In this case, the distribution generation unit 15 is centered on the cell S, which is the new candidate point Sa, with respect to the cell S, which is the new candidate point Sa at the first time, and the cells S around the cell S. A new frequency distribution can be generated by voting for the frequency of the normal distribution. Then, this new frequency distribution can be used as the observation result (observation map) of the moving object at the first time.

Subsequently, the estimation unit 16 calculates the estimation result of the moving object at the first time (step S7: estimation step). This step S7 will be described more specifically. In this step S7, based on the multiplication result of the observation result (frequency distribution P3 or a new frequency distribution) of the first time generated in step S6 and the estimation result (pre-estimation map) of the moving object at the second time. , Calculate the estimation result of the moving object at the first time.

FIG. 7A shows an example of the estimation result of the moving object at the second time. As shown in FIG. 7A, the estimation result P4 of the moving object at the second time has already been generated through the same steps as described above, and the moving object exists for each of the cells S. Probability (estimation) to do is set. In the estimation result P4, the values of the cells S are normalized to be 1 when the values of all the cells S are summed, so that the values of the cells S are the existence probabilities of the moving objects.

In this step S7, each cell S of the frequency distribution P3 as shown in FIG. 6B is multiplied by each cell S of the past estimation result P4 as shown in FIG. 7A. As a result, the estimation result P5 as shown in FIG. 7 (b) is calculated. The multiplication here is performed only between the cells S corresponding to each other. That is, for example, the value of the cell S at the coordinates (3,3) of the frequency distribution P3 is multiplied by the value of the cell S at the coordinates (3,3) of the estimation result P4, and the coordinates (3,3) of the estimation result P5 are multiplied. It is the value of cell S in 3).

As described above, in this step S7, the moving object at the first time is estimated based on the multiplication result of the frequency distribution P3 at the first time generated in the step S6 and the estimation result P4 of the moving object at the second time. The result is calculated. The frequency distribution P3 is generated based on the information of the candidate point Sa. Therefore, in this step S7, the estimation unit 16 calculates the estimation result P5 of the first time based on the candidate point Sa of the first time and the estimation result P4 of the second time.

Further, in this step S7, the estimation unit 16 normalizes the calculated estimation result P5 so that the total value of each cell S is 1. The subsequent object determination can be performed based on the estimation result P5 after this normalization.

On the other hand, the estimation result P5 at the first time calculated as described above is used for multiplication with the frequency distribution at the third time when calculating the estimation result at the third time after the first time. As shown in FIG. 7B, the estimation result P5 includes the cell S having a value of zero. Therefore, among the cells of the estimation result of the third time, the value of the cell calculated by multiplication with the cell S in which the value of the estimation result P5 of the first time is 0 does not depend on the value of the original frequency distribution. It becomes zero (the existence of moving objects is not estimated). In order to avoid such a situation, in this step S7, the estimation result P5 is processed so that the minimum value of the cell S is a value larger than zero (for example, 0.001) (for example, it is constant for all cells S). After being normalized so that the total value of the values in each cell S becomes 1, it is used as a pre-estimation map when calculating the estimation result at the third time. NS.

In the subsequent step, the detection unit 17 determines the object (step S8: detection step). More specifically, the detection unit 17 detects the moving object Q existing in the region R at the first time based on the estimation result P5 at the first time. In this step S8, the detection unit 17 detects the cell S having a certain value or more and the maximum value among the cells S of the estimation result P5 as the cell in which the moving object Q exists.

As described above, in the object detection device 1 and the object detection method according to the present embodiment, the edges of the image of the region R around the vehicle C are extracted, and the edges of the image of the plurality of cells S corresponding to each region of the image are extracted. The edge amount is set for each. Further, the cell S having an edge amount of a certain amount or more is set as a candidate point Sa in which the moving object Q exists at the first time. Further, the estimation result P5 of the moving object Q at the first time is generated based on the candidate point Sa at the first time and the estimation result P4 of the moving object Q at the second time before the first time. Then, the moving object Q is detected based on the estimation result P5. As described above, in these devices and methods, the edge amount of the image having a lower calculation load is used when detecting the moving object Q, so that a high-performance processing device becomes unnecessary. Therefore, an increase in cost can be suppressed when detecting the moving object Q.

Further, in the object detection device 1 according to the present embodiment, the cell S which is the candidate point Sa is centered on the cell S which is the candidate point Sa at the first time and the cells S around the cell S. A distribution generation unit 15 that generates a frequency distribution P3 by voting for a frequency in a normal distribution pattern is provided. Then, the estimation unit 16 calculates the estimation result P5 of the moving object Q at the first time based on the multiplication result of the frequency distribution P3 and the estimation result P4 of the moving object Q at the second time. In this case, the influence of noise is reduced and the detection accuracy is improved.

In particular, in this case, the existence probability of an object having a small relative movement amount (small relative velocity) with the vehicle C is largely estimated between the first time and the second time. That is, for example, with respect to an accessory of the road surface Rs having a relatively high relative speed (stationary) with the moving vehicle C, although it exists in the estimation result P4 at the second time, it is from the second time to the first time. By separating from the region R around the vehicle C in the meantime, it may not exist in the frequency distribution P3 at the first time, and in this case, it is not detected in the estimation result P5. Therefore, in this case, the moving object Q moving in the same direction as the vehicle C is detected more remarkably, and the detection accuracy is improved.

In the object detection device 1, for example, the cell of the estimation result P5 is adjusted in the process of adjusting the spread of the distribution V used for voting the frequency and setting the minimum value of the cell S of the estimation result P5 to a value larger than zero. By adjusting the value to be added to S, the strength of the correlation between the first time and the second time can be adjusted.

On the other hand, in the object detection device 1 according to the present embodiment, the candidate point setting unit 14 selects the cell S whose frequency is equal to or higher than a certain value in the frequency distribution P3 as a new candidate for the cell S in which the moving object Q exists at the first time. It can be set as a point Sa. Further, in the distribution generation unit 15, the cell S, which is the new candidate point Sa, is centered on the cell S, which is the new candidate point Sa at the first time, and the cells S around the cell S. By voting the frequency of the normal distribution, a new frequency distribution P3 can be generated. Then, the estimation unit 16 may calculate the estimation result P5 of the moving object Q at the first time based on the multiplication result of the new frequency distribution P3 and the estimation result P4 of the moving object Q at the second time. In this case, the influence of noise is further reduced and the detection accuracy is further improved.

Further, the object detection device 1 according to the present embodiment uses an image captured by the image pickup device 50 attached to the mirror stay Cs of the vehicle C having a relatively high vehicle height such as a large vehicle. Therefore, the distortion of the image during the projective transformation is reduced, and the detection accuracy is further improved.

The above embodiments have described one aspect of the present invention. Therefore, the present invention is not limited to the above-mentioned one form, and can be arbitrarily modified.

1 ... object detection device, 12 ... edge extraction unit, 13 ... edge amount setting unit, 14 ... candidate point setting unit, 15 ... distribution generation unit, 16 ... estimation unit, 17 ... detection unit, C ... vehicle, Q ... moving object , R ... Area (peripheral area).

Claims (4)

An object detection device for detecting a moving object in the peripheral area of the vehicle while the vehicle is moving.
An edge extraction unit that extracts the edges of the image of the peripheral area acquired at the first time, and an edge extraction unit.
An edge amount setting unit that sets an edge amount based on the edge extraction result for a plurality of cells corresponding to each area of the image, and an edge amount setting unit.
A candidate point setting unit that sets the cell having an edge amount of a certain amount or more as a candidate point of the cell in which the moving object exists at the first time.
An estimation unit that calculates the estimation result of the moving object at the first time based on the candidate point at the first time and the estimation result of the moving object at the second time before the first time. ,
Based on the estimation result at the first time, a detection unit that detects the moving object existing in the peripheral region at the first time, and a detection unit.
An object detection device comprising. The frequency distribution is performed by voting the frequency in a normal distribution pattern so that the cell, which is the candidate point, is the center of the cell, which is the candidate point at the first time, and the cells around the cell. Equipped with a distribution generator that generates
The estimation unit calculates the estimation result of the moving object at the first time based on the multiplication result of the frequency distribution and the estimation result of the moving object at the second time.
The object detection device according to claim 1. The candidate point setting unit sets the cell in which the frequency is constant or higher in the frequency distribution as a new candidate point of the cell in which the moving object exists at the first time.
The distribution generation unit has a normal distribution pattern such that the cell, which is the new candidate point, is centered on the cell, which is the new candidate point at the first time, and the cell around the cell. By voting for the frequency of, a new frequency distribution is generated,
The estimation unit calculates the estimation result of the moving object at the first time based on the multiplication result of the new frequency distribution and the estimation result of the moving object at the second time.
The object detection device according to claim 2. It is an object detection method for detecting a moving object in the peripheral area of the vehicle while the vehicle is moving.
An edge extraction step of extracting the edges of the image of the peripheral region acquired at the first time, and
An edge amount setting step of setting an edge amount in each of the cells based on the extraction result of the edge for a plurality of cells corresponding to each region of the image.
A candidate point setting step of setting the cell having an edge amount of a certain amount or more as a candidate point of the cell in which the moving object exists at the first time.
An estimation step of generating an estimation result of the moving object at the first time based on the candidate point at the first time and the estimation result of the moving object at the second time before the first time. ,
A detection step of detecting the moving object existing in the peripheral region at the first time based on the estimation result at the first time.
An object detection method comprising.

Images