JP4479174B2 - Object detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an object detection device that is useful for monitoring a moving body such as a vehicle and that can be loaded on the moving body.
The object detection device of the present invention can be effectively used when a vehicle has functions such as a lane keeping function and a forward vehicle following function.
[0002]
[Prior art]
[Patent Document 1]
Japanese Patent Laid-Open No. 7-25286
[Patent Document 2]
JP-A-8-188104
[Patent Document 3]
Japanese Patent Laid-Open No. 11-53692
[Patent Document 4]
JP-A-8-329393
[Patent Document 5]
JP 2002-277540 A
[0003]
As a conventional technique (reliability improvement technique) for detecting a vehicle by using a so-called edge histogram relating to an image obtained by imaging the vehicle, for example, there are the following.
(1) According to Patent Document 1 above. The movement amount of the center of gravity of the area and the aspect ratio of the area are verified.
(2) According to Patent Document 2 above. Verify the amount of movement of boundary candidates.
(3) According to Patent Document 3 above. The consistency with the optical flow in the candidate area is verified.
[0004]
Other related technologies include, for example, the following.
(4) What is described in Patent Document 4 above. It is evaluated and verified whether the object existence area obtained as the image analysis result and the area detected by the laser radar overlap each other.
(5) What is described in Patent Document 5 above. When evaluating images at different times, a measured value (or an estimated value) of the distance to the object is used.
[0005]
[Problems to be solved by the invention]
In the method of detecting the vehicle by the edge histogram, it is affected by the background edge, and as a result, erroneous detection may occur. In particular, such false detections are often seen when vertical edges are easily detected, such as when poles, roadside lights, road signs, guardrails, or other pillars are visible in the background. .
Further, when the host vehicle moves greatly due to steering or the like, it is difficult to recognize an object (image analysis processing) with the methods of Patent Literature 1 and Patent Literature 2.
In addition, when the host vehicle moves greatly due to steering or the like, it is often difficult to obtain the optical flow even in other conventional techniques.
[0006]
The present invention has been made to solve the above-described problems, and an object of the present invention is to greatly improve the detection accuracy and reliability of an object detection device useful for forward monitoring of a traveling vehicle.
A further object of the present invention is to alleviate, reduce, or eliminate the CPU bottleneck in the object detection device by greatly reducing the amount of arithmetic processing required for object recognition (image analysis processing). is there.
However, it is sufficient that the above-mentioned individual objects are achieved individually by at least one of the individual means of the present invention, and the individual inventions of the present application simultaneously solve all the above-mentioned problems. It does not necessarily guarantee that there is a means that can be solved.
[0007]
[Means for Solving the Problem, Action, and Effect of the Invention]
In order to solve the above problems, the following means are effective.
That is, the first means of the present invention is an object detection apparatus that can be loaded on a moving body, and includes a distance measurement means for measuring a distance to an object to be detected, and an entire image of an assumed detection target range. Image capturing means for capturing, analysis target area limiting means for limiting the analysis target area in the entire image based on the distribution state of the distance, distances measured by the distance measuring means at a plurality of processing times, and image capturing means Information holding means for storing a set of information obtained by processing an image captured by Partial images that make up the analysis target area , Where the x-axis is taken in the horizontal direction, the x-axis position is taken as the x-coordinate, the y-axis is taken in the vertical direction, and the y-axis position is taken as the y-coordinate, the x-axis direction relative to the pixel information of the partial image Differential image generating means for generating a horizontal differential image and a vertical differential image of a partial image using differential processing in the horizontal direction and the vertical direction that is the y-axis direction, and each x coordinate in the horizontal differential image on the y-axis A one-dimensional horizontal boundary candidate histogram is obtained by calculating the absolute value sum of pixel values on a parallel line as a function of x, and for each y coordinate in the vertical differential image, the absolute value sum of the pixel values on the line parallel to the x axis is obtained. Histogram creation means for obtaining a one-dimensional vertical boundary candidate histogram as a function of y, a pixel value in a vertical differential image having a certain y coordinate for each x coordinate, and a vertical boundary candidate at that y coordinate Intermediate histogram creating means for obtaining a sum obtained by changing the y-coordinate of the product with the value of the strogram as an intermediate histogram which is a function of x; Left and right determination histogram creation means for obtaining horizontal boundary position candidate selection means for obtaining the x coordinate taking the extreme value of the horizontal boundary candidate histogram as a horizontal boundary candidate position, and each horizontal boundary candidate position obtained by the horizontal boundary position candidate selection means With respect to the horizontal boundary candidate position, the maximum value of the absolute value of the left / right determination histogram in the minute region including the horizontal boundary candidate position is set as the first evaluation value, and each horizontal boundary candidate position is set to the left according to the sign of the left / right determination histogram value. A first evaluation value calculating means for classifying into a horizontal boundary candidate position and a right horizontal boundary candidate position; and a first evaluation value calculating means Symmetry which calculates | requires the 2nd evaluation value regarding the left-right symmetry of the pair of the left horizontal boundary candidate position and right horizontal boundary candidate position in each class classified into two from the value of the horizontal boundary candidate histogram for each horizontal boundary candidate position In order to convert information into information corresponding to a common distance based on the first distance and the second distance to the object at different first processing times and second processing times held by the evaluation value calculation means and the information holding means The scale ratio adjusting means for obtaining the scale ratio of the lengths in the x-axis direction and the y-axis direction and converting the horizontal boundary candidate histogram value and the x coordinate, and the scale ratio adjusted by the scale ratio adjusting means for each time A collation means for obtaining a collation degree between the horizontal boundary candidate histograms for each horizontal boundary candidate position and calculating as a third evaluation value of each horizontal boundary candidate position, and for each horizontal boundary candidate position, a horizontal boundary candidate histogram Based on the gram value, the first evaluation value, the second evaluation value, and the third evaluation value, the reliability calculation means for calculating the reliability that the horizontal boundary candidate position matches the contour of the object, and the reliability It has a contour estimation means for estimating the contour of an object.
[0008]
According to the above configuration, object detection is not performed at all positions in the image. First, a candidate for an object is detected using a distance measurement unit, and a candidate region based on the candidate by an analysis target region limiting unit. (: Analysis target area) can be generated. For this reason, the amount of calculation processing required for object recognition (image analysis processing) can be significantly reduced. Further, in the evaluation, the reliability of object detection can be improved by comparing the images in the analysis target area at different times using the multiple time information holding means.
[0009]
Therefore, according to the above configuration, the boundary of the object can be detected from the image at high speed and with high reliability.
Further, according to the above configuration, the CPU overhead can be greatly reduced by rationalizing the calculation (image analysis processing), so that the CPU bottleneck in the object detection apparatus can be alleviated, reduced or eliminated.
In particular, from the viewpoint of reducing CPU overhead, it is desirable to use a wave sensor such as a millimeter wave radar or a laser radar as the distance measuring means.
[0010]
[0011]
For example, when following a forward vehicle, in addition to the distance to the forward vehicle, the position and width of the forward vehicle in the horizontal direction (particularly the left-right direction) are very important in recognizing the forward vehicle. become. For example, in such a case, the above-described one-dimensional histogram is created at least in the horizontal direction by using the above-described histogram creating means, and based on the one-dimensional histogram, the position coordinates of the contour candidate of the preceding vehicle at least in the horizontal direction. Can be identified at high speed. Therefore, according to such a method, since the analysis target area is narrowly limited and the calculation for creating the one-dimensional histogram is relatively simple, the horizontal position of the preceding vehicle is very fast. And width can be specified.
Of course, calculation of the contour coordinates of the object based on such a one-dimensional histogram may be performed not only in the horizontal direction but also in any direction.
[0012]
[0013]
As the “image processing data”, the above one-dimensional histogram may be used, the above contour candidates may be used, or the above partial image (image of the analysis target area) may be used as it is.
According to such a configuration, even when the distance between the object detection device and the subject (detection target) dynamically varies, the scale ratio adjusting means adversely affects the image processing data (: Inconsistency) can be canceled out, so that the above data collation process between a plurality of times can be performed at high speed, simply and accurately.
[0014]
When following a forward traveling vehicle, the image of the forward traveling vehicle is usually left-right symmetric. Therefore, by using such symmetry, the erroneous detection due to the background as described above can be greatly reduced, and the reliability of object detection can be greatly improved.
By the above means of the present invention, the above-mentioned problem can be effectively or rationally solved.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described based on specific examples. However, the present invention is not limited to the following examples.
(Example)
FIG. 1 is a system configuration diagram illustrating the main configuration of an object detection apparatus 100 according to an embodiment of the present invention. The object detection apparatus 100 mainly includes a CPU 101, ROM 102, RAM 103, IF (input / output interface 104), camera 107 (imaging means), millimeter wave radar 108 (distance detection means), ECU (electronic control unit 110), and the like. It is configured.
However, for example, a stereo camera or the like can be used as an alternative means of the distance detection means (millimeter wave radar 108).
[0016]
FIG. 2 illustrates a basic control procedure of the object detection apparatus 100 according to the embodiment of the present invention. One of the major features of the object detection apparatus 100 is that it uses a moving stereo method, and the multiple time information holding means for realizing it is realized by step 200, step 210, step 220, and the like. . In step 210, synchronous processing (waiting processing) is executed so that step 220 is periodically executed. This period is hereinafter referred to as Δt.
That is, this object detection apparatus 100 first executes step 200, and then periodically executes steps 210 to 290 with the above-mentioned period Δt, whereby each means (distance) of the object detection apparatus 100 of the present invention. Measuring means, image capturing means, analysis target area limiting means, contour candidate extracting means, multiple time information holding means, reliability calculating means, contour estimating means) are configured to function efficiently in conjunction with each other.
Hereinafter, each step from Step 230 to Step 290 will be described in detail with reference to FIGS.
[0017]
[1] Determination of processing boundaries
FIG. 3 is an image diagram schematically showing how the processing area is determined in step 230.
When executing step 230, first, the presence / distance and orientation of an object are detected by a millimeter wave radar or the like, and a processing area (analysis target area) in the image is determined from the detected distance and orientation (FIG. 3). ). The upper left coordinate of the processing area is (x0, y0), and the width and height are w and h, respectively. In this embodiment, it is assumed that the coordinate origin in the image coincides with a point where the camera optical axis intersects the imaging surface.
The processing area (analysis target area) can also be determined by detecting the traveling lane. Further, the entire screen may be used as the processing area. Hereinafter, the symbol I (x, y) represents pixel brightness information (captured pixel information) at the image coordinates (x, y) in this embodiment.
[0018]
[2] Detection of object boundary candidates
FIG. 4 illustrates a processing procedure of a subroutine for executing object boundary candidate detection (step 240).
A well-known Prewitt filter is applied to the processing area of the captured image I (x, y) to create a horizontal differential image IDX (x, y) and a vertical differential image IDY (x, y). The coefficients of the Prewitt filter are shown below. Multiplying the Prewitt filter means multiplying the 3 × 3 pixels adjacent to each pixel position in the processing region by the following coefficients, summing them, and calculating the function value in the (x, y) coordinates: Means seeking.
[0019]
(Prewitt filter coefficient)
(Horizontal differential) (Vertical differential)
-1 0 1 -1 -1 -1
-1 0 1 0 0 0
-1 0 1 1 1 1
The above process corresponds to the process of step 410 in FIG.
[0020]
Further, the following vertical / horizontal line enhancement processing is performed.
[Expression 1]
Idx (x, y) = IDX (x, y) (| IDX (x, y) | >> | IDY (x, y) |),
Idx (x, y) = 0 (| IDX (x, y) | ≦ | IDY (x, y) |) (1)
[Expression 2]
Idy (x, y) = IDY (x, y) (| IDY (x, y) | >> | IDX (x, y) |),
Idy (x, y) = 0 (| IDY (x, y) | ≦ | IDX (x, y) |) (2)
By this enhancement process (step 420), it is possible to effectively and effectively select noise, leading candidates, etc., and to highlight only leading candidates.
[0021]
Next, for each vertical line of the horizontal differential image Idx, the absolute value sum of the pixel values of the pixels on the same vertical line is obtained, and a horizontal boundary candidate histogram H is created (FIG. 5A).
FIG. 5 is an image diagram schematically showing a creation form of an object boundary histogram (V, H). Here, symbol Idx (x, y) is an enhanced horizontal differential image (processed pixel information), and symbol Idy (x, y) is an enhanced vertical differential image (processed pixel information). It is.
[0022]
Similarly in the vertical direction, the absolute value sum of the pixel values of the pixels on the same horizontal line is obtained for each horizontal line of the vertical differential image Idy, and a vertical boundary candidate histogram V is created (FIG. 5B). . Thereafter, the following filters are applied to the histograms V and H respectively.
[Equation 3]
Fv = {1 2 1},
Fh = {1 4 5 4 1} (3)
[0023]
Next, the local peak position of each histogram is detected. If the detected peak value is equal to or greater than a predetermined threshold, the position is set as a horizontal boundary candidate and a vertical boundary candidate position, and the coordinates of each candidate are represented as xHi and yVj. Further, the reliability of each boundary candidate is defined by the following equation (4).
(symbol)
xHi: Position of boundary candidate (position in the horizontal axis direction; 1 ≦ i ≦ n)
yVj: Position of boundary candidate (position in the vertical axis direction; 1 ≦ j ≦ m)
[Expression 4]
Ch (xHi) = α · H (xHi),
Cv (yVj) = β · V (yVj) (4)
[0024]
However, the peak position of the horizontal boundary candidate histogram is set as the horizontal boundary candidate position xHi, and the peak position of the vertical boundary candidate histogram is set as the vertical boundary candidate position yVj. Α and β are appropriate normalization coefficients. That is, the reliability of each position is directly proportional to each value of the histogram.
Note that the scale of the boundary candidate histogram (V, H) itself may be standardized so that α = β = 1. Ch (xHi) may be defined as another monotonically increasing function for H (xHi). The same applies to the relationship between Cv (yVj) and V (yVj).
[0025]
[3] Horizontal boundary judgment
An intermediate histogram HM is created from the vertical differential image Idy and the vertical boundary histogram V.
FIG. 6 is a flowchart illustrating a processing procedure of a subroutine for executing the horizontal boundary left / right determination (step 250), and FIG. 7 is a histogram relating to the left / right boundary of the object (intermediate histogram HM, left / right determination histogram LR). It is an image figure showing typically the creation form.
[0026]
First, in step 620, an intermediate histogram HM (FIG. 7B) is created according to the following equation (5).
[Equation 5]
HM (x) = _{y = y0} Σ ^{y0 + h-1} (V (y) · Idy (x, y)) (5)
In step 640, the created intermediate histogram HM is subjected to a differential filter of the following equation (6) to create a left / right determination histogram LR (FIG. 7C).
[Formula 6]
Fdif = {− 1 −1 0 1 1} (6)
[0027]
Next, in step 660, “Cl: likelihood of left boundary” and “Cr: likelihood of right boundary” defined by the following equation (7) are used as selection criteria, as shown in the following equation (8): From xHi}, the left boundary candidate position {xHil} and the right boundary candidate position {xHir} are separated (selected), respectively.
[Expression 7]
Cl (xHi) = max (LR (∀x))
(XHi−mlr1 ≦ x ≦ xHi + mlr2),
Cr (xHi) = max (−LR (∀x))
(XHi−mlr2 ≦ x ≦ xHi + mlr1),
(Example: mlr1 = 4, mlr2 = 1) (7)
[Equation 8]
{XHil} = {xHi | Cl (xHi)> 0},
{XHir} = {xHi | Cr (xHi)> 0} (8)
[0028]
[4] Verification of boundary candidates by moving stereo
FIG. 8 is a conceptual diagram for explaining a basic processing concept (collation method) in the moving stereo method in step 260.
[0029]
FIG. 8A shows an image at time t1 (: t = t1) and a candidate area (analysis target area) at that time, and FIG. 8B shows an image and candidate area at time t2 (: t = t2 = t1 + Δt). Shown in Here, the distance to the vehicle at time t1 is Z1, and the distance to the vehicle at time t2 is Z2.
Further, the horizontal boundary candidate histogram at time t1 is H1, and the horizontal boundary candidate histogram at time t2 is H2. Further, the upper left coordinate of the processing area in the image at time t1 is (x01, y01), and the upper left coordinate of the processing area at time t2 is (x02, y02).
(Other symbols)
R1 ... Domain of histogram H1
R2 ... Domain of histogram H2
H1 '... Horizontal boundary histogram obtained by scaling H1
[0030]
First, a position corresponding to the range R1 of the horizontal boundary candidate histogram H1 in the horizontal boundary candidate histogram H2 is obtained. Here, the left end position of the region R1 is x0R1, and the width is wR1. However, as a method for determining the region R1, for example, a method according to the following procedure may be considered.
(Procedure 1) Based on the distance Z1 detected by the radar, the width W in the image corresponding to 2.5 m is determined.
(Procedure 2) Then, an area (: definition area [xr−W, xr + W]) widened to the left and right around the azimuth xr detected by the radar is defined as R1.
[0031]
Hereinafter, the analysis process procedure in the horizontal direction will be described in detail.
FIG. 9 is a flowchart exemplifying a processing procedure of a subroutine for executing a collation process (: step 260) based on the moving stereo method.
[4-1] Horizontal analysis processing procedure
This process corresponds to step 810 to step 840.
(A) First, in step 810, the histogram H1 'is generated by multiplying the histogram H1 by Z1 / Z2. At this time, the left end position x0R1 ′ and the width wR1 ′ of the region R1 ′ in H1 ′ corresponding to the region R1 of H1 are determined based on the following equation (9).
[0032]
The right side of the following equation (9) is multiplied by Z1 / Z2, respectively. This is because the size (width, etc.) of the same object in the captured image is inversely proportional to the distance to the object. It is designed based on a very basic geometrical relationship (distance and size correspondence) between times.
[Equation 9]
x0R1 '= x0R1 · Z1 / Z2
wR1 ′ = wR1 · Z1 / Z2 (9)
[0033]
(B) Next, in step 820, the region R2 that is most similar to the region R1 ′ in the histogram H1 ′ is obtained by correlation calculation in the histogram H2, and for example, as shown in the following equation (10), H1 Find the corresponding position of 'and H2. The correlation calculation is performed using the sum SAD (x) of absolute values of differences between histogram values for each element in the region R1 ′.
However, hereinafter, argmin is a symbol for deriving an argument that minimizes the function value of the function written on the right side of the symbol. For example, argminSAD (x) means an argument (independent variable) x that gives a minimum value to the function SAD. In addition, the appropriate range (search range) of x when searching for the minimum value matches the definition area unless otherwise specified.
Similarly, argmax is a symbol indicating an argument that maximizes the function value of the function written on the right side of the symbol.
[0034]
[Expression 10]
x0R2 = argminSAD (x),
SAD (x) = _{i = 0} Σ ^wR1 ′ ^-1 | H2 (x + i)
−H1 ′ (x0R1 ′ + i) | (10)
Assuming that the left end position of the region R2 is x0R2, the position x1 ′ in the histogram H1 ′ corresponding to the position x2 of the histogram H2 follows the following equation (11).
## EQU11 ##
x1 '= x2-x0R2 + xR01' (11)
[0035]
(C) Next, in step 830, a difference histogram Hd between the histogram H2 and the histogram H1 ′ is created by the difference calculation for each element. At this time, when the position x1 ′ corresponding to the position x2 of H2 is outside H1 ′, the value of the element is set to 0.
[Expression 12]
Hd (x2) = | H2 (x2) −H1 ′ (x1 ′) |
(0 ≦ x1 ′ ≦ wR1 ′),
Hd (x2) = 0 (other than above) (12)
[0036]
(D) Next, in Step 840, the reliability Cms2 (xHi) by the moving stereo verification at the position x2 is calculated as in Expression (13) using Expression (14).
[Formula 13]
Cms2 (xHi) = Cms (xHi−ofms) (xHi ≦ xc),
Cms2 (xHi) = Cms (xHi + ofms) (other than above) (13)
[Expression 14]
Cms (x2) = H2mean-[{H2mean + _{i = xc} Σ ^x2 Hd (i)}
/ | X2-xc |],
H2mean = { _{x = 0} Σ ^wR2-1 H2 (x)} / WR2 (14)
[0037]
Here, xc is a position that separates the left and right boundaries, and for example, xc = x0R2 + wR2 / 2 may be set. However, the reliability of the boundary candidate is obtained by using the reliability at the position (± ofms = ± 2) shifted by the width of two pixels inward from the position xHi of the boundary candidate as exemplified in the following equation (13). Redefine.
[0038]
The reason for such redefinition is as follows.
In the reliability Cms (x2), the reliability is obtained by integrating Hd within the range [xc, x2]. The position of xHi is the peak of the vertical edge (horizontal boundary candidate) histogram H, that is, the value of Big position. Further, since Hd is the difference between H2 and H1 ′, at a position xHi where the value of H1 ′ is large, it is easily affected by the scale conversion error of H1 ′. As a result, the value of Hd becomes an unreasonably large value. Cases are likely to occur. In order to mitigate the influence of the error, a required comparison operation is performed at a position where the value of the histogram H1 ′ itself is relatively small (a position where the influence of the error is relatively small) that is slightly inside (ofms = 2). It is desirable to perform this, and according to such a calculation technique, a more reliable calculation result can be obtained.
The same tendency can be seen in the vertical direction.
[0039]
[4-2] Vertical analysis processing procedure
Note that the vertical analysis process is also executed in steps 860 to 890 in FIG. 9 in substantially the same manner as the horizontal analysis process described above. However, similarly in the vertical direction, a function obtained as a result of the analysis processing is Cms3 (yVj). That is, the function Cms3 (yVj) is a function on the vertical direction side corresponding to the function Cms2 (xHi).
[0040]
[5] Verification of symmetry
Next, a method for calculating the reliability of the boundary candidate using the symmetry in the horizontal direction will be described. First, the symmetry of the region between the left boundary candidate position xHil and the right boundary candidate position xHir is defined by the following equation (15).
[Expression 15]
Csym (xHil, xHir) = MAX Fsym (xHil + a, xHir + b)
((A, b) = (0,0), (-1,0), (0,1)),
Fsym (X1, X2) = _{i = 0} Σ ^{(X2-X1) / 2} {H (X1 + i) × H (X2-i)}
... (15)
[0041]
However, here, the symbol MAX is used as a symbol for selecting (a, b) that maximizes the function value of Fsym from any one of the above three (a, b).
Note that performing multiplication {H (X1 + i) × H (X2-i)} in Σ {} on the right side of the second expression of Expression (15) is effective for problems such as noise. However, for example, a difference calculation may be performed in Σ {} on the right side. However, in that case, various changes such as changing the MAX operation to the MIN operation are necessary.
[0042]
Then, the reliability of the left boundary candidate position xHil and the right boundary candidate position xHir is defined by the following equation (16).
[Expression 16]
Clsym (xHil) = max (Csym (xHil, ∀r))
(Rε {xHir}),
Crsym (xHir) = max (Csym (∀l, xHir))
(L∈ {xHil}) (16)
FIG. 10 illustrates a subroutine processing procedure (step 310 to step 375) for obtaining the first calculation result (Clsym (xHil)) of the equation (16), and FIG. 16 illustrates a processing procedure of a subroutine for obtaining the second calculation result (Crsym (xHir)) of 16). For example, following these processing procedures, step 270 (: left / right symmetry verification processing) can be effectively realized.
[0043]
[6] Calculation of reliability (step 280 in FIG. 2)
Based on the respective reliability Ch, Cv, Cl, Cr, Cms2, Cms3, Clsym, and Crsym calculated by the above processing, the final reliability (Cleft, Cright, Ctop, Cbtm) are defined by the following equations (17) to (20).
[Expression 17]
Cleft (xHil) = a1 · Ch (xHil)
+ A2 · Cl (xHil)
+ A3 · Cms2 (xHil)
+ A4 · {Clsym (xHil)} ^1/2 ... (17)
[Formula 18]
Cright (xHir) = a1 · Ch (xHir)
+ A2 · Cr (xHir)
+ A3 · Cms2 (xHir)
+ A4 · {Crsym (xHir)} ^1/2 ... (18)
[Equation 19]
Ctop (yVjt) = b1 · Cv (yVjt)
+ B2 · Cms3 (yVjt),
{YVjt} = {yVj | y0 <yVj <y0 + h / 2} (19)
[Expression 20]
Cbtm (yVjb) = b1 · Cv (yVjb)
+ B2 · Cms3 (yVjb),
{YVjb} = {yVj | y0 + h / 2 <yVj <y0 + h} (20)
[0044]
The position of the upper boundary candidate yVjt and the position of the lower boundary candidate yVjb are the vertical boundary candidate positions {yVj} and the upper boundary candidate position {yVjt} in the middle of the candidate area (analysis target area) as described above. The lower boundary candidate position {yVjb} is divided into two groups.
Further, all of a1, a2, a3, a4, b1, and b2 are appropriate adjustment coefficients, and may be about 1 in this embodiment.
(Setting example of each adjustment factor)
[Expression 21]
a1 = a2 = 1,
a3 = a4 = 1,
b1 = b2 = 1 (21)
[0045]
[7] Determination of boundary (step 290 in FIG. 2)
The contour coordinates of the object to be finally output are defined as follows. That is, the coordinates of each boundary candidate that gives the maximum value to each reliability described above are the left end contour coordinates: xHl, the right end contour coordinates: xHr, the upper end contour coordinates: yVt, and the lower end contour coordinates: yVb.
[Expression 22]
xHl = argmax (Cleft (xHil)),
xHr = argmax (Cright (xHir)),
yVt = argmax (Ctop (yVjt)),
yVb = argmax (Cbtm (yVjb)) (22)
[0046]
FIG. 12 is an image diagram illustrating a specific example of contour coordinates that are output information of FIG. 1. According to the object detection apparatus 100 of the present embodiment, the same calculation results as in the image diagram of FIG. 12 are obtained in the actual experimental results using the vehicle.
That is, for example, the present invention can be effectively realized by the embodiment as described above.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram illustrating a main configuration of an object detection apparatus 100 according to an embodiment of the present invention.
FIG. 2 is a general flowchart illustrating a basic control procedure of the object detection apparatus 100 according to the embodiment of the present invention.
FIG. 3 is an image diagram schematically showing how a processing area is determined in step 230;
FIG. 4 is a flowchart illustrating a processing procedure of a subroutine that executes detection of an object boundary candidate (step 240).
FIG. 5 is an image diagram schematically showing a form of creating an object boundary histogram (V, H).
FIG. 6 is a flowchart illustrating a processing procedure of a subroutine for executing a horizontal boundary left / right determination (step 250).
FIG. 7 is an image diagram schematically showing a creation form of histograms (HM, LR) related to the left and right boundaries of an object.
FIG. 8 is a conceptual diagram for explaining a basic processing concept (collation method) in the moving stereo method in step 260;
FIG. 9 is a flowchart illustrating a processing procedure of a subroutine for executing a collation process based on the moving stereo method (step 260).
FIG. 10 is a flowchart (first half) illustrating a processing procedure of a subroutine for executing a left / right symmetry verification process (step 270).
FIG. 11 is a flowchart (second half) illustrating the processing procedure of a subroutine for executing the left / right symmetry verification processing (step 270).
12 is an image diagram illustrating a specific example of contour coordinates that are the output result of FIG. 1;
[Explanation of symbols]
100 ... Object detection device
101 ... CPU
102… ROM
103 ... RAM
104 ... IF (input / output interface)
107: Camera (imaging means)
108 ... millimeter wave radar (distance detection means)
110 ... ECU (Electronic Control Unit)
I (x, y): Pixel brightness at image coordinates (x, y) (image information of captured pixels)
Idx (x, y) ... Horizontal differential image with enhanced processing (processed pixel information)
Idy (x, y)… vertical differential image with enhanced processing (processed pixel information)
H… Horizontal boundary histogram
V ... Vertical boundary histogram
HM ... Intermediate histogram
LR: Histogram for left / right determination
Z1 ... Distance to the object at time t = t1
H1 ... Horizontal boundary histogram at time t = t1
R1 ... Domain of histogram H1
Z2: Distance to the object at time t = t2
H2 ... Horizontal boundary histogram at time t = t2
R2 ... Domain of histogram H2
H1 '... Horizontal boundary histogram obtained by scaling H1
xHi: Position of boundary candidate (position in the horizontal axis direction; 1 ≦ i ≦ n)
xHil ... Left boundary candidate position
xHir ... Right boundary candidate position
yVj: Position of boundary candidate (position in the vertical axis direction; 1 ≦ j ≦ m)
yVjt ... Position of the upper boundary candidate
yVjb ... Lower boundary candidate position
xHl ... Contour coordinates (final output position of the left edge of the object)
xHr: Contour coordinates (final output, right edge position of the object)
yVt ... Contour coordinates (final output position of the upper end of the object)
yVb ... Contour coordinates (final output position of the lower end of the object)

Claims (1)

An object detection device that can be loaded on a moving object,
Distance measuring means for measuring the distance to the object to be detected;
Image capturing means for capturing an entire image of an assumed detection target range;
Analysis target region limiting means for limiting the analysis target region in the whole image based on the distribution state of the distance;
Information holding means for storing the distance measured by the distance measuring means at a plurality of processing times and information obtained by processing the image captured by the image capturing means as a set;
In the partial image constituting the analysis target area, when the x-axis is taken in the horizontal direction, the x-axis position is taken as the x-coordinate, the y-axis is taken in the vertical direction, and the y-axis position is taken as the y-coordinate, Differential image creation means for creating a horizontal differential image and a vertical differential image of the partial image using differential processing in the horizontal direction that is the x-axis direction and the vertical direction that is the y-axis direction with respect to the pixel information of the partial image;
For each x coordinate in the horizontal differential image, the absolute sum of pixel values on a line parallel to the y axis is obtained as a function of x to obtain a one-dimensional horizontal boundary candidate histogram, and for each y coordinate in the vertical differential image, a histogram creating means for obtaining a one-dimensional vertical boundary candidate histogram by obtaining an absolute value sum of pixel values on a line parallel to the x-axis as a function of y;
For each x-coordinate, the sum obtained by changing the y-coordinate of the product of the pixel value in the vertical differential image having a certain y-coordinate and the value of the vertical boundary candidate histogram in that y-coordinate is a function of x Intermediate histogram creation means to obtain as a histogram;
a left / right determination histogram creating means for differentiating the intermediate histogram which is a function of x with respect to the x coordinate to obtain a left / right determination histogram;
Horizontal boundary position candidate selecting means for obtaining an x coordinate taking an extreme value of the horizontal boundary candidate histogram as a horizontal boundary candidate position;
For each horizontal boundary candidate position obtained by the horizontal boundary position candidate selecting means, the maximum absolute value of the left and right determination histogram values in a minute region including the horizontal boundary candidate position is set as a first evaluation value, and the left and right determination is performed. First evaluation value calculating means for classifying each horizontal boundary candidate position into a left horizontal boundary candidate position and a right horizontal boundary candidate position according to the sign of the histogram,
A second evaluation value related to the left-right symmetry of the pair of the left horizontal boundary candidate position and the right horizontal boundary candidate position in each of the two classes classified by the first evaluation value calculation means is calculated for each horizontal boundary candidate position. Symmetry evaluation value calculation means to be obtained from the value of the boundary candidate histogram;
In order to convert the information into information corresponding to a common distance based on the different first processing time held by the information holding means, the first distance to the object at the second processing time, and the second distance, a scale ratio adjusting means for obtaining a scale ratio of lengths in the x-axis direction and the y-axis direction, and converting a value of the horizontal boundary candidate histogram and an x coordinate;
The collation degree between the horizontal boundary candidate histograms at each time, which is scale ratio adjusted by the scale ratio adjusting means, is obtained for each horizontal boundary candidate position, and is calculated as a third evaluation value of each horizontal boundary candidate position. Matching means;
For each horizontal boundary candidate position, based on the value of the horizontal boundary candidate histogram, the first evaluation value, the second evaluation value, and the third evaluation value, the horizontal boundary candidate position becomes the contour of the object. A reliability calculation means for calculating the matching reliability;
An object detection apparatus comprising: an outline estimation unit that estimates an outline of the object based on the reliability.

Images