JP2001338281A - Moving object detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving object detector capable of preventing the relative moving direction of an subject from being detected erroneously, when the subject provided by a camera touches the terminal part of an image. SOLUTION: When the subject contained in a reference image provided by a camera 1R touches the terminal part of the reference image, a representative deciding point at the position of that subject is moved from a center of gravity G in the area of the subject to a point GE at the terminal part of a circumscribing rectangle close to the center of the image and on the basis of the moved deciding point, the moving direction of the subject is detected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving object detecting apparatus for detecting a moving direction of an object based on image information of the object obtained by two cameras arranged apart from each other.

[0002]

2. Description of the Related Art A vehicle is equipped with two CCD cameras and has two CCD cameras.
The distance between the target object and the vehicle is detected based on the displacement of images obtained from the two cameras, that is, based on the parallax.
2. Description of the Related Art A crossing object detecting device for detecting a pedestrian in front of 0 to 60 m has been conventionally known (Japanese Patent Laid-Open No. 9-2264).
No. 90).

[0003]

However, in the above-mentioned conventional apparatus, the possibility of collision is determined by directly using the optical flow direction of the object detected based on the image obtained by the camera. There is a problem that the determination accuracy is reduced depending on the relative distance between the host vehicle and the target or the vehicle speed. That is, for example, when the vehicle speed is higher than the moving speed of the object, the optical flow on the image corresponding to the object that is actually moving toward the center of the road is a vector that goes to the outside of the road. Erroneous judgment occurs.

Therefore, in order to accurately detect the movement vector of the object, the area centroid of the object image included in the image obtained by the camera is used as a determination point representing the position of the object image, and the determination point is determined. Is obtained based on the images obtained by the two cameras. When the object image contacts the end of the image obtained by the cameras, the following is performed. Problems arise. The area center of gravity means the center of gravity on the assumption that the target object image is a thin plate having a uniform thickness in a three-dimensional space.

That is, as shown in FIG. 23 (a), an object 101 is stationary or a moving vehicle 102
At the time t = k−2, k− obtained by the camera 103 mounted on the vehicle 102 when moving toward
1, k, the time-series object images 101a, 101b,
101c changes as shown by hatching in FIG. 11B, and the area centroid G moves leftward on the image.
The determination points of the object image included in this image (the image obtained by the right camera 103), that is, the position of the area centroid, and the determination of the object image (not shown) included in the image obtained by the left camera 104 When the parallax is calculated from the position of the point and the distance between the object 101 and the vehicle 102 is obtained from the parallax and the relative movement vector of the object 101 is obtained, the object 101 is actually stationary as shown by a solid line in FIG. Alternatively, a relative movement vector as indicated by a broken line is obtained even though the vehicle is moving in parallel with the traveling direction of the vehicle 102. As a result, there is a possibility that there is no possibility of colliding with the vehicle 102 but erroneous determination is made.

The present invention has been made in view of this point, and when the object image obtained by the camera is in contact with the edge of the image, the relative movement direction of the object is erroneously determined. It is an object of the present invention to provide a moving object detection device that can prevent detection.

[0007]

In order to achieve the above-mentioned object, the invention according to the first aspect is provided by the present invention, in which two
In a moving object detection device that detects a moving direction of the object based on image information of the object obtained by the two cameras, an object image obtained by at least one of the two cameras is an image of the object. The image edge determining means for determining that the image is in contact with the edge of the image, and the target image determined to be in contact with the edge of the image by the image edge determining means is representative of the position of the object image. Determination point moving means for moving the determination point to be moved from the area center of gravity of the object image to a predetermined point close to the center of the image, and movement for detecting the moving direction of the object based on the determination point after the movement. And a direction detecting means.

According to this configuration, for an object image determined to be in contact with the edge of the image, a determination point representing the position of the object image is located at the center of the image from the area center of gravity of the object image. Since the moving direction of the object is detected based on the determined point after the movement to the predetermined point close to the moving point, there is no such a problem that the moving direction is detected from the time series data of the area center of gravity of the object image. , An accurate moving direction can be detected.

[0009] The "predetermined point near the center of the image" may be an end of the object image and the corresponding object image near the center of the image, or the object image and the corresponding object near the center of the image. It is the end of the circumscribed rectangle of the object image. The determination point moving means moves one of the images obtained by the two cameras as a reference image and moves the determination point with respect to an object image in the reference image. In addition, it is preferable that the determination point moving means changes the position data acquired and stored in the past to the position data using the determined determination point after the movement of the object to which the determination point has been moved.

According to a second aspect of the present invention, there is provided a moving object detecting apparatus for detecting a moving direction of an object based on image information of the object obtained by two cameras arranged apart from each other. An object image obtained by at least one of the two cameras,
An image edge determining unit that determines that the edge of the image is in contact with the edge of the image is provided. It is characterized by not being performed.

According to this configuration, since the moving direction is not detected for the object corresponding to the object image determined to be in contact with the end of the image, the moving direction of such an object is erroneously determined. Recognition can be prevented. Further, in the case where a distance detecting device configured to detect a distance to the target based on image information of the target obtained by two cameras disposed apart from each other is configured as described above,
Based on an object image (OBJR2) included in an image obtained by one of the two cameras, a correlation operation is performed on an image obtained by the other camera, and a first obtained result is obtained. Corresponding object image (COBJL
When 2) touches the end of the image, a correlation operation is performed on the image obtained by the one camera based on the first corresponding object image (COBJL2), and the resulting second image is obtained. It is desirable to calculate a parallax based on the second corresponding target object image (COBJR2) and the first corresponding target object image (COBJL2), and calculate a distance to the target object from the calculated parallax.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a view showing a configuration of a moving object detecting device mounted on a vehicle according to a first embodiment of the present invention. This device has a structure capable of detecting far infrared rays. Two infrared cameras 1R and 1L, a yaw rate sensor 5 for detecting a yaw rate of the vehicle, a vehicle speed sensor 6 for detecting a traveling speed (vehicle speed) VCAR of the vehicle, and a brake sensor 7 for detecting an operation amount of a brake. An image processing unit 2 that detects an object such as an animal in front of the vehicle based on the image data obtained by the cameras 1R and 1L and issues an alarm when there is a high possibility of a collision, and issues an audio alert. Speaker 3 and camera 1R or 1L
And a head-up display (hereinafter, referred to as “HUD”) 4 for displaying an image obtained by the above-mentioned method and allowing the driver to recognize an object having a high possibility of collision.

As shown in FIG. 2, the cameras 1R and 1L are arranged in front of the vehicle 10 at positions substantially symmetrical with respect to the central axis of the vehicle 10 in the lateral direction.
The optical axes of L are fixed so that they are parallel to each other and their heights from the road surface are equal. Infrared camera 1R,
1L has the characteristic that the higher the temperature of the object, the higher the output signal level (the higher the luminance).

The image processing unit 2 includes an A / D conversion circuit for converting an input analog signal into a digital signal, an image memory for storing a digitized image signal, a CPU (Central Processing Unit) for performing various types of arithmetic processing, and a CPU for performing arithmetic operations. RAM used to store intermediate data (Ra
ROM (Read Only Memory) for storing programs, tables, maps, etc., executed by the CPU.
mory), an output circuit for outputting a drive signal for the speaker 3, a display signal for the HUD 4, and the like.
1L and the output signals of the sensors 5 to 7 are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided such that a screen 4a is displayed at a position in front of the driver in a front window of the vehicle 10.

FIG. 3 is a flowchart showing the processing procedure in the image processing unit 2. First, the camera 1R, 1
The output signal of L is A / D converted and stored in the image memory (steps S11, S12, S13). The image data stored in the image memory is grayscale image data including luminance information. FIGS. 5A and 5B are diagrams for explaining grayscale images (a right image is obtained by the camera 1R and a left image is obtained by the camera 1L) obtained by the cameras 1R and 1L, respectively. The hatched area is an intermediate gray level (gray) area, and an area surrounded by a thick solid line has a high luminance level (at a high temperature) and an area of an object displayed as white on a screen (hereinafter, “ High brightness area "). In the right image and the left image, the horizontal position of the same object on the screen is shifted, so that the distance to the object can be calculated from the shift (parallax).

In step S14 of FIG. 3, the right image is used as a reference image, and the digital image data is binarized, that is, an area brighter than an experimentally determined luminance threshold value ITH is set to "1" (white) and darkened. A process for setting the area to “0” (black) is performed. FIG. 6 shows an image obtained by binarizing the image shown in FIG. In this figure, the hatched area is black,
The high brightness area surrounded by the thick solid line indicates that it is white. In this specification, an image (left image) paired with the reference image (right image) is referred to as a “search image”.

In the following step S15, a process for converting the binarized image data into run-length data is performed. FIG. 7A is a diagram for explaining this. In FIG. 7A, an area whitened due to binarization is represented by a line L1 at a pixel level.
ＬL8. Each of the lines L1 to L8 has a width of one pixel in the y direction and is actually arranged without a gap in the y direction, but is shown apart for the sake of explanation. Lines L1 to L8 are respectively 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels in the x direction.
Pixels have a length of 8 pixels. The run-length data indicates the lines L1 to L8 by using the coordinates of the start point of each line (the left end point of each line) and the length (the number of pixels) from the start point to the end point (the right end point of each line). It is shown.
For example, the line L3 is (x3, y5), (x4, y5)
And (x5, y5), the run-length data is (x3, y5, 3).

In steps S16 and S17, FIG.
A process of extracting the target object is performed by labeling the target object as shown in FIG. That is, among the lines L1 to L8 converted into run-length data, the lines L1 to L3 having a portion overlapping in the y direction are regarded as one object 1,
The lines L4 to L8 are regarded as one object 2, and object labels 1 and 2 are added to the run-length data. By this processing, for example, the high luminance areas shown in FIG.

In step S18, as shown in FIG. 7 (c), the area gravity center G and area S of the extracted object image and the aspect ratio ASPECT of the circumscribed rectangle shown by the broken line are calculated. The area S is calculated by integrating the length of the run-length data for the same object, and the aspect ratio APECT is calculated as shown in FIG.
The ratio (Dy / Dx) of Dy and Dx shown in (c) is calculated, and the coordinates (xg, yg) of the area centroid G are calculated by the following equation (1).
It is calculated by:

(Equation 1) Here, S is the area, and x (i) and y (i) are the coordinates of the pixels forming the object image. The position of the area center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.

In step S19, the object is tracked over time, that is, the same object is recognized at each sampling period. The time at which the time t as the analog amount is discretized at the sampling period is k, and when the object images 1 and 2 are extracted at the time k as shown in FIG.
The identity judgment is performed between the object images 3 and 4 extracted in +1) and the object images 1 and 2. Specifically, when the following identity determination conditions 1) to 3) are satisfied, the object images 1 and 2 are determined to be the same as the object images 3 and 4, and the object images 3 and 4 are determined. By changing the labels to 1, 2 respectively, time tracking is performed.

1) Object image i (= 1, 1) at time k
The coordinates of the area centroid position in 2) are expressed as (xi (k), y
i (k)), and an object image j at time (k + 1)
The coordinates of the area centroid position of (= 3, 4) are expressed as (xj (k +
1), yj (k + 1)), | xj (k + 1) −xi (k) | <Δx | yj (k + 1) −yi (k) | <Δy. Here, Δx and Δy are allowable values of the moving amount on the image in the x direction and the y direction, respectively.

2) Object image i (= 1, 1) at time k
When the area of 2) is Si (k) and the area of the object image j (= 3, 4) at time (k + 1) is Sj (k + 1), Sj (k + 1) / Si (k) <1 ± ΔS That. Here, ΔS is an allowable value of the area change.

3) Object image i (= 1, 1) at time k
If the aspect ratio of the circumscribed rectangle in 2) is ASPECTi (k) and the aspect ratio of the circumscribed rectangle of the object image j (= 3, 4) at time (k + 1) is ASPECTj (k + 1), ASPECTj (k + 1) / ASPECTi (k) <1
± ΔASPECT. However, ΔASPECT is an allowable value of the change in the aspect ratio.

8 (a) and 8 (b), each object image has a larger size, but the object image 1 is larger.
And 3 satisfy the above-mentioned identity determination condition, and the object images 2 and 4
Satisfies the above-mentioned conditions for determining identity,
Are recognized as corresponding to the object images 1 and 2, respectively. The position coordinates of the area barycenter of each object image recognized in this way are stored in the memory as time-series position data, and are used in subsequent arithmetic processing. Step S1 described above
The processing of 4 to S19 is executed for the binarized reference image (the right image in the present embodiment).

In step S20 of FIG. 3, the vehicle speed sensor 6
Speed VCAR and yaw rate sensor 5 detected by
By reading the detected yaw rate YR and integrating the yaw rate YR with time, the turning angle θr of the host vehicle 10 is obtained.
(See FIG. 18).

On the other hand, in step S31, step S1
In parallel with the processing of S9 and S20, processing of calculating the distance z between the target object and the host vehicle 10 (FIG. 4) is executed. Since this calculation requires a longer time than steps S19 and S20, a period longer than steps S19 and S20 (for example, step S1
1 to about 20 times the execution cycle of S20).

In step S41 of FIG. 4, 1) one of the object images tracked by the binarized image of the reference image (right image) is selected, as shown in FIG. From the right image, an object image OBJR1 (here, the entire area surrounded by a circumscribed rectangle is extracted as an object image)
2) A search area for searching for an image corresponding to the target object image OBJR1 (hereinafter referred to as “corresponding target image”) is set from the search image (left image), and a correlation operation is performed.

More specifically, as shown in FIG. 9B, a search area SL1 is set in the search image according to each vertex coordinate of the object image OBJR1, and the object image O is set in the search area SL1.
Correlation parameter C indicating the degree of correlation with BJR1
(A, b) is calculated by the following equation (2). The correlation parameter C indicates that the smaller the value is, the higher the correlation is. Note that this correlation operation is performed using a grayscale image instead of a binary image. When there is past position data on the same object, an area SL1a narrower than the search area SL1 based on the position data (FIG. 9B)
Is indicated by a broken line) as a search area.

(Equation 2) Here, IR (m, n) is the object image OB shown in FIG.
The luminance value at the position of the coordinates (m, n) in JR1, IL
(A + m−M, b + n−N) is the luminance value at the position of the coordinates (m, n) in the local region LR having the same shape as the target object image OBJR1, based on the coordinates (a, b) in the search region. It is. By changing the coordinates (a, b) of the base point, the position at which the correlation parameter C (a, b) is minimized and the value of the correlation parameter at that time (hereinafter referred to as “minimum value”) CMIN are obtained.

In a succeeding step S42, it is determined whether or not the degree of correlation is high, that is, the minimum value CMI calculated in the step S41.
It is determined whether or not N is equal to or less than a predetermined threshold value CMINTH. When CMIN> CMINTH and the degree of correlation is low, it is determined that the corresponding target image does not exist in the search image. In this case, since the parallax cannot be calculated, the distance is not determined (undefined) (step S49), and the process ends.

On the other hand, if CMIN ≦ CMINTH and the degree of correlation is high, the process proceeds to step S43. Step S
In steps 43 to S45, cases C0 to C45 are determined depending on whether the target object image in the reference image and / or the corresponding target image in the search image is in contact with (hangs over) the end of each image.
The distance is calculated by dividing the case into C4.

FIG. 11 shows an object image OBJ1 in the reference image.
To OBJ3 are in contact with the end of the reference image (hereinafter referred to as “image end”) WR, and the target object image OBJ4 is
Indicates a state where it is not in contact with. As each object image is grasped by run-length data as described above,
It is determined that the target object image in which the coordinates of the starting point indicated by the black circles are present at the image edge WR is in contact with the image edge WR. Similarly, an object image whose end point coordinates obtained by adding the length (run length) to the start point coordinates are present at the image end WR is determined to be in contact with the image end WR.

FIG. 12 shows the case where neither the object image in the reference image nor the corresponding object image in the search image touches the image edge (case C0), and the object image in the reference image is , But the corresponding object image in the search image is in contact with the end of the image (case C2). In case C0 shown in the figure, the object image OBJ in the reference image
R1 and the corresponding object image COB extracted by the correlation operation
Since neither JL1 is in contact with the end of the image, the parallax can be calculated using the area centroid G of both, and the distance z to the object can be calculated. Note that the correlation operation is actually performed on an area slightly larger than the object image OBJR1 so as to include a little background portion.
JL1 is extracted as an area slightly wider than the true corresponding object image OBJL1 having the same shape as the object image OBJR1.

On the other hand, in case C2, the object image OBJR2
Indicates that a true corresponding object image having the same shape as the object image OBJR2, which is not in contact with the image edge, is indicated by a solid line and a broken line.
When it is located as shown as JL2 (the broken line part is
The part which is not actually obtained as an image) is shown in the corresponding object image COBJL extracted by the correlation operation.
2 is extracted in a state where it is in contact with the image end of the search image. Note that regions SL1 and SL2 shown in FIG. 12 are search regions.

FIG. 13A shows an object image O in the reference image.
The case where BJR1 is in contact with the image end and the true corresponding object image OBJL1 in the search image is not in contact with the image end (case C1) is shown. FIG. Although the object image OBJR2 in the image does not touch the end of the image, the true corresponding object image O in the search image
FIG. 13C shows a case where BJL2 is in contact with the edge of the image.
Is that the object image OBJR3 in the reference image is in contact with the image edge, the true corresponding object image OBJL3 in the search image is also in contact with the image edge, and the object image OBJR in the reference image is
A case where R3 is smaller than the true corresponding target object image OBJL3 (case C3) is shown, and FIG. 4D shows that the target object image OBJR4 in the reference image is in contact with the image edge and the corresponding one in the search image. The object image COBJL4 is also in contact with the image edge,
The case where the object image OBJR4 in the reference image is larger than the true corresponding object image OBJL4 (case C4) is shown. In FIG. 13, regions SL1 to SL4 and SR1,
SR2 is a search area.

In case C1, since the true corresponding object image OBJL1 included in the search image is not in contact with the end of the image, correct matching is performed and the corresponding object image COB
JL1 is extracted. Therefore, the object image OBJR1
Then, the parallax is calculated using the area centroid of the corresponding object image COBJL1, and the distance to the object is calculated.

In case C2, the corresponding object image COBJL2 extracted by the correlation operation from the reference image to the search image
And the object image OBJR2 do not have a correct matching relationship. Therefore, in this case, a correlation calculation from the search image to the reference image, that is, a correlation calculation for searching the reference image for the target image corresponding to the corresponding target image COBJL2 is executed. Corresponding target object image COB in the resulting reference image
Since JR2 and the corresponding target object image COBJL2 in the search image have a correct matching relationship, the parallax is calculated using the area centroid of these images, and the distance to the target object is calculated.

In case C3, since correct matching is performed as in case C1, parallax is calculated using the area centroids of the object image OBJR3 and the corresponding object image COBJL3, and the distance to this object is calculated. In case C4, as in case C2, a corresponding target object image COBJL4 extracted by a correlation operation from the reference image to the search image,
The target object image OBJR4 does not have a correct matching relationship. Therefore, also in this case, the correlation calculation from the search image to the reference image, that is, the correlation calculation for searching the reference image for the target image corresponding to the corresponding target image COBJL4 is executed.
Corresponding target object image COBJR in the resulting reference image
4 and the corresponding target object image COBJL4 in the search image have a correct matching relationship. Therefore, the parallax is calculated using the area centroid of these images, and the distance to the target object is calculated.

Returning to FIG. 4, in step S43, it is determined whether or not the object image in the reference image is in contact with the image edge and whether the corresponding object image in the search image is in contact with the image edge, that is, case C0. Is determined, and if the result of the correlation operation corresponds to case C0, step S4 is immediately performed.
Proceeding to 8, the distance is calculated from the parallax, and this processing ends.

When the result of the correlation operation does not correspond to case C0, it is determined whether the object image in the reference image is in contact with the image edge and the corresponding object image in the search image is in contact with the image edge. It is determined whether or not the case corresponds to Case C4. If the result of the correlation operation corresponds to Case C4, the correlation operation from the search image to the reference image is executed as described above (Step S46). Next, whether the degree of correlation is high, that is, the minimum value CMIN of the degree of correlation parameter is equal to the predetermined threshold CM
It is determined whether or not the value is less than or equal to INTH (step S47).
If IN> CMINTH, the process proceeds to step S49.
Proceed to. If CMIN ≦ CMINTH and the degree of correlation is high, the process proceeds to step S48 to calculate the distance.

If the result of the correlation operation does not correspond to case C4 in step S44, that is, if it corresponds to cases C1 to C3, whether the corresponding object image is in contact with the edge of the image,
That is, it is determined whether or not the case C2 applies (step S45). If the case C2 applies, the process proceeds to the step S46. If the case corresponds to the case C1 or C3, the process immediately proceeds from the step S45 to the step S48.

The process of calculating the distance from the parallax in step S48 is performed as follows. As shown in FIG. 14, the area dG (number of pixels) between the area centroid position of the object image OJBR1 and the left end of the image and the distance dL (number of pixels) between the area centroid position of the corresponding object image COBJL1 and the image left end are obtained. The distance z between the host vehicle 10 and the target object is calculated by applying the following equation (3).

(Equation 3) Here, B is the base line length, that is, the distance in the horizontal direction (x direction) between the center position of the image sensor 11R of the camera 1R and the center position of the image sensor 11L of the camera 1L as shown in FIG. F is the lens 12R, 12L
Is the pixel interval in the imaging elements 11R and 11L, and Δd (= dL−dR) is the parallax. The calculation of the distance z is performed as described above.

Returning to FIG. 3, in step S21, the coordinates (x, y) in the image and the distance z calculated by the equation (3) are applied to the following equation (4), and the real space coordinates (X, Y, Z) are obtained. ). Here, the real space coordinates (X, Y, Z) are shown in FIG.
As shown in (a), the position of the middle point of the mounting positions of the cameras 1R and 1L (the position fixed to the host vehicle 10) is the origin O
The coordinates in the image are determined as shown in FIG.
As shown in the figure, the horizontal direction is x,
The vertical direction is defined as y.

(Equation 4)

Here, (xc, yc) represents the coordinates (x, y) on the right image with the real space origin O based on the relative positional relationship between the mounting position of the camera 1R and the real space origin O. This is converted into coordinates in a virtual image in which the center of the image is matched. F is a ratio between the focal length F and the pixel interval p.

When converting the real space coordinates, the object image determined to be in contact with the image edge among the object images in the reference image by the processing in step S31 described above (see FIG. 11). In the example shown in FIG. 17, for the object images OBJ1 to OBJ3), as shown in FIG. 17A, a determination point representing the position of the object image is defined by a circumscribed rectangle closer to the center of the reference image from the area centroid G. And the coordinates after the movement are converted into real space coordinates. That is, with respect to the target object image OBJ1 that is in contact with the left end of the reference image, the determination point is moved to a point GE1 that has the same y coordinate as the area center of gravity G1 and is located at the right end of the circumscribed rectangle. Of the object image OBJ2 in contact with the upper end of the reference image, the determination point is moved to a point GE2 having the same x coordinate as the area center of gravity G2 and located at the lower end of the circumscribed rectangle, and in contact with the right end of the reference image. With respect to the target object image OBJ3, the determination point is moved to a point GE3 that has the same y coordinate as the area gravity center G3 and is located at the left end of the circumscribed rectangle.

For the object whose judgment point has been moved in this manner, the past position data is similarly calculated for the area centroid G.
From the data corresponding to the point GE at the end of the circumscribed rectangle closer to the center of the reference image. By doing so, the time-series data used for calculating the relative movement vector, which will be described later, becomes unified time-series data at the determination point after the movement, and an accurate relative movement vector can be obtained. Note that the determination points are shown in FIG.
As shown in (b), points GE1a, GE2a, and GE3a located at the end of the target object image, not at the end of the circumscribed rectangle.
May be moved.

In step S22, a turning angle correction for correcting a positional shift on the image due to the turning of the host vehicle 10 is performed. As shown in FIG. 18, from time k (k +
During the period up to 1), the own vehicle is turned to the left, for example, by a turning angle θr.
Just turning around, on the image obtained by the camera,
As shown in FIG. 19, since it is shifted in the x direction by Δx, this is a process for correcting this. Specifically, by applying the real space coordinates (X, Y, Z) to the following equation (5), the corrected coordinates (Xr,
Yr, Zr) is calculated. The calculated real space position data (Xr, Yr, Zr) is stored in the memory in association with each object. In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

(Equation 5)

In step S23, as shown in FIG. 20, for the same object, N real space position data (for example, about N = 10) obtained during the period of ΔT after turning angle correction, ie, a time series From the data, the object and the vehicle 10
An approximate straight line LMV corresponding to the relative movement vector is obtained. Specifically, assuming that a direction vector L = (lx, ly, lz) (| L | = 1) indicating the direction of the approximate straight line LMV, a straight line represented by the following equation (6) is obtained.

(Equation 6)

Here, u is a parameter having an arbitrary value, and Xav, Yav, and Zav are the average value of the X coordinate, the average value of the Y coordinate, and the average value of the Z coordinate of the real space position data sequence, respectively. is there. The equation (6) becomes the following equation (6a) if the parameter u is eliminated. (X-Xav) / lx = (Y-Yav) / ly = (Z-Zav) / lz (6a)

FIG. 20 is a diagram for explaining the approximate straight line LMV, in which P (0), P (1), P (2),
, P (N-2) and P (N-1) indicate the time series data after the turning angle correction, and the approximate straight line LMV indicates the average position coordinates Pav (= (Xav, Yav, Za) of the time series data.
v)) is obtained as a straight line that minimizes the average value of the square of the distance from each data point. Here, the numerical value in parentheses attached to P indicating the coordinates of each data point indicates that the larger the value is, the more past the data is. For example, P
(0) indicates the latest position coordinates, P (1) indicates the position coordinates one sample cycle ago, and P (2) indicates the position coordinates two sample cycles ago. D (j), X (j), Y in the following description
The same applies to (j), Z (j), and the like.

More specifically, a vector D (j) = (DX (j), DY (j), DZ) from the average position coordinates Pav to the coordinates P (0) to P (N-1) of each data point.
(J)) = (X (j) -Xav, Y (j) -Yav, Z
An inner product s of (j) -Zav) and the direction vector L is calculated by the following equation (7), and a direction vector L = (lx, ly, lz) that maximizes the variance of the inner product s is obtained. s = lx.DX (j) + ly.DY (j) + lz.DZ (j) (7)

The variance-covariance matrix V of the coordinates of each data point is
Since the eigenvalue σ of this matrix corresponds to the variance of the inner product s, the directional vector L corresponding to the largest eigenvalue among the three eigenvalues calculated from this matrix is the directional vector L to be obtained. In order to calculate eigenvalues and eigenvectors from the matrix of Expression (8), a method known as the Jacobi method (for example, shown in “Numerical Calculation Handbook” (Ohm)) is used.

(Equation 7)

Next, the latest position coordinates P (0) = (X
(0), Y (0), Z (0)) and the position coordinates P (N−1) = (X (N−) before (N−1) samples (before time ΔT).
1), Y (N-1), Z (N-1)) are approximated to the straight line LMV
Correct to the upper position. Specifically, Z in the above equation (6a)
By applying the coordinates Z (0) and Z (N-1), that is, by the following equation (9), the corrected position coordinates Pv
(0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1),
Zv (N-1)).

(Equation 8)

The position coordinates Pv (N−
A relative movement vector is obtained as a vector from 1) toward Pv (0). In this manner, the influence of the position detection error is reduced by calculating the approximate straight line that approximates the relative movement trajectory of the target object with respect to the own vehicle 10 from the plurality (N) of data within the monitoring period ΔT. As a result, it is possible to more accurately predict the possibility of collision with the object.

Returning to FIG. 3, in step S24, it is determined based on the relative movement vector calculated in step S23 whether or not there is a high possibility that the object collides with the host vehicle 10. A process for issuing a warning to the driver by voice or display using HUD3 or HUD4 is executed. After executing step S24, the process returns to step S11.

As described above, in the present embodiment, when an object image in the reference image is in contact with the edge of the image, the determination point representing the position of the object image is determined from the area center of gravity of the object image. Move to a point located at the end of the circumscribed rectangle of the target object image (or a point located at the end of the target object image) near the image end of the shift reference image, and use the determination point after the movement to perform relative movement. Since the movement vector is calculated, the relative movement direction of the object can be accurately detected.

For example, as shown in FIG. 13B, a correlation operation is performed on the search image based on the object image OBJR2 included in the reference image, and the corresponding object image COBJL2 obtained as a result is obtained. , When touching the edge of the image,
Based on the corresponding target object image COBJL2, a correlation operation is performed on the reference image, and a parallax is calculated based on the resulting corresponding target object image COBJR2 and corresponding target object image COBJL2, and the calculated disparity is calculated. Since the distance to the target object is calculated from the parallax, an accurate distance can be calculated even when the corresponding target object image in the search image extracted by the correlation operation touches the image end.

In the present embodiment, the image processing unit 2
An image edge determining unit, a determination point moving unit, and a moving direction detecting unit are configured. More specifically, step S4 in FIG.
Steps S41, S42, S46, S47 in FIG. 4 and step S21 in FIG. 3 correspond to the determination point moving means, and steps S2 in FIG.
Reference numeral 3 corresponds to a moving direction detecting means.

(Second Embodiment) FIG. 21 shows a second embodiment of the present invention.
It is a flowchart of the obstacle detection / warning process according to the embodiment. This processing is obtained by adding step S17a to the processing shown in FIG. 3 and changing step S31 in FIG. 3 to step S31a. FIG.
Is a flowchart showing the distance calculation processing in step S31a. This processing deletes steps S43, S44 and S47 from the processing shown in FIG.
Step S46 is changed to step S46a. Except for the above points, it is the same as the first embodiment.

In step S17a, among the objects extracted in step S17, those whose target image in the reference image is in contact with the image edge are excluded from monitoring.
In step S31a, a distance calculation process shown in FIG. 22 is executed. When it is determined in step S42 that the degree of correlation is high, it is determined whether or not the corresponding target image obtained by the correlation calculation is in contact with the image end of the search image (step S4).
5) If not in contact, the distance is calculated from the parallax (step S48). On the other hand, when the corresponding target object image is in contact with the end of the image, a process of removing this target object from the monitoring target is performed (step S46a), and the present process ends immediately.

As described above, in the present embodiment, the object whose object image is in contact with the image end of the reference image and the object whose corresponding object image extracted by the correlation operation is in contact with the image end of the search image are monitored. Therefore, it is possible to prevent the moving direction of the object and the distance to the object from being erroneously recognized.

In this embodiment, the image processing unit 2
The image edge determining means is constituted, and more specifically, step S45 in FIG. 22 corresponds to the image edge determining means. Note that the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, in the present embodiment, an infrared camera is used as a camera for obtaining an object image. However, for example, as shown in JP-A-9-226490, a television camera that can detect only ordinary visible light is used. Is also good. However, by using an infrared camera, the extraction processing of an animal, a running vehicle, and the like can be simplified, and even an arithmetic device having a relatively low arithmetic capability can be realized.

[0062]

As described above in detail, according to the first aspect of the present invention, with respect to an object image determined to be in contact with an edge of an image, a determination point representing the position of the object image is determined. ,
From the area centroid of the object image, it is moved to a predetermined point near the center of the image, and the moving direction of the object is detected based on the determination point after the movement. It is possible to detect an accurate moving direction without any trouble as in the case of detecting the moving direction.

According to the second aspect of the present invention, since the moving direction is not detected for the object corresponding to the object image determined to be in contact with the edge of the image, the moving object is not detected. It is possible to prevent erroneous recognition of the moving direction.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving object detection device according to an embodiment of the present invention.

FIG. 2 is a view for explaining a mounting position of the camera shown in FIG. 1;

FIG. 3 is a flowchart illustrating a procedure of a process performed by the image processing unit in FIG. 1;

FIG. 4 is a flowchart illustrating a distance calculation process of FIG. 3 in detail;

FIG. 5 is a diagram showing a gray scale image obtained by an infrared camera with hatching applied to a halftone portion;

FIG. 6 is a diagram showing a black area with hatching to explain an image obtained by binarizing a grayscale image.

FIG. 7 is a diagram for explaining a conversion process to run-length data and labeling.

FIG. 8 is a diagram for explaining time tracking of an object.

FIG. 9 is a diagram for explaining a search target object image in a right image and a search region set in a left image.

FIG. 10 is a diagram for explaining a correlation calculation process for a search area.

FIG. 11 is a diagram illustrating a case where an object image contacts an end of an image.

FIG. 12 is a diagram for explaining a case where an object image in a search image is in contact with an end of the image.

FIG. 13 is a diagram for explaining four cases where an object image is in contact with an end of an image.

FIG. 14 is a diagram illustrating a method of calculating parallax.

FIG. 15 is a diagram for explaining a method of calculating a distance from parallax.

FIG. 16 is a diagram illustrating a coordinate system according to the present embodiment.

FIG. 17 is a diagram for explaining movement of a determination point.

FIG. 18 is a diagram for explaining turning angle correction.

FIG. 19 is a diagram illustrating a displacement of an object position on an image caused by turning of the vehicle.

FIG. 20 is a diagram for describing a calculation method of a relative movement vector.

FIG. 21 is a flowchart illustrating a procedure of processing by an image processing unit according to the second embodiment of the present invention.

FIG. 22 is a flowchart showing the distance calculation processing of FIG. 21 in detail.

FIG. 23 is a diagram for explaining a problem of the conventional method.

FIG. 24 is a diagram for explaining a problem of the conventional method.

[Explanation of symbols]

1R, 1L infrared camera (camera) 2 Image processing unit (image end determining means, determining point moving means, moving direction detecting means) 5 Yaw rate sensor 6 Vehicle speed sensor

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification FI theme coat ゛ (Reference) G08G 1/16 G08G 1/16 C (72) Inventor Hiroshi Hattori 1-4-1, Chuo, Wako-shi, Saitama Inside the Honda R & D Co., Ltd. (72) Inventor Masato Watanabe 1-4-1 Chuo, Wako-shi, Saitama F-term in the Honda R & D Co., Ltd. 5B057 AA16 BA02 CA12 CA16 CB13 CB16 DA07 5H180 AA01 CC02 CC04 LL01 LL07 5L096 BA04 CA05 HA04 JA24

Claims (2)

[Claims] 1. A moving object detecting apparatus for detecting a moving direction of an object based on image information of the object obtained by two cameras arranged apart from each other, wherein: Image edge determining means for determining that an object image obtained by at least one of the cameras is in contact with an edge of the image, and an object determined by the image edge determining means to be in contact with the edge of the image For the object image, a judgment point moving means for moving a judgment point representing the position of the object image from a center of area of the object image to a predetermined point close to the center of the image, and a judgment point after the movement. And a moving direction detecting means for detecting a moving direction of the object based on the moving object. 2. A moving object detecting apparatus for detecting a moving direction of an object based on image information of the object obtained by two cameras arranged apart from each other, wherein: The image processing apparatus further includes image edge determining means for determining that an object image obtained by at least one of the cameras touches an edge of the image, and the image edge determining means determines that the image touches an edge of the image. A moving object detection device, wherein a moving direction is not detected for an object corresponding to the object image.