JP2001084497A - Position detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position detector for a vehicle which implements correction of excluding an influence of pitching of the vehicle with a simple constitution to accurately detect a position. SOLUTION: Plural objects are extracted from a picture obtained by a camera, and an average value of extents of displacement of the positions in a (y) direction (perpendicular direction) of the plural objects is calculated as a pitching flow Δyp (S41). When the pitching flow Δyp is larger than a prescribed threshold ΔyTH, (y) coordinates of individual objects are corrected by the pitching flow Δyp (S42 and S43).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a position detecting device for detecting a position of an object based on an image obtained by an image pickup device mounted on a vehicle.

[0002]

2. Description of the Related Art Conventionally, a peripheral monitoring device which mounts an image pickup device on a vehicle, detects a position of an object from an image obtained by the image pickup device, and determines a possibility of collision between the object and the vehicle has been conventionally used. Proposed. In such a device, since the imaging device is mounted on a moving vehicle, an optical flow (hereinafter, referred to as “pitching flow”) occurs in an image obtained by the imaging device fixed to the vehicle by pitching the vehicle. Since this pitching flow does not occur due to the movement of the object (or the traveling of the vehicle), when the position of the object is detected from the image obtained by the imaging device, the processing excluding the effect of the pitching flow is performed. Otherwise, there is a problem that the displacement of the detection position becomes large.

[0003]

Therefore, it is conceivable to provide a vehicle height sensor for detecting the height (vehicle height) of the vehicle from the road surface and correct the detection position based on the vehicle height detected by this sensor. However, in that case, a vehicle height sensor is required, and the structure of the device is complicated.

SUMMARY OF THE INVENTION The present invention has been made in view of this point, and a vehicle position detecting device capable of performing accurate position detection by realizing correction with a simple configuration excluding the effect of pitching of the vehicle. The purpose is to provide.

[0005]

According to a first aspect of the present invention, a position of an object in a real space coordinate system is detected based on an image obtained by an image pickup means mounted on a vehicle. And a correction amount calculating unit that extracts a plurality of objects from an image obtained by the imaging unit, and calculates a pitching correction amount according to a position displacement amount in the height direction of the plurality of objects. Correction means for correcting the positions of the plurality of objects on the image in accordance with the pitching correction amount.

According to this configuration, a plurality of objects are extracted from the image obtained by the imaging means, and a pitching correction amount is calculated in accordance with the amount of positional displacement of the plurality of objects in the height direction. Since the position of the target object on the image is corrected in accordance with the pitching correction amount, even when a position displacement in the height direction in the image occurs due to the pitching of the vehicle, without using a vehicle height sensor or the like. Exact influence can be detected with a simple configuration and accurate position detection can be performed.

It is preferable that the correction amount calculating means calculates an average value of the positional displacement amounts for the plurality of objects and sets the average value as the pitching correction amount. Is larger than a predetermined threshold, it is desirable to execute the correction based on the pitching correction amount. When the average value of the positional displacement amounts for a plurality of objects is large, the positional displacement amount on the image due to factors other than pitching (running of the vehicle or movement of the object in real space) is determined on the image by pitching. Since it is considered that the amount is small compared to the position displacement amount, an effective pitching correction amount can be obtained with a simple configuration by this configuration.

[0008]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a vehicle periphery monitoring device including a position detection device according to an embodiment of the present invention. This device includes two infrared cameras 1R and 1L capable of detecting far infrared rays, and A yaw rate sensor 5 for detecting a yaw rate of the vehicle, and a traveling speed (vehicle speed) of the vehicle
A vehicle speed sensor 6 for detecting a VCAR, a brake sensor 7 for detecting an operation amount of a brake, and an object such as an animal in front of the vehicle are detected based on image data obtained by these cameras 1R and 1L, and a collision is detected. The image processing unit 2 that issues an alarm when the possibility of collision is high, the speaker 3 that issues an alarm by voice, the image obtained by the camera 1R or 1L, and the driving of an object with a high possibility of collision And a head-up display (hereinafter, referred to as “HUD”) 4 for allowing a person to recognize.

As shown in FIG. 2, the cameras 1R and 1L are arranged in front of the vehicle 10 at positions substantially symmetrical with respect to the central axis of the vehicle 10 in the lateral direction.
The optical axes of L are fixed so that they are parallel to each other and their heights from the road surface are equal. Infrared camera 1R,
1L has the characteristic that the higher the temperature of the object, the higher the output signal level (the higher the luminance).

The image processing unit 2 includes an A / D conversion circuit for converting an input analog signal into a digital signal, an image memory for storing digitized image signals, a CPU (Central Processing Unit) for performing various types of arithmetic processing, and a CPU for performing arithmetic operations. RAM used to store intermediate data (Ra
ROM (Read Only Memory) for storing programs, tables, maps, etc., executed by the CPU.
mory), an output circuit for outputting a drive signal for the speaker 3, a display signal for the HUD 4, and the like.
1L and the output signals of the sensors 5 to 7 are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided such that a screen 4a is displayed at a position in front of the driver in a front window of the vehicle 10.

FIG. 3 is a flowchart showing the procedure of the object position detection process in the image processing unit 2. First, the output signals of the cameras 1R and 1L are A / D converted and stored in the image memory (steps S11 and S12). , S13).
The image data stored in the image memory is grayscale image data including luminance information. FIG. 5 (a)
(B) is a diagram for explaining a grayscale image (a right image is obtained by the camera 1R and a left image is obtained by the camera 1L) obtained by the cameras 1R and 1L, respectively, and is a hatched area. Is a region of an intermediate gradation (gray), and a region surrounded by a thick solid line has a high luminance level (at a high temperature) and is a region of an object displayed as white on a screen (hereinafter, referred to as a “high luminance region”). ). In the right image and the left image, the horizontal position of the same object on the screen is shifted, so that the distance to the object can be calculated from the shift (parallax).

In step S14 of FIG. 3, the right image is used as a reference image, and the image signal is binarized, that is, an area brighter than an experimentally determined luminance threshold ITH is set to "1".
(White) and the dark area is set to “0” (black). FIG. 6 shows an image obtained by binarizing the image shown in FIG.
This figure shows that the hatched area is black, and the high-luminance area surrounded by the thick solid line is white.

In the following step S15, a process for converting the binarized image data into run-length data is performed. FIG. 7A is a diagram for explaining this. In FIG. 7A, an area whitened due to binarization is represented by a line L1 at a pixel level.
ＬL8. Each of the lines L1 to L8 has a width of one pixel in the y direction and is actually arranged without a gap in the y direction, but is shown apart for the sake of explanation. Lines L1 to L8 are respectively 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels in the x direction.
Pixels have a length of 8 pixels. The run-length data indicates the lines L1 to L8 by using the coordinates of the start point of each line (the left end point of each line) and the length (the number of pixels) from the start point to the end point (the right end point of each line). It is shown.
For example, the line L3 is (x3, y5), (x4, y5)
And (x5, y5), the run-length data is (x3, y5, 3).

In steps S16 and S17, FIG.
A process of extracting the target object is performed by labeling the target object as shown in FIG. That is, among the lines L1 to L8 converted into run-length data, the lines L1 to L3 having a portion overlapping in the y direction are regarded as one object 1,
The lines L4 to L8 are regarded as one object 2, and object labels 1 and 2 are added to the run-length data. By this processing, for example, the high luminance areas shown in FIG.

In step S18, as shown in FIG. 7C, the area gravity center G and area S of the extracted object image and the aspect ratio ASPECT of the circumscribed rectangle indicated by the broken line are calculated. The area S is calculated by integrating the length of the run-length data for the same object, and the coordinates of the area centroid G are calculated along the x-coordinate and the x-direction of a line that bisects the area S along the y-direction. Is calculated as the y-coordinate of the line bisected by
ECT is the ratio Dy / D between Dy and Dx shown in FIG.
x is calculated. The position of the center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.

In step S19, the object is tracked over time, that is, the same object is recognized in each sampling cycle. The time at which the time t as the analog quantity is discretized at the sampling period is k, and when the objects 1 and 2 are extracted at the time k as shown in FIG.
The identity of the objects 3 and 4 extracted in 1) and the objects 1 and 2 is determined. Specifically, the following identity determination conditions 1)
When 3) are satisfied, it is determined that the objects 1 and 2 are the same as the objects 3 and 4, and the objects 3 and 4 are respectively 1 and 2.
The time tracking is performed by changing the label to "."

1) Object i at time k (= 1, 2)
The coordinates of the position of the center of gravity on the image of (xi (k),
yi (k)) and the object j at time (k + 1)
The coordinates of the position of the center of gravity of the image of (= 3, 4) are represented by (xj (k
+1), yj (k + 1)), | xj (k + 1) −xi (k) | <ΔxM | yj (k + 1) −yi (k) | <ΔyM. Here, ΔxM and ΔyM are allowable values of the amount of displacement on the image in the x direction and the y direction, respectively. In addition,
The allowable value ΔyM in the y direction is set in consideration of the amount of displacement on the image due to pitching of the vehicle 10.

2) Object i at time k (= 1, 2)
Is the area on the image Si (k), and the area of the object j (= 3, 4) on the image at time (k + 1) is Sj
When (k + 1), Sj (k + 1) / Si (k) <1 ± ΔS. Here, ΔS is an allowable value of the area change.

3) Object i at time k (= 1, 2)
Let ASPECTi (k) denote the aspect ratio of the circumscribed rectangle of ASPECTj (k + 1), and let ASPECTj (k + 1) denote the aspect ratio of the circumscribed rectangle of the object j (= 3, 4) at time (k + 1). ) <1
± ΔASPECT. However, ΔASPECT is an allowable value of the change in the aspect ratio.

8 (a) and 8 (b), the size of each object on the image is larger in FIG. 8 (b), but the objects 1 and 3 The objects 3 and 4 are recognized as the objects 1 and 2, respectively, since the objects 2 and 4 satisfy the above-described conditions for the sameness. The position coordinates (of the center of gravity) of each object recognized in this way are stored in a memory as time-series position data,
It will be used for the subsequent arithmetic processing. In step S20 of FIG. 3, the pitching correction process shown in FIG.
When the optical flow on the image occurs due to the pitching, the processing of correcting the position of the object on the image is executed.

FIG. 9 shows a case where the number N of extracted objects is N = 3 as an example.
FIG. 7 is a diagram illustrating a relationship between M1 to VM3, a pitching vector Vp which is an optical flow generated by pitching, and displacement vectors VM1p to VM3p on an image of each object when pitching is not performed. O in this figure
BJOLDi and OBJNOWi are respectively the object i
(= 1, 2, 3) indicates the previous position (the position one sample cycle earlier) and the current position.

In step S41 of FIG. 4, the following equation (1)
, The amount of displacement in the y-direction (hereinafter referred to as “pitching flow”) Δyp on the image caused by pitching is calculated.

(Equation 1) Here, yiNOW and yiOLD are the y-coordinate of the center of gravity of the current object i (i = 1 to N) and the y-coordinate of the center of gravity of the previous object i (one sample cycle before), respectively, where N is
The number of objects being extracted. The pitching flow Δyp of the equation (1) is calculated by an evaluation function E expressed by the following equation (2).
Is calculated as the y coordinate displacement amount Δy that minimizes, and corresponds to the average value of the y direction displacement amounts per unit time (per one sample period in the present embodiment) on the images of N objects. I do. The amount of displacement in the y direction on the image (the amount of displacement of the object in the y direction when there is no pitching) due to the traveling of the vehicle 10 or the movement of the object is smaller than the amount of displacement caused by pitching. , And the optical flow (pitching vector Vp) generated by pitching can be uniformly approximated for all objects without depending on the position on the image.
The pitching flow Δyp is defined as Δy that minimizes the evaluation function E of Expression (2).

(Equation 2)

Next, it is determined whether or not the absolute value of the pitching flow Δyp is greater than a predetermined threshold value ΔyTH (step S42). If | Δyp |> ΔyTH, it is determined that the pitching of the vehicle 10 is large, and Object i (i
= 1 to N) pitching flow Δy to y coordinate yiNOW
Pitch correction is performed by adding p (step S43). The predetermined threshold value ΔyTH is set to an appropriate value by an experiment.

As described above, the displacement in the y direction on the image (the displacement in the y direction of the object when there is no pitching) due to the movement of the vehicle 10 or the object is smaller than the displacement caused by the pitching. Therefore, when the absolute value of the pitching flow Δyp calculated by the equation (1) exceeds a predetermined threshold value ΔyTH, Δyp is regarded as the displacement amount generated by pitching, and the y coordinate is corrected. Things. OBJNOWip of FIG.
The position indicated by (i = 1, 2, 3) corresponds to the corrected position of each object.

On the other hand, in step S42, | Δyp | ≦ ΔyT
If it is H, it is determined that the effect of pitching can be ignored, and the process immediately proceeds to step S44, where the y coordinate is not corrected. In step S44, the current value yiNOW of the y coordinate of each object is set to the previous value yiOLD, and the process ends. Thereby, in the processing after one sample period, the current value in the current processing is used as the previous value.

According to the processing of FIG. 4, the displacement Δyp in the y direction, that is, the height direction due to the pitching of the vehicle 10 is calculated based on the image itself obtained by the camera 1R, whereby the y coordinate of the object is calculated. Since the correction is performed, the pitching correction can be realized with a simple configuration, and as a result, the position of the object in the real space coordinate system described below can be detected more accurately. Note that the processing of steps S14 to S20 in FIG. 3 described above is executed for the binarized reference image (the right image in the present embodiment).

In step S31 of FIG. 3, an object image R2 in the left image corresponding to the object image R1 included in the right image is obtained as shown in FIG.
The distance dR between TR and the center of gravity of the object image R1 and the distance dL between the center line LCTR of the left image and the center of gravity of the object image R2 are calculated, and these are applied to the following equation (3) to calculate the distance from the host vehicle 10. The distance z to the object is calculated. Step S31
Takes a longer time than the processing in steps S19 and S20, and is therefore executed at a longer cycle than steps S19 and S20 (for example, about three times the execution cycle of steps S11 to S20).

(Equation 3)

Here, B is the base line length, that is, the horizontal direction (x) between the center position of the image sensor 11R of the camera 1R and the center position of the image sensor 11L of the camera 1L as shown in FIG.
Direction) (the distance between the optical axes of both cameras), F is the lens 1
The focal lengths of 2R and 12L and p are the image pickup devices 11R and 11R.
This is the pixel interval within L, and Δd (= dR + dL) is the amount of parallax. F is the ratio between the focal length F and the pixel interval p.

In step S21, the coordinates (x,
y) and the distance z calculated by the equation (3) are expressed by the following equation (4).
To convert to coordinates (X, Y, Z) in the real space coordinate system. Here, the real space coordinate system XYZ is shown in FIG.
As shown in (a), the position of the middle point of the mounting positions of the cameras 1R and 1L (the position fixed to the host vehicle 10) is the origin O
The coordinate system in the image is defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin, as shown in FIG.

(Equation 4)

Here, (xc, yc) represents the coordinates (x, y) on the right image, based on the relative positional relationship between the mounting position of the camera 1R and the real space origin O. Are converted into coordinates in a virtual image in which the center of the image is matched. F is a ratio between the focal length F and the pixel interval p.

As described above, in the present embodiment, the vehicle 10
Is calculated from the image data obtained by the camera 1R to correct the y-coordinate of the target object. Accurate position data can be obtained excluding the effect of pitching.

The image processing unit 2 determines the possibility of collision between the subject and the host vehicle 10 based on the position information of the target in the real space calculated by the processing of FIG. When the driver has high performance, a warning is issued to the driver via the speaker 3 and the HUD 4.

In the present embodiment, the image processing unit 2 constitutes a position detecting device. More specifically, steps S16 to S19 in FIG. 3 and step S41 in FIG. 4 correspond to the correction amount calculating means. Steps S42 and S43 correspond to the correction means. The present invention is not limited to the embodiment described above, and various modifications are possible. For example, in the above-described embodiment, the average value of the displacement amounts of the y-coordinates of all the extracted objects is set as the pitching flow Δyp, but y of a part of the extracted objects (however, at least two or more objects) is used. The pitching flow Δyp may be calculated from the coordinate displacement.

In this embodiment, an infrared camera is used as the image pickup means.
As shown in Japanese Patent Publication No. 0, a television camera capable of detecting only ordinary visible light may be used. However, the use of an infrared camera can simplify the process of extracting an animal or the like or a running vehicle or the like, and can realize even an arithmetic unit having a relatively low arithmetic capability.

[0035]

As described above in detail, according to the present invention, a plurality of objects are extracted from an image obtained by the imaging means, and pitching is performed in accordance with the amount of positional displacement of the plurality of objects in the height direction. The correction amount is calculated, and the positions of the plurality of objects on the image are corrected in accordance with the pitching correction amount. Therefore, even when a position displacement in the height direction in the image occurs due to the pitching of the vehicle, the vehicle can be corrected. Accurate position detection can be performed with a simple configuration without using a high sensor or the like and removing its influence.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a vehicle periphery monitoring device including a position detection device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a mounting position of an infrared camera.

FIG. 3 is a flowchart illustrating a procedure of processing for detecting a position where an object is extracted.

FIG. 4 is a flowchart of a pitching correction process of FIG. 3;

FIG. 5 is a diagram showing a gray scale image obtained by an infrared camera with hatching applied to a halftone portion;

FIG. 6 is a diagram showing a black area with hatching to explain an image obtained by binarizing a grayscale image.

FIG. 7 is a diagram for explaining a conversion process to run-length data and labeling.

FIG. 8 is a diagram for explaining time tracking of an object.

FIG. 9 is a diagram for explaining pitching correction.

FIG. 10 is a diagram for explaining a method of calculating parallax.

FIG. 11 is a diagram for explaining a method of calculating a distance from parallax.

FIG. 12 is a diagram illustrating a coordinate system according to the present embodiment.

[Explanation of symbols]

 1R, 1L infrared camera (imaging means) 2 Image processing unit (correction amount calculating means, correcting means)

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Masato Watanabe 1-4-1 Chuo, Wako-shi, Saitama Prefecture Inside Honda R & D Co., Ltd. (72) Inventor Hiroshi Hattori 1-4-1 Chuo, Wako-shi, Saitama Honda R & D Co., Ltd.

Claims (1)

[Claims] 1. A position detecting device for detecting a position of an object in a real space coordinate system based on an image obtained by an image pickup device mounted on a vehicle, wherein a plurality of objects are detected from the image obtained by the image pickup device. And a correction amount calculating means for calculating a pitching correction amount in accordance with a positional displacement amount in the height direction of the plurality of objects, and a position of the plurality of objects on the image in accordance with the pitching correction amount. A position detection device comprising: a correction unit configured to perform correction.