JPH04153785A - Environment recognizer for mobile vehicle - Google Patents

Abstract

PURPOSE:To correct the total shift of an image by extracting the image information corresponding to a prescribed reference line at a prescribed time interval, secures the correlation of image information at least between two screens, and correcting an input image. CONSTITUTION:The convolutions of the line data Pa and Pb extracted by a pilot line data extracting parts 2 and 4 are calculated. Then it is decided whether the dispersion obtained from the convolution has a peak or not. If not, a vertical position shift is judged between the images 11 and 12 where the data Pa and Pb are extracted. A shift extent calculation part 3 detects a shift between both images in the pitch direction as a shift extent of the images. In addition, the continuous images are corrected by the shift correction processing for extraction of a road structure. The corrected images are outputted to a road structure extracting part 8. As a result, the images can be corrected in regard of the shift occurred in the pitch direction.

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial Field of Application) The present invention relates to an environment recognition device for a moving vehicle that recognizes the structure of a traveling road through image processing.

(Prior art) Conventionally, when recognizing the structure of a driving road, it is difficult to reduce the influence of noise from an image taken at a certain moment and then use that image for recognition. ,
Even if a single noise appears in the processed image, it can be removed.

(Problem to be Solved by the Invention) However, in the above-mentioned conventional example, if the image for recognizing the running road structure is captured by an image input unit provided in the vehicle, the image may be distorted due to the vibration in the pitch direction of the vehicle. There is a problem that blurring occurs, and unless this blurring is corrected, it is not possible to accurately recognize the structure of the driving route.

(Means for Solving the Problems) The present invention has been made for the purpose of solving the above-mentioned problems, and includes the following configuration as a means for solving the above-mentioned problems.

That is, the environment recognition device for a moving vehicle is equipped with an image input means for recognizing the outside world, and extracts image information corresponding to a predetermined reference line provided on the input image from the input image at predetermined time intervals. means for correlating the image information between at least two screens, and means for correcting the input image based on the correlation.

(Function) With the above configuration, it is possible to correct image blur caused by vehicle shaking and create an image in which the road structure is easy to recognize.

(Embodiment) Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram of an environment recognition device for a moving vehicle (hereinafter referred to as the device), which is an embodiment of the present invention.

In FIG. 1, an image input unit 1 includes a COD camera (not shown), etc., and inputs the driving road environment (mainly road surface condition) in front of the vehicle as image information. Then, it is sent to the pilot line data extraction section 2 and the coordinate transformation section 5 as image information at time t.

The pilot line data extraction unit 2 sets a pilot line, which will be described later, on the input image and extracts image data corresponding to the pilot line. The extracted image data is stored in the line buffer 3a of the movement amount calculation section 3.

On the other hand, the coordinate transformation section 5 sends the image information input from the image input section 1 at time t-1 to the pilot line data extraction section 4. Like the pilot line data extraction section 2, the pilot line data extraction section 4 sets a pilot line on the input image information and extracts image data corresponding to the pilot line. The extracted image data is stored in the line buffer 3a of the movement amount calculation section 3.

The movement amount calculation section 3 correlates the image information from the pilot line data extraction sections 2 and 4 stored in the line buffer 3a, and detects a shift in the pitch direction of the image based on the result. Then, the detection result is sent to the coordinate transformation section 5.

The coordinate transformation section 5 performs coordinate transformation of the image, that is, image displacement correction, based on the image displacement information from the movement amount calculation section 3. The corrected image is stored as it is in the coordinate conversion unit 5 as image information for the next movement amount calculation, and
1 to one of the plurality of buffers 7 selected by the selector 6.
Each frame is transferred.

The buffer 7 sequentially outputs the frames to the road structure extraction unit 8 as soon as the buffer is overflowing with the transferred image information of the plurality of frames. The road structure extraction section 8 analyzes the image information that is continuously sent, extracts the road structure, and outputs the result to the main control section 9.

The main control section 9 controls the entire device and recognizes the road environment based on the road structure extraction result from the road structure extraction section 8.

Next, image shift correction in this embodiment will be described in detail.

FIG. 2 schematically shows image shift correction.

In FIG. 2, image 11 is image information at time t stored in pilot line data extraction section 2, and image 12 is image information at time t-1 stored in pilot line data extraction section 4. It is information. Three pilot lines are placed on each image to detect image features.
1a and 12a are set, one of which is provided at the center of the image, and the other two are provided one each in the left and right directions of the image, avoiding the edges of the image. The edge of the image is avoided here because image data at the edge is strongly affected by noise, so the positions of the other two pilot lines are convenient for extracting the line at the tangled edge. Select a location.

Note that the position of the pilot line once set is fixed regardless of subsequent processing.

Accordingly, image shift correction processing will be described with reference to the schematic diagram shown in FIG. 2 and the flowchart shown in FIG. 3.

In step S1 of FIG. 3, the pilot line data extracting unit 2 extracts image data (which is converted into line data
In step S2, the pilot line data extraction unit 4 extracts image data (referred to as a) corresponding to the three pilot lines 12a from the image at time t-1.
This is referred to as line data pb).

In step S3, the pilot line data extraction unit 2,
A convolution of the line data Pa and Pb extracted in step S4 is calculated, and in step S4 it is determined whether or not there is a peak in the variance obtained as a result of the convolution. If it is determined that there is a peak in the dispersion, it is determined that the correlation between the line data Pa and Pb is strong, and that the entire image has a certain characteristic movement. Therefore, the process proceeds to step S8, and the movement amount calculation section 3 stores the image 11 in FIG. 2, which is the image information at time t, in the coordinate conversion section 5 as it is without correcting it. Needless to say, this image becomes the image information at time t-1 in the next correlation calculation process.

On the other hand, if it is determined in step S4 that there is no peak in the variance, images 11.12 from which line data pa and Pb are extracted
It is determined that there was a vertical positional shift between them. Assuming that this is a blurring of the image due to the shaking of the vehicle, the movement amount calculation section 3 detects the deviation of the image in the pitch direction in step S5, and uses it as the image movement amount.

In the next step S6, the coordinate conversion section 5 performs image correction, that is, vertical coordinate position conversion, based on the amount of image movement calculated in step S5. Then, in step S7, the corrected image (image 13 in FIG. 2) is stored in the coordinate conversion section 5 as image information at time t-1 in the next correlation calculation process, and the image is stored in the coordinate conversion section 5 as image information at time t-1 in the next correlation calculation process. transfer to one,
After the storing process in step S7 or step S8, in step S9, it is determined whether or not to continue the image shift correction process.
Return to 1.

In the above-described shift correction process, the continuous images for extracting the road structure are corrected, and the corrected images are output to the road structure extraction section 8 (see FIG. 1). In the end, these images become images that the road structure extraction section 8 can easily perform inter-image calculations on.

As explained above, according to this embodiment, a pilot line is set on two consecutive images, and the corresponding vertical image data is extracted and correlated with each other, thereby improving the image data. Since deviations in the pitch direction can be detected,
This has the effect that the shift of the entire image can be easily corrected based on the obtained shift.

Further, by inputting the image that has been corrected for deviation as a continuous image for extracting the road structure, the recognition processing in the road structure recognition unit can be performed more accurately and easily.

Note that the present invention is not limited to the above embodiments (
A pilot line may be set for two or more consecutive images, vertical image data may be extracted, and a correlation may be established between the image data. By doing so, the correlation between image data becomes clearer, and it becomes possible to extract relatively gentle image blur.

In addition, while taking the correlation between the line data Pa and Pb, it also determines whether there is randomness in the blurring of the image, and if the blurring is random, it is assumed that the blurring is not caused by the shaking of the vehicle, and the image is corrected. You may choose not to do so.

(Effects of the Invention) As explained above, according to the present invention, by extracting image information corresponding to a reference line on an image and detecting deviations in the vertical direction, image information corresponding to deviations in the pitch direction can be detected. This has the effect that correction can be performed.

[Brief explanation of drawings]

Fig. 1 is a block diagram of an environment recognition device for a moving vehicle which is an embodiment of the present invention; Fig. 2 is a diagram schematically showing image shift correction; and Fig. 3 is a diagram showing image shift correction processing. This is a flowchart. In the figure, 1... image input section, 2.4... pilot line data extraction section, 3... movement amount calculation section, 3a...
Line buffer, 5... Coordinate conversion section, 6... Selector, 7... Buffer, 8... Road structure extraction section, 9.
...Main control unit, 11...Image at time t, 12...Image at time t-1, lla, 12a-pilot line,
13... This is an image after misalignment correction. Nauria;

Claims (1)

[Scope of Claims] An environment recognition device for a moving vehicle equipped with an image input means for recognizing the outside world, comprising: extracting image information corresponding to a predetermined reference line provided on the input image from an input image for a predetermined period of time; An environment recognition device for a moving vehicle, comprising: means for extracting the image information at intervals; means for correlating the image information between at least two screens; and means for correcting the input image based on the correlation. .