JPH01273113A - Image processor for moving car - Google Patents

Abstract

PURPOSE:To detect a misalignment of an image input means to a moving car without using a misalignment detection use sensor by detecting the misalignment of the image input means, based on an image which has been inputted to the image input means. CONSTITUTION:An image sensor 2 is provided in front of front glass of an automobile so that external world information can be inputted as an image, and also, so that a projecting part being a reference point provided on the center position of the upper part of the front end of the automobile always enters into a visual field. An image signal which has been outputted from the image sensor 2 is inputted to an external world recognition use image processing means 40, and also, inputted to a misalignment detecting means. In the misalignment detecting means 4, a position of a reference point of a car body in an image is detected, and the misalignment quantity between its reference point and a target position is calculated. Subsequently, from the misalignment quantity, a motor rotation angle is determined by a correcting means 6, and the image sensor 2 is corrected to a state being free from an initial misalignment quantity.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to an image processing device for a moving vehicle such as an automobile or a robot, which is equipped with an image input means for recognizing the outside world.

(Prior art) For example, Japanese Patent Application Laid-Open No. 81-240307 discloses that a moving vehicle is imaged by a camera serving as an image input means installed at a predetermined point, and the image input to the camera is processed to locate the moving vehicle. A technique has been disclosed in which the travel of a moving vehicle is controlled based on the calculation results.

In addition, unlike the above case, an image input means is installed in the moving vehicle, the image input means captures an image of the outside world of the moving vehicle (the actual driving environment), and the image input to the image input means is used to For example, technology has been available for processing images to recognize the state of the outside world and controlling the movement of vehicles based on the recognition results.

By the way, when the image input means is installed in a moving vehicle, the image input means is likely to be misaligned (deviation in its installation position with respect to the moving vehicle) due to its own weight, vibrations during travel, and the like. However, if such a positional shift of the image input means with respect to the moving vehicle occurs, that is, if a shift in the positional relationship between the image input means and the moving vehicle occurs, the recognition based on the image input to the image input means may occur. The positional relationship between the vehicle and the outside world that has been recognized also shifts, and therefore, when the travel control of the vehicle is performed based on the recognized external world, problems arise such as the travel control not being performed properly.

Therefore, in order to solve this problem, for example, an angle sensor (
It is conceivable to provide a potentiometer (potentiometer) or a linear displacement sensor and correct the image input to or input to the image input means based on the positional deviation information of the image input means obtained by these sensors.

(Problem to be Solved by the Invention) However, the method of detecting the positional deviation of the image input means using the various sensors as described above requires the preparation of such a separate positional deviation detection sensor. In addition, the use of such sensors complicates the entire system and increases costs.Furthermore, considering the fact that it is difficult to understand what causes the positional deviation of the image input means, it is difficult to predict the direction of the deviation. The problem is that it is difficult to provide a sensor.

In view of the above circumstances, it is an object of the present invention to detect the positional deviation of an image input means with respect to a moving vehicle without using the positional deviation detection sensor as described above, thereby eliminating the inconvenience caused by providing the positional deviation detection sensor. An object of the present invention is to provide an image processing device for a moving vehicle that is free from the risk of occurrence of such occurrence.

(Means for Solving the Problems) In order to achieve the above object, an image processing device for a moving vehicle according to the present invention is an image processing device for a moving vehicle equipped with an image input means for external world recognition, which comprises: a positional deviation detection means for detecting a positional deviation of said image inputting means installed in a moving vehicle based on an image inputted to said image inputting means; It is characterized by comprising a correction means for correcting an image input to the image input means or a positional deviation of the input image.

The positional deviation of the image input means is a deviation in the installation position of the image input means on the moving vehicle, and the installation position deviation refers to a deviation in the installation location and/or a deviation in the installation angle.

The positional shift of the above-mentioned image refers to a two-dimensional shift (image shift) of the manually inputted or manually inputted image to the above-mentioned image inputting means with respect to the standard image, which is an image in which the image inputting means does not produce any positional displacement. ) means.

As a correction means for correcting the positional deviation of the image input to the image input means, a means for correcting the installation position of the image input means to a state without positional deviation based on the positional deviation information using an actuator can be considered. .

The correction means for correcting the positional deviation of the image input to the image input means includes a means for software-shifting the inputted image based on the positional deviation information so that it is at the same position as the reference image. is possible.

(Function) The image input means installed in the moving vehicle originally functions to detect the positional deviation (absolute position) of the moving vehicle with respect to the outside world, but this function can be improved by installing the image input means on the moving vehicle. It can also be used as a positional deviation detection means.

For example, if a predetermined part of the moving vehicle is set as a reference point and the reference point of the moving vehicle is captured in the field of view of the image input means, the image input means and the moving vehicle can be at a fixed position. In other words, as long as the image input means is not misaligned, the reference point of the moving vehicle is always at the predetermined target position in the field of view of the image input means, that is, the predetermined target in the image input to the image input means. Therefore, it is difficult to determine whether or not the reference point of the moving vehicle is located at the target position in the image input to the image input means, and if not, to what extent the two are shifted. It is possible to detect a positional deviation between the image input means and the moving vehicle.

Therefore, if the positional deviation of the image input means is detected based on the image input to such an image input means and the image is outputted to correct the positional deviation, it is necessary to separately provide various sensors as described above. This is not necessary, and the problems described above caused by the provision of such a sensor can be avoided.

(Embodiments) Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1A is a block diagram schematically showing an embodiment including an actuator for correcting the installation position of the image input means as a correction means, and the image input to the image input means 2 is sent to an image processing means for external world recognition (not shown). The input image is also input to the positional deviation detection means 4, and the positional deviation detection means 4 detects a deviation in the installation position of the image inputting means 2 with respect to the moving vehicle based on the input image, and based on the positional deviation information. Then, the installation position of the image input means 2 is corrected by a correction means 6 having an actuator such as a motor.

Next, a more detailed embodiment of the above type of image processing apparatus will be described with reference to FIGS. 2 to 6.

In this embodiment, the apparatus according to the present invention is applied to an image processing apparatus for an automobile in which automatic steering is performed, and an image sensor using a CCD is used as an image input means.

As shown in FIG. 2, the image sensor 2 is disposed in front of the windshield of an automobile 10 via a pan head 8, and is capable of inputting external world information in front of the automobile 10 in the traveling direction as an image. A protrusion serving as a reference point 12 provided at the center of the upper front end of the image sensor 2 is always within the field of view of the image sensor 2. Of course, the image sensor 2 may be installed at an equivalent position in the vehicle interior, and the reference point 12 may be set at any part of the vehicle as long as it is within the field of view of the image sensor 2.

As shown in FIG. 3A, which is a front view (view from the front of the vehicle body), and FIG. 3B, which is a right side view, the pan head 8 includes a base 14 that is fixed to the vehicle body, and a base 14 that is fixed to the vehicle body. A first gear 20 is mounted via a bearing 16 having a rotatable ball thereon and is rotatable about a yaw shaft 18 extending substantially in the vertical direction of the vehicle body; A first support stand 22 provided therein, a second gear z6 rotatably supported on the first support stand 22 about a pitch axis 24 extending substantially in the vehicle width direction, and a second gear z6 fixedly attached to the second gear 26. a second support base 28; a first motor and first pinions 30a, 30b for rotating the first gear 20;
6 and a second pinion 32a,
12b, and the image sensor 2 is fixed to the second support base 28 of the pan head.

That is, this image sensor 2 is outputted so that the yaw angle around the yaw axis 18 and the pitch angle around the pitch axis 24 can be adjusted. and pitch angle).

Image output from the above image sensor 2 (image signal)
As shown in FIG. 6, which is a block diagram of this embodiment, is manually inputted into the external world recognition image processing means 40. The processing means 4
0, the image signal input from the image sensor 2 is processed by various image processing algorithms, and edges, areas, etc. indicating the driving route are extracted.

The extracted edge/area information is input to the local map generation means 42, which generates a local map based on the edge/area information, that is, the state of the outside world in the immediate vicinity of the vehicle. A map is created. This local map information is used by the optimal route generating means 44.
The optimal route to the destination is determined based on the above local map information and information such as the destination that has been input in advance, and the travel direction and travel speed for traveling on the optimal route are determined. Ru. Information regarding the driving direction and speed is input to the controller 46, which controls the steering and speed of the vehicle based on the driving direction and speed information.

On the other hand, the image input to the image sensor 2 is outputted to the positional deviation detection means 4, and the reference point detection image processing means 4B that outputs the detection means 4 determines the position of the reference point 12 of the vehicle body in the image. At the same time, the positional deviation calculation means 50 calculates the amount of positional deviation between the reference point 12 and the target position where the reference point 12 would be located if the image sensor 2 had not been misaligned. The amount of deviation in the installation position of No. 2, that is, the amount of deviation in the yaw angle and pitch angle is calculated.

The position of the reference point 12 of the vehicle body can be detected by image processing, for example, by adding a predetermined feature to the reference point 12, for example, by forming it with a high-luminance material, or by adding a predetermined conspicuous color. Can be easily detected.

The above positional deviation is calculated as follows. That is, in the case where a positional deviation occurs, the reference point 12 and the target position 64 in the image (field of view) 62 are separated by Δx1 in the vehicle width direction and Δy in the vertical direction, as shown in FIG. 4, for example. (The units of ΔX and Δy are both pixels). This ΔX,
Δy is directly determined from the image 62, and this determined ΔX
, Δy, the amount of deviation of the yaw angle and pitch angle of the image sensor 2 is calculated by the following formula.

Yaw angle θ knee tan-1 (k - ΔX / Q,) pw k
・Δx/Q.

Pitch angle θ2 -tan -' (k φΔy/L) ~ - Δy/Q
.

However, in the above equation, k is a proportionality constant (trta/vixel) specific to the image sensor 2, and k is the distance (am) between the center of the lens of the image sensor 2 and the screen surface.
It is. Note that the lens center is the center of the lens portion 2a of the image sensor 2, as shown in FIG.
surface) 2b.

Once the yaw angle θ1 and the pitch angle θ2, which are the amount of deviation of the image sensor 2, are calculated as described above, the yaw angle θ1
The information on the pitch angle θ2 is outputted to the correction means 6, and the motor control means 52 outputting the correction means 6 controls the motor rotation angle corresponding to the yaw angle θ and the pitch angle θ2 (the first motor 30a and the The rotation angle of the two motors 32a) is determined, and the motors 54 (30a, 30a,
32a) is rotated, whereby the image sensor 2 is corrected to its initial position without deviation, and the reference point 12 is located at the target position 64 in the image 60.

The positional deviation detection by the positional deviation detection means 4 and the correction of the positional deviation of the image sensor 2 by the correction means 6 may be carried out all the time, or may be carried out at appropriate time intervals. In the latter case, the image input from the image sensor 2 to the external world recognition image processing means 40 may be a positionally shifted image, so in that case, for example, the above-mentioned ΔX detected by the positional shift detecting means 4, Recognize Δy to the outside world”
It is also possible to input the signal to the image processing means 40, and the processing means 40 outputs it so as to perform external world recognition in consideration of ΔX and Δy.

In addition, it is sufficient to correct the positional deviation of the image sensor by correcting the pitch angle and yaw angle, which have a large influence on the positional deviation of the input image, as in the above embodiment, but if necessary, it is necessary to correct the positional deviation of the image sensor by correcting the positional deviation of the input image. It may also be output in such a way that any discrepancies can be corrected.

In that case, it is necessary to detect deviations in the roll angle and deviations in the installation location, but these can be detected by, for example, setting a plurality of reference points 12 on the vehicle body.

FIG. 1B is a block diagram schematically showing an embodiment including a means for shifting an image by software as a correction means,
The image input to the image input means 2 is output to the correction means 6 and is also input to the positional deviation detection means 4, the positional deviation of the image input means is detected by the positional deviation detection means 4, and the positional deviation information is Based on this, the image inputted to the correction means 6 is shifted by the correction means 6 to a state without positional deviation and outputted.

Next, a more detailed embodiment of this type of image processing apparatus will be described with reference to FIG.

Like the embodiment shown in FIG. 6, this embodiment is also applied to image processing of an automobile in which automatic steering is performed, and the same image input means as in the embodiment shown in FIG. 6 is used. However, in the case of this embodiment, the output capable of correcting the positional deviation of the image sensor as in the embodiment shown in FIG. 6 may be omitted.

In the case of this embodiment, the image input to the image sensor 2 is first stored in the image memory BO, and the image is extracted from the memory 60 in response to a request from the external world recognition image processing means 40. Based on the image, external world recognition is performed by the external world recognition image processing means 40 in exactly the same manner as in the embodiment shown in FIG. and the traveling speed are determined, and the controller 46 controls the steering and position of the moving vehicle.

On the other hand, the image input to the image sensor 2 is outputted to the positional deviation detection means 4, and the reference point 12 in the image is detected by the reference point detection image processing means 48 as in the case of the embodiment shown in FIG. The displacement calculation means 50 calculates the displacement amounts ΔX and Δy between the reference point 12 and the target position 64 (calculating these ΔX and Δy means ultimately calculating the displacement amount of the image input means). (synonymous with). These deviation amounts ΔX and Δy are stored in the position deviation register 5B that outputs the correction means 6, and when the image processing means 40 consisting of a CPU accesses the image memory 60 and retrieves the image, the address (x, y) is The positional deviation register 56 is inputted to the positional deviation register 56, and the positional deviation register 56 receives the address ((
X+Δx, y+Δy) is output, and an image signal of this address (X+Δx, y+Δy) is extracted via the address generator 58 and input to the image processing means 40. Therefore, the image input to this image processing means 40 is an image without positional deviation (an image in which the reference point 12 is located on the target position 64 in the image), and therefore, in this image processing means 40, the moving vehicle relative to the outside world is Positional relationships are recognized correctly.

The correction means 6 in this embodiment is ultimately the image processing means 4.
When 0 accesses the image memory 60, it is hardware that increases or decreases the address depending on the amount of positional shift, and the image is shifted within the image memory 60.

The image input from the image input means in the present invention may be used for recognizing the outside world, and the manner in which it is used is not particularly limited. In other words, while the above embodiment performs autopilot by recognizing the external world from the image, it also performs partial autopilot, which forcibly performs steering or braking only when an obstacle is detected in front. It may be possible to provide a warning in such a case, and furthermore, the image may be processed as necessary and displayed on a CRT or the like mounted on the moving vehicle to make the driver etc. aware of the outside world. It may be something.

(Effects of the Invention) As described above, the image processing device for a moving vehicle according to the present invention detects the deviation of the installation position of the image input means with respect to the moving vehicle based on the image input to the image input means for external world recognition. Based on the detected positional deviation, the image is input to an image input means or outputted to correct the positional deviation of the input image. Even if the installation position of the input means is misaligned with respect to the vehicle, it is possible to correctly recognize the positional relationship between the outside world and the vehicle, and, for example, it is possible to appropriately control the travel of the vehicle based on this.

Furthermore, since the detection of the deviation in the installation position of the image input means is performed based on the image input to the image input means,
There is no need for various sensors to detect deviations in the installation position,
Problems such as complication of the system, high cost, and difficulty in arranging the sensors, which occur when using such sensors, do not arise.

[Brief explanation of the drawing]

FIG. 1A is a block diagram schematically showing an embodiment of the present invention, FIG. 1B is a block diagram schematically showing another embodiment, and FIG. 2 is a side view showing an example of how the image input means is installed in a moving vehicle. Figures 3A and 3B are front and side views showing details of the image input means shown in Figure 2, and Figure 4 shows an example of an image input to the image input means shown in Figure 2. 5 is a side view of the image input means shown in FIG. 2, FIG. 6 is a more detailed block diagram of the embodiment shown in FIG. 1A, and FIG. 7 is a more detailed block diagram of the embodiment shown in FIG. 1B. It is a diagram. 2... Image input means 4... Positional deviation detection means 6
...Correction means 10...Moving vehicle Figure 1A
Figure 5 Figure 6 Figure 7 Old figure Figure 2 Figure 3 A figure 3B figure

Claims (1)

[Scope of Claims] An image processing device for a moving vehicle equipped with an image input means for external world recognition, wherein a positional shift of the image input means installed in the moving vehicle is input to the image input means. a positional deviation detection means for detecting based on the image; and a correction means for correcting the positional deviation of the image input to the image input means or the input image based on the positional deviation information output from the positional deviation detection means. An image processing device for a moving vehicle, comprising: