JP2004274114A - Image display apparatus and image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display apparatus and an image display method capable of reducing the possibility that a vehicle hits an obstacle. <P>SOLUTION: The image display apparatus superimposes a viewpoint-converted image obtained by viewpoint-converting a video image photographed with cameras 1a-1n on an image of the own vehicle, and display the superimposed image on a display 8. The apparatus has a distance measurement sensor 2 for measuring the distance to an obstacle around the own vehicle, a distance coordinate calculating means 4 for displaying the information about the presence of the obstacle in a direction of the movement of the own vehicle which is calculated on the basis of a result of measurement by the sensor 2 so that the information is superimposed on the viewpoint-converted image, and a second image processing apparatus 5. If the own vehicle 10 begins to move from a position of the vehicle 10 where the measurable range of the sensor 2 is smaller than the projecting range of the vehicle 10, the apparatus informs a driver of the possibility that an obstacle exists at a position closer to the vehicle 10 than the position displayed by the presence information. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image display device and an image display method for converting an image captured by an electronic camera into a viewpoint and displaying the image.
[0002]
[Prior art]
[Patent Document] JP-A-2002-316602.
[0003]
A viewpoint-converted image obtained by changing the viewpoint of an image captured by a camera is converted as if a certain plane, for example, an object existing on the ground, is viewed from the same viewpoint. There is a fundamental property that it is not converted to For this reason, for example, when converting a video image captured by a rear video camera installed at the rear of the own vehicle to a video image viewed from directly above the own vehicle with respect to the ground, an object existing near the own vehicle may be changed. It may be displayed as if it is far away. For example, projections and the like existing on the wall surface are not displayed accurately in both position and size. Therefore, when approaching the wall surface by focusing only on the unevenness on the image, there is a possibility that the projections on the wall surface may come into contact with the projections on the wall surface.
[0004]
On the other hand, in the above-mentioned patent document, a virtual warning line surrounding the image of the own vehicle prepared in advance is superimposed and displayed around the viewpoint converted image of the own vehicle, and the possibility that an object exists near the own vehicle is displayed. It proposes reducing the possibility of contact by warning and avoiding inadvertently approaching surrounding objects.
[0005]
[Problems to be solved by the invention]
However, since this virtual warning line is prepared in advance assuming an obstacle around the own vehicle to a certain size, if there is no obstacle at all, it should be closer to the original case. Occurs, and conversely, if the obstacle is larger than assumed, it may contact the obstacle even if the virtual warning line is believed, which is not a fundamental solution.
[0006]
An object of the present invention is to provide an image display device and an image display method that can reduce the possibility of contact with an obstacle.
[0007]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides an image display device that superimposes and displays a viewpoint-converted image obtained by performing viewpoint conversion on a captured image and an image of the own vehicle, and sets a distance to an obstacle around the own vehicle. It is configured to include a distance measuring means for measuring, and a means for superimposing and displaying the existence information of the obstacle calculated based on the measurement result of the distance measuring means on the viewpoint converted image.
[0008]
【The invention's effect】
According to the present invention, it is possible to provide an image display device and an image display method that can reduce the possibility of contact with an obstacle.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings described below, those having the same functions are denoted by the same reference numerals, and repeated description thereof will be omitted.
FIG. 1 is a block diagram illustrating a system configuration of an image display device according to an embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes at least one electronic camera that captures an image of the surroundings of the host vehicle, and here is constituted by a plurality of cameras 1a to 1n installed at appropriate plural locations of the host vehicle. The plurality of cameras 1a to 1n include a camera for photographing a rear image installed at the rear of the vehicle (here, this is referred to as camera 1a). Note that two or more cameras may be installed as cameras for capturing the rear video, and in this case, the distortion of the viewpoint converted image can be reduced as compared with the case where one camera is used.
Reference numeral 2 denotes a distance measuring sensor (for example, a radar) installed at the rear of the vehicle to measure a distance to an obstacle around the vehicle.
Reference numeral 3 denotes a first image processing apparatus to which images captured by a plurality of cameras 1a to 1n are input and a viewpoint conversion image is created.
Reference numeral 4 denotes a distance coordinate calculation device that receives an output signal of the distance measurement sensor 2 and calculates distance coordinates.
Reference numeral 5 denotes a second image processing device to which an output signal of the first image processing device 3 and an output signal of the distance coordinate calculation device 4 are input.
Reference numeral 8 denotes a display (display) installed in the own vehicle.
Reference numeral 6 denotes a display changeover switch which is installed in the own vehicle and switches the display of the display 8.
Reference numeral 7 denotes a display switch which switches the display of the output signal of the second image display device 5 and the output signal of another image on the display 8 by operating the display switch 6.
[0010]
FIG. 2A is a side view illustrating the operation in the present embodiment, and FIG. 2B is a diagram illustrating a viewpoint-converted image. 2, reference numeral 10 denotes a host vehicle, 11 denotes a rear portion of the host vehicle 10, 12 denotes a wall surface, and 13 denotes a protrusion existing on the wall surface 12.
FIGS. 3A to 3C are diagrams showing the relationship between the measurable range of the ranging sensor 2 and the rear projection range of the host vehicle. The case where the parking operation is started from a position that can be covered (that is, the position of the host vehicle whose measurement range of the distance measurement sensor 2 is the same as or larger than the projection range of the host vehicle) is shown. (A), (b) is a side view, (c) is a figure of a viewpoint conversion image. In the viewpoint-converted image of FIG. 3C, 15 is a white line drawn in the parking lot, 16 is a projection whose viewpoint has been changed, 17 is the actual position of the projection, and 20 is a warning line according to the present invention.
4A to 4C are diagrams showing the relationship between the measurable range of the distance measuring sensor 2 and the rear projection range of the own vehicle, wherein the measurable range of the distance measuring sensor 2 is the rear projection range of the own vehicle. The case where the parking operation is started from a position where the vehicle cannot be covered (that is, a position of the host vehicle where the measurable range of the distance measurement sensor 2 is smaller than the projection range of the host vehicle) is shown. (A), (b) is a side view, (c) is a figure of a viewpoint conversion image. In FIG. 4B, reference numeral 14 denotes a second protrusion different from the protrusion 13, in FIG. 4C, reference numeral 18 denotes a second protrusion whose viewpoint has been changed, and reference numeral 19 denotes an actual second protrusion. Is a warning display according to the present invention.
[0011]
FIG. 5 is a flowchart illustrating the flow of the process according to the present embodiment.
[0012]
Hereinafter, the operation (flow of the image display method) of the image display device of the present embodiment will be described with reference to FIGS.
1) The driver operates the display changeover switch 6 (FIG. 1) provided in the host vehicle 10 when the host vehicle 10 moves backward to park the host vehicle 10 (FIG. 2) in, for example, a parking lot. Then, the rear image captured by the rear image capturing camera 1a provided at the rear portion 11 of the vehicle 10 is displayed on the display 8 in the vehicle. Note that the image may be switched by interlocking the shift lever without providing the display changeover switch 6. That is, the rear image may be automatically displayed on the in-vehicle display 8 when the driver operates the shift lever to enter the “reverse (back)” position.
[0013]
2) The images captured by the plurality of cameras 1a to 1n are input to the first image processing device 3, and a viewpoint conversion image is created according to a predetermined pixel coordinate movement rule (S1 in FIG. 5). ).
[0014]
3) The distance measurement sensor 2 (FIG. 2) provided at the rear portion 11 of the host vehicle 10 measures a distance L (DW) from the host vehicle 10 to the rear wall surface 12 (S2 in FIG. 5).
[0015]
Here, the calculated distance L to the wall surface 12 behind the host vehicle 10 and the rear projection range of the host vehicle 10 (the projection range is a range in which the host vehicle is projected on a plane perpendicular to the moving direction of the host vehicle, ie, The range based on the vehicle width and the vehicle height of the own vehicle) PA (FIG. 3A) is within the measurable range (detection area) DA on the wall surface 12 of the distance measuring sensor 2. Is compared with the distance L0 (S3 in FIG. 5).
[0016]
The distance L0 is calculated based on the angle of view of the distance measuring sensor 2 and the height of each part of the rear portion 11 of the host vehicle 10.
[0017]
In the example of FIG. 3A, assuming that the vertical angle of view of the distance measuring sensor 2 is 2θ, the height of the trunk at the rear part 11 of the vehicle 10 is h1, and the height of the lower part of the bumper is h2, the following formula is used. .
[0018]
L0 = (h1-h2) / 2tan θ
The same calculation can be performed in the horizontal direction, but the description of the horizontal direction is omitted.
[0019]
When L ≧ L0, the following 4) to 7) are executed.
[0020]
4) The output signal of the distance measurement sensor 2 is input to the distance coordinate calculation device 4, and the distance DW from the own vehicle 10 to the wall surface 12 according to a predetermined scale ratio between the actual distance and the display distance (FIG. 2). And the distance DD from the wall 12 to the protrusion 13 calculated from the distance DW from the host vehicle 10 to the wall 12 and the distance DO from the host vehicle 10 to the protrusion 13 as a coordinate position and distance matching the display size. (S5 in FIG. 5).
[0021]
5) The viewpoint transformed image obtained by the first image processing device 3 and the distance coordinate information calculated by the distance coordinate calculation device 4 are input to the second image processing device 5.
[0022]
6) The display coordinates of the warning line 20 on the display 8 are calculated (S6 in FIG. 5). The second image processing device 5 includes a plan view image (a photographic image or a CG image) of the rear portion 11 of the host vehicle 10 prepared in advance and the wall surface calculated by the distance coordinate calculation device 4 on the viewpoint converted image. The warning line 20 is superimposed on the position close to the host vehicle 10 by a distance from the coordinates of 12 to the protrusion 13 in parallel with the host vehicle 10 (S7 in FIG. 5, FIG. 2B).
[0023]
7) The output image of the second image processing device 5 is displayed on the display 8 in the vehicle (S8 in FIG. 5).
[0024]
8) On the other hand, when L <L0, a warning display 21 indicating that there is a possibility that there is an obstacle in front of the warning line 20 (on the side of the host vehicle 10) in 5) simultaneously with the execution of the above 4) to 7). The display is superimposed on the screen (S4 in FIG. 5) or a voice guide is reproduced. The warning display 21 indicates, for example, as shown in FIG. 4C, "There is a possibility that there is an obstacle before the warning line!"
[0025]
9) Thereafter, the above steps 2) to 8) (S1 to S8) are repeated until the backward change image is no longer selected by the display changeover switch 6 (S9 in FIG. 5). When the display switch 6 is turned off, the screen is switched to the default screen (S10 in FIG. 5).
[0026]
As described above, in the image display device according to the present embodiment, an image display in which the viewpoint-converted image obtained by converting the images captured by the cameras 1a to 1n and the image of the vehicle 10 is superimposed and displayed on the display 8 is provided. In the apparatus, a distance measuring sensor 2 for measuring a distance to an obstacle around the host vehicle 10 and a distance for superimposing and displaying the existence information of the obstacle calculated based on the measurement result of the distance measuring sensor 2 on the viewpoint conversion image. The configuration includes a coordinate calculation unit 4 and a second image processing device 5. It should be noted that the display 8 in FIG. 1 is used as the display means in the claims, the distance measuring sensor 2 is used as the distance measuring means, and the distance coordinate calculating means 4 and the second image processing device 5 are used to display the obstacle presence information. This corresponds to a means for superimposing and displaying the image on the viewpoint conversion image.
[0027]
As described above, the own vehicle 10 is provided with the distance measuring sensor 2 for detecting the rear obstacle, and the fact that the obstacle is present at the position actually measured by the distance measuring sensor 2 is superimposed and displayed as the warning information on the viewpoint converted image by In addition, it is possible to visually confirm on a screen even irregularities that cannot be determined only by a camera image, and it is possible to avoid contact with an obstacle in addition to easy visual confirmation in a viewpoint-converted image.
[0028]
Further, in the image display method according to the present embodiment, in the image display method in which the viewpoint-converted image obtained by performing the viewpoint conversion on the images captured by the cameras 1a to 1n and the image of the own vehicle 10 are superimposed and displayed, The distance to the surrounding obstacle is measured, and the existence information of the obstacle calculated based on the measurement result is superimposed and displayed on the viewpoint conversion image.
[0029]
With such a configuration, it is possible to visually confirm on the screen even irregularities that cannot be determined from the camera image alone, and to avoid contact with obstacles in addition to easy visual confirmation in the viewpoint conversion image. Becomes possible.
[0030]
Further, the configuration is such that the obstacle existence information is the obstacle existence information in the moving direction of the vehicle 10. With such a configuration, it is possible to avoid contact with an obstacle in the moving direction of the vehicle 10.
[0031]
Further, when the movement of the host vehicle 10 is started from the position of the host vehicle 10 where the measurable range of the distance measuring sensor 2 is smaller than the projection range of the host vehicle 10, an obstacle is located closer to the host vehicle 10 than the position indicated by the presence information. Is notified that there is a possibility that
[0032]
As described above, when the distance measurement sensor 2 starts the parking operation from a position where the rear projection range of the vehicle 10 cannot be covered, for example, and approaches the wall surface 12 backward, the obstacle is located before the warning line 20 actually measured and displayed on the screen. By notifying the driver of the possibility of an object, the possibility of contact with an obstacle can be reduced.
[0033]
Although the present invention has been specifically described based on the embodiment, the present invention is not limited to the above embodiment, and it is needless to say that various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, an example has been described in which the own vehicle is backed up and parked using the camera 1a for capturing a rear image and the distance measurement sensor 2 installed at the rear of the own vehicle 10, but is not limited thereto. However, it is needless to say that the present invention can be applied to a case where a camera and a distance measuring sensor provided in a portion other than the rear portion of the vehicle are used.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a system configuration of an image display device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating effects of the embodiment of the present invention.
FIG. 3 is a diagram illustrating an effect of the embodiment of the present invention.
FIG. 4 is a diagram illustrating an effect of the embodiment of the present invention.
FIG. 5 is a flowchart showing the operation of the embodiment of the present invention.
[Explanation of symbols]
Reference numerals 1, 1a to 1n Camera 2 Distance measuring sensor 3 First image processing device 4 Distance coordinate calculating device 5 Second image processing device 6 Display switch 7 Display switch 8 Display 10 Own vehicle 11 ... rear part 12 ... wall surface 13 ... projection 14 ... second projection 15 ... white line 16 ... viewpoint 17 converted projection 17 ... actual projection position 18 ... viewpoint converted second projection 19 ... actual position of the second protrusion 20 ... warning line 21 ... warning display

Claims (6)

In an image display apparatus that superimposes a viewpoint-converted image obtained by performing viewpoint conversion on an image captured by a camera and an image of the vehicle and displays the image on a display unit,
Distance measuring means for measuring a distance to an obstacle around the own vehicle,
Means for superimposing and displaying the presence information of the obstacle calculated based on the measurement result of the distance measuring means on the viewpoint-converted image. The image display device according to claim 1, wherein the obstacle presence information is the obstacle presence information in a moving direction of the host vehicle. When the measurable range of the distance measuring means starts moving the host vehicle from the position of the host vehicle narrower than the projection range of the host vehicle,
The image display device according to claim 1, wherein the possibility that the obstacle is present closer to the host vehicle than the position indicated by the presence information is notified. In an image display method for superimposing and displaying a viewpoint-converted image obtained by performing viewpoint conversion on an image captured by a camera and an image of the own vehicle,
Measure the distance to obstacles around the vehicle,
An image display method, wherein the obstacle presence information calculated based on the measurement result is superimposed on the viewpoint converted image. The image display method according to claim 4, wherein the obstacle presence information is the obstacle presence information in a moving direction of the host vehicle. When the measurable range starts moving the host vehicle from the position of the host vehicle narrower than the projection range of the host vehicle,
5. The image display method according to claim 4, wherein the possibility that the obstacle is present on the side of the host vehicle relative to the position indicated by the presence information is notified.

Images