JP2004289386A - Image display method, image display apparatus, and image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display method capable of displaying even a spatial obstacle by decreasing or eliminating dead angles while ensuring easiness of camera installation. <P>SOLUTION: The image display method includes: a depression angle camera photographing step for photographing a prescribed region with a depression angle camera 8 installed to have a depression angle; an elevation angle camera photographing step for photographing the upper part of the photographing region of the depression angle camera 8 by means of an elevation angle camera 9 installed to have an elevation angle; an overlook image generating step for generating an overlook image resulting from looking down on the photographing region of the depression angle camera 8 on the basis of a photographing image with the depression angle camera 8; an inverse overlook image generating step of generating an inverse overlook image obtained by looking up at the photographing region of the elevating angle camera on the basis of the photographing image with the elevating angle camera 9; a composite image generating step of composing the overlook image and the inverse overlook image to generate a composite image; and a display step of displaying the composite image on a monitor. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image display method and an image display device that generate a composite image with a small blind spot based on images captured by a plurality of cameras, and an image processing device used in the image display device.
[0002]
[Prior art]
2. Description of the Related Art As an image display device that displays an image captured by a camera or an image based on the image on a monitor, an image behind the vehicle captured by a camera installed near a trunk or a license plate (license plate) of the vehicle is displayed on a monitor in a vehicle compartment. Parking assist systems are known. In such a device, the camera is usually installed obliquely downward so as to have a depression angle, and the area of view below the vehicle including the ground is the main field of view. The obstacle behind can be checked. However, with such a device that captures an image of the rear of the vehicle obliquely downward and displays it on a monitor, it is not easy to grasp the sense of distance between the vehicle and the obstacle and the positional relationship between the vehicle and the obstacle, and improvements have been desired. .
[0003]
On the other hand, as described in Patent Literature 1, there is also an overhead view image display system that captures images with a plurality of cameras, converts the viewpoint of the captured image so as to look down from directly above the vehicle, and displays the converted image on a monitor. According to this device, since the shape of the obstacle, the sense of distance to the obstacle, and the positional relationship are easy to understand, it is very advantageous for the driver to intuitively recognize the immediate case.
[0004]
By the way, in an actual overhead image display system, a camera is hardly attached to the uppermost part of the vehicle as described in Patent Document 1 because of structural problems and the like, and a rear monitoring camera called a back-eye camera is used. Often, a bird's-eye view image (an image when the shooting area is viewed from above) is obtained using a video shot by a camera. Specifically, as shown in FIG. 8, (a camera a real thus also referred to as a real camera.) The camera trunk lid of the vehicle 1 2 was placed for shooting the imaging area S _1, the imaging region By generating a bird's-eye view image as if captured by a virtual camera (non-existent camera) 3 assumed to be 2-3 m above the sky above S ₁ based on the captured image of the real camera 2, the obstacle The shape, size, sense of distance, positional relationship, and the like are intuitively transmitted to the driver.
[0005]
[Patent Document 1]
JP-A-58-110334
[Problems to be solved by the invention]
However, overhead image as if shot by a virtual camera 3 based on the image captured by the real camera 2 as described above, does not appear in the overhead image for not included in the imaging area S ₁ information, the virtual camera the portion indicated by oblique lines with respect to the imaging area S ₂ when 3 is existent in a dead angle.
[0007]
That is, in many cases, the obstacles on the ground, since in the form which extends upwardly from the ground, the obstacle enters gradually imaging area S ₁ in accordance with the vehicle 1 moves backward, the driver this time it It is possible to recognize and avoid collisions. However, when there is a spatial obstacle that is not directly on the ground, such as a protrusion from a wall or a tree branch, the obstacle collides with the vehicle 1 in the spatial blind spot indicated by the diagonal lines. fear and thus, there is a risk that it even entered the imaging area S ₁ is collides with the vehicle 1 without while avoiding suddenly at a by the driver, in order to further enhance the safety by the monitor display Blind spots need to be reduced.
[0008]
The blind spot is not included in the imaging area S ₃ of the will be almost completely eliminated if installed real camera 5 to the (or patent as in Reference 1) the highest position of the vehicle 4 as shown in FIG. 9 (real camera 5 and, although the area included in the imaging area S ₄ of the virtual camera 6 to assume thereabove is a blind spot, most of the region does not substantially problematic for the upper of the vehicle 4.), described above As described above, installing a camera on the top of the vehicle involves structural difficulties (physical difficulties), and when an obstacle exists at the same height as the camera, the obstacle is not photographed. The possibility that collision with an object cannot be avoided remains.
[0009]
The present invention has been made in view of the above circumstances, and an image that can display a spatial obstacle by reducing or eliminating a blind spot while securing the ease of installation of a camera. An object of the present invention is to provide a display method, an image display device, and an image processing device used for the image display device.
[0010]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the invention according to claim 1 includes a depression angle camera photographing step of photographing a predetermined area by a depression angle camera installed so as to have a depression angle, and the elevation angle camera arranged to have an elevation angle. An elevation camera photographing step of photographing above the photographing area of the depression camera, and a bird's-eye image generation step of generating a bird's-eye image when the photographing area of the depression camera is viewed from above, based on the photographed image by the depression camera, A reverse bird's-eye view image generating step of generating a reverse bird's-eye view image when the photographing region of the elevation camera is viewed from below, based on the image captured by the elevation camera, and a composite image combining the bird's-eye view image and the reverse bird's-eye view image And a display step of displaying the composite image on a display means.
[0011]
According to a second aspect of the present invention, in the image display method according to the first aspect, in the composite image generating step, one of the bird's-eye view image and the inverted bird's-eye view image is reversed left and right to be translucent with the other and overlapped. The combined image is generated by combining them.
[0012]
The invention according to claim 3 is a depression camera that is installed to have a depression angle and shoots a predetermined area, an elevation camera that is installed to have an elevation angle and shoots an upper part of the imaging area of the depression camera, A bird's-eye view image when the shooting area of the depression camera is viewed from above is generated based on an image captured by the depression camera, and a shooting area of the elevation camera is viewed from below based on the image captured by the elevation camera. An image display device includes: an image processing unit that generates an inverted bird's-eye view image, combines the overhead bird's-eye image and the reverse bird's-eye view image to generate a composite image, and a display unit that displays the composite image.
[0013]
According to a fourth aspect of the present invention, in the image display device according to the third aspect, the image processing unit inverts one of the bird's-eye view image and the inverted bird's-eye view image to the left and right and makes the other translucent and overlaps with the other. Thus, the composite image is generated.
[0014]
According to a fifth aspect of the present invention, there is provided first decoding means for decoding an image taken by a depression camera which is installed to have a depression angle and shoots a predetermined area, and stores an image decoded by the first decoding means. First storage means for decoding an image captured by an elevation camera, which is installed to have an elevation angle and captures an image above the imaging area of the depression camera, and decoding by the second decoding means. A second storage means for storing the obtained image, and, based on the image stored in the first storage means, to generate a bird's-eye view image when the imaging area of the depression angle camera is viewed from above, Image conversion means for generating an inverted bird's-eye view image when the shooting area of the elevation camera is viewed from below based on the image stored in the storage means, and combining the bird's-eye view image and the reverse bird's-eye view image An image synthesizing means for generating a composite image Te, and wherein the image processing apparatus having an encoding means for encoding the composite image.
[0015]
According to a sixth aspect of the present invention, in the image processing apparatus according to the fifth aspect, the image synthesizing unit inverts one of the overhead view image and the inverted overhead view image left and right and makes the other translucent and overlaps with the other. Thus, the composite image is generated.
[0016]
According to the present invention, a bird's-eye view image when the photographing region of the depression camera is viewed from above is generated based on the photographed image of the depression camera, and based on the photographed image of the elevation camera photographing the photographing region of the depression camera. Therefore, an inverted bird's-eye view image when the shooting area of the elevation camera is viewed from below is generated, and a combined image generated by combining the bird's-eye view image and the reverse bird's-eye view image is displayed on the display means. The information in the space above the photographing area can be captured by the elevation camera, reflected on the composite image, and displayed on the display means. Therefore, the blind spot can be reduced or eliminated, and a spatial obstacle can be displayed.
[0017]
Further, since it is not necessary to install the depression angle camera and the elevation angle camera in a difficult-to-install position such as the top of the vehicle, it is possible to ensure the ease of installation of the cameras.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described with reference to the drawings.
[0019]
FIG. 1 shows a schematic configuration of an image display device (two-system two-camera image processing system: an image processing system having two cameras and two input systems) according to the present embodiment. The image display device 7 includes a depression camera 8, an elevation camera 9, a monitor 10 as a display unit, and a signal processing device 11 as an image processing device.
[0020]
Depression angle camera 8, as shown in FIG. 2, are installed to have a depression angle theta ₁ to the rear of the vehicle 12, to photograph the imaging area S ₅ including the rear of the ground G of the vehicle 12. Elevation camera 9, as shown in FIG. 3, the vicinity (in this case lower) of the depression angle camera 8 at the rear of the vehicle 12 is placed so as to have an elevation angle theta _2, the virtual in the sky 2~3m in rear space of the vehicle 12 shooting imaging area S ₆ comprising a planar V. Imaging area S ₆ is located above the imaging area S _5, the imaging area S ₆ and the imaging area S ₅ is adapted to at least partially overlap in a plan view.
[0021]
The monitor 10 may be a liquid crystal or a CRT, and is provided on an instrument panel or the like in the cabin of the vehicle 12. The monitor 10 projects an image behind the vehicle 12 based on the images taken by the cameras 8 and 9 as described later. However, the monitor 10 does not need to be a dedicated product for the cameras 8 and 9 and can also serve as a monitor for a navigation device or the like. Good.
[0022]
The signal processing device 11 includes an input system 13 connected to the depression camera 8 and an input system 14 connected to the elevation camera 9. The input system 13 includes a decoder unit 15 as a first decoding unit and a first decoding unit. The input system 14 includes a decoder unit 17 as a second decoding unit and an image memory 18 as a second storage unit.
[0023]
Further, the signal processing unit 11 includes an image processing unit 19 and an image processing memory 20 as an image conversion unit, an encoder unit 21 as an encoding unit, a control unit 22 for integrally controlling these, and data referred to by the control unit 22. ROMs 23, 24 and 25 are provided.
[0024]
Photographing is performed by the cameras 8 and 9 (Step 1 (referred to as S.1 in FIG. 4; the same applies hereinafter)). When the image signals of the images captured by the cameras are input to the decoder units 15 and 17, the control unit 22 Separates the synchronization signal from the image signal. The control unit 22 further decodes the image signals input to the decoder units 15 and 17 into RGB signals and performs A / D conversion (step 2), and stores the digital RGB image information obtained in the image memories 16 and 18. Then, jitter (data delay) is absorbed (step 3).
[0025]
Next, the control unit 22, an image stored in the image memory 16 by the image processing unit 19 with reference to the data ROM23 with the image processing, from directly above the imaging area S ₅ based on the image information of the imaging area S ₅ A bird's-eye view image that looks down is generated (step 4). The control unit 22, an image stored in the image memory 18 by the image processing unit 19 with reference to the data ROM24 with the image processing, looked up from beneath the imaging area S ₆ based on the image information of the imaging area S ₆ Such an inverted bird's-eye view image is generated (step 5). Then, the image processing unit 19 combines the bird's-eye view image and the inverted bird's-eye view image with reference to the data ROM 25, and stores the generated combined image in the image processing memory 20 (step 6).
[0026]
Subsequently, the control unit 22 causes the image signal of the synthesized image stored in the image processing memory 20 to be output from the image processing unit 19 to the encoder unit 21 for D / A conversion and encoding (step 7). Output an analog signal to the monitor 10, and the composite image is displayed on the monitor 10 (step 8).
[0027]
For example, as shown in FIGS. 2 and 3, there is an obstacle 26 behind the vehicle 12, and the obstacle 26 is a round bar-shaped vertical member 27 erected on the ground G and a vehicle 12 from the vertical member 27. of the protruding to the side is assumed that consists horizontal member 28. shape of a round bar which has a space obstacle, the imaging area S ₅ of the depression angle camera 8 enters the lower end portion 27a of the ground G and the vertical member 27, FIG. As schematically shown in FIG. ₅ , the bird's-eye view image 29 obtained by changing the viewpoint so as to look down on the photographing area S5 from directly above includes an image 27a 'of the lower end 27a. On the other hand, the imaging region S ₆ elevation camera 9, in the rear space of the vehicle 12 enters the tip portion 28a of a virtual plane parallel to the ground G V and the horizontal member 28, viewpoint conversion as looking up from beneath the imaging area S ₆ The inverted bird's-eye view image 30 includes an image 28a 'of the tip 28a.
[0028]
As shown in FIG. 6, here, the control unit 22 converts an overhead view image 29 into a translucent image 50 with a transparency of 50% and an inverted overhead view image 30 by horizontally flipping an image 32 into a translucent image 32 with a transparency of 50%. By superimposing, a composite image 33 is generated. As a result, as shown in FIG. 7, a composite image 33 is displayed on the monitor 10 as if the camera 34 was supposed to be in the sky above the vehicle 12, and the driver of the vehicle 12 could use the horizontal material 28 can be confirmed on the monitor 10.
[0029]
In the image display device 7 according to this embodiment, based on the image captured by the depression angle camera 8, both the overhead image 29 (and the image 31. The following these when viewed imaging area S ₅ of the depression angle camera 8 from above referred to as "overhead image".) is generated based on the captured image of the elevation angle camera 9 to shoot above the imaging area S ₅ of the depression angle camera 8, the reverse when viewed imaging area S ₆ of the elevation angle camera 9 from below A bird's-eye view image 30 (image 32; hereinafter, each of which is referred to as a “reverse bird's-eye view image”) is generated, and a composite image 33 generated by combining the bird's-eye view image and the reverse bird's-eye view image is displayed on the monitor 10, so that In the camera 8, information (for example, the horizontal member 28) in the space above the shooting area S ₅ , which is a blind spot, can be captured by the elevation camera 9 and displayed on the monitor.
[0030]
That is, in the past, an obstacle located at a position higher than the installation position of the depression camera could not be displayed on the monitor in principle. However, in the present embodiment, an obstacle located above the installation position of the depression camera 8 can be displayed. Is found from an inverted bird's-eye view image (elevation view, frog view) created from an image captured by the elevation camera 9, and this is superimposed on a bird's-eye view image (bird's-eye view) by the depression camera 8, and one monitor display image is displayed. By generating the (synthesized image), it is possible to display on a monitor a viewpoint-converted image having more originally required spatial information. In other words, when a camera is installed in a vehicle, the blind spot is at least between the camera installation position and the top of the vehicle unless the camera is at the top of the vehicle so far, and the spatial obstacle between them is the vehicle. Although the driver had to avoid the collision visually, the driver had to avoid it visually, but in the present embodiment, the elevation camera captures a range higher than the installation height of the camera. Even if it is not at the top, the driver can check the spatial information up to the vehicle height on the monitor, and the safety and reliability are further improved.
[0031]
In addition, since the camera does not need to be installed at a difficult-to-install position such as the top of the vehicle, the present embodiment also ensures the ease of installation of the camera.
[0032]
Note that the present invention is not limited to the above-described embodiment. For example, even if a composite image is generated using three or more cameras, or a calculation by a CPU is performed instead of referring to a data ROM at the time of image processing. You may. Alternatively, instead of setting the transparency when the overhead image 29 or the inverted overhead image 30 is made semi-transparent to 50%, for example, a change in weighting such that the overhead image is the main (low transparency) and the inverse overhead image is the secondary (high transparency) May be given as appropriate.
[0033]
【The invention's effect】
ADVANTAGE OF THE INVENTION Since this invention was comprised as demonstrated above, there exists an effect that a spatial obstacle can also be displayed by reducing or eliminating a blind spot, ensuring the ease of installation of a camera. .
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a schematic configuration of an image display device according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing an installation position and a shooting area of a depression angle camera.
FIG. 3 is an explanatory diagram showing an installation position and a photographing area of an elevation camera.
FIG. 4 is a flowchart showing processing from shooting to display on a monitor in the image display device.
FIG. 5 is an explanatory diagram schematically showing an overhead image and a reverse overhead image.
FIG. 6 is an explanatory diagram showing synthesis of a bird's-eye view image and a reverse bird's-eye view image.
FIG. 7 is an explanatory diagram showing a virtual camera, a bird's-eye view image, and a reverse bird's-eye view image.
FIG. 8 is an explanatory diagram showing an installation example of a conventional vehicle-mounted camera.
FIG. 9 is an explanatory view showing another installation example of a conventional vehicle-mounted camera.
[Explanation of symbols]
7 image display device 8 depression angle camera 9 elevation angle camera 10 monitor (display means)
11 signal processing device (image processing means, image processing device)
12 vehicles

Claims (6)

A depression camera photographing step of photographing a predetermined area by a depression camera installed to have a depression angle,
An elevation camera photographing step of photographing above the photography area of the depression angle camera by an elevation camera installed to have an elevation angle,
A bird's-eye view image generation step of generating a bird's-eye view image when the shooting area of the depression angle camera is viewed from above, based on an image captured by the depression angle camera,
A reverse bird's-eye view image generation step of generating a reverse bird's-eye view image when the photographing area of the elevation camera is viewed from below, based on the image captured by the elevation camera,
A composite image generating step of generating a composite image by combining the overhead image and the reverse overhead image;
Displaying the combined image on a display means. 2. The composite image generation step according to claim 1, wherein the composite image is generated by inverting one of the overhead image and the inverted overhead image and translucently overlapping the other with the other, and generating the composite image. The image display method described. A depression camera that is installed to have a depression angle and shoots a predetermined area,
An elevation camera that is installed to have an elevation angle and shoots above the imaging area of the depression angle camera,
When an overhead view image of the shooting area of the depression camera is viewed from above based on the image captured by the depression camera, and when the shooting area of the elevation camera is viewed from below based on the image captured by the elevation camera Image processing means for generating a reverse bird's-eye image of the above, and combining the bird's-eye image and the reverse bird's-eye image to generate a composite image,
Display means for displaying the composite image. 4. The composite image according to claim 3, wherein the image processing unit generates one of the composite image by inverting one of the overhead image and the inverted overhead image, translucently overlapping the other, and superimposing the other. Image display device. First decoding means for decoding a captured image of a depression camera that is installed to have a depression angle and captures a predetermined area,
First storage means for storing an image decoded by the first decoding means,
Second decoding means for decoding a captured image of the elevation camera that is installed to have an elevation angle and captures an image of the imaging area of the depression camera,
Second storage means for storing an image decoded by the second decoding means,
A bird's-eye view image of the photographing area of the depression angle camera when viewed from above is generated based on the image stored in the first storage means, and the elevation angle is calculated based on the image stored in the second storage means. Image conversion means for generating an inverted bird's-eye view image when the shooting area of the camera is viewed from below,
Image synthesis means for generating a synthesized image by synthesizing the overhead image and the inverted overhead image,
An image processing apparatus comprising: an encoding unit that encodes the composite image. The said image synthesis | combination means produces | generates the said synthetic | combination image by making either one of the said bird's-eye view image and the said reverse bird's-eye view image left-right reversed, making it translucent with another, and superimposing, on the other hand. Image processing device.

Images