JP2002262281A - Display device for image around vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device for image around vehicle with which as many pieces of information as possible is obtained by effectively utilizing a display screen to display a behavior around a vehicle together with the vehicle. SOLUTION: Coordinate transformation is applied to photographed image data captured from a camera to generate transformed image data denoting a bird's eye view looking down at a ground face in a vertical direction. The transformed image data is compared with display image data denoting an image displayed on a monitor to extract an attached area at the outside of a visual field of the camera and the attached area is composited with a transformed image to generate a peripheral image for updating display on the monitor (S110-S150). The drawing area of the peripheral image and the image drawing position to display its own vehicle are set depending on the size of the attached area, as the size of the attached area is smaller, the area to display a real time image (conversion image) from the camera is extended, and conversely the area to display the image of the area at the outside of the visual field (attached area) is reduced (S160-S170).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery image display device for photographing the periphery of a vehicle and displaying the photographed image on a monitor.

[0002]

2. Description of the Related Art Conventionally, a camera (hereinafter, referred to as a "rear camera") is attached to a rear portion of a vehicle in order to support an operation when the vehicle retreats, and an image captured by the rear camera is directly displayed on a monitor. A device to be displayed or an image captured by a rear camera is converted into an image representing a bird's-eye view looking down on the ground surface from a vertical direction, and the image (hereinafter, “converted image”)
(Hereinafter referred to as "vehicle display")
(For example, JP-A-10-211849) is known.

[0003] In particular, in the latter device for synthesizing the display of the own vehicle, the positional relationship between the own vehicle and the object (parking frame or the like) shown in the converted image becomes clear. The usability is improved as compared with the former device displayed in (1). However, in the latter device, the display area of the converted image on the monitor is limited to a narrow range only behind the own vehicle display, so that the display of the peripheral image is difficult to see,
There was a problem that the display screen of the monitor could not be used effectively. For example, actually, the parking frame W on the road surface is as shown in FIG.
Even if it is drawn as shown in Fig. 4,
As shown in (b), only a part is displayed, and sufficient information cannot be obtained. In addition, parking frame W
When the vehicle enters, the parking frames W located on both sides of the vehicle deviate from the field of view of the rear camera and are not displayed on the monitor. Could not be confirmed from the monitor.

On the other hand, as shown in FIG. 5, as the vehicle retreats (in the order of (a) → (b) → (c) in the figure), an image SO of a portion out of the field of view of the rear camera is displayed. Is extracted by comparing the captured image from the rear camera with the currently displayed image, and combining the extracted out-of-field image with the captured image SI from the rear camera, A device for displaying an image SO on a monitor has also been proposed.

[0005]

However, even with such an apparatus for synthesizing an image SO outside the field of view, immediately after the vehicle starts moving (see FIG. 5 (a)), the field of view to be synthesized can be obtained. Since the outside image SO does not exist, the peripheral image is displayed only in a limited area in the rear part of the vehicle similarly to the conventional device that does not perform the synthesis, and thus there is a problem that sufficient information cannot be obtained from the monitor. there were.

The present invention has been made to solve the above problems.
It is an object of the present invention to provide a vehicle peripheral image display device in which a display screen for displaying a state around a vehicle together with the own vehicle is effectively used, and as much information as possible is always obtained.

[0007]

According to a first aspect of the present invention, there is provided a vehicle peripheral image display device, comprising: a photographing means mounted on a vehicle so as to photograph a diagonally downward view. The photographing image obtained by photographing by the photographing means is converted by the image converting means into an image representing a bird's-eye view looking down on the ground from the vertical direction.

[0008] Then, when there is an area that matches between the converted image generated by the image converting means and the peripheral image being displayed by the display means, the synthesizing means determines an area that does not match the converted image. The extracted area is extracted from the peripheral image, and the extracted area is combined with the converted image as an additional image.

Then, the vehicle display means displays the own vehicle at a specific position in the display image of the display means, and the display control means is generated by the image conversion means so as to correspond to the display of the own vehicle. The peripheral image displayed by the display unit is updated with the converted image or the converted image obtained by synthesizing the additional image by the synthesizing unit.

[0010] When the area occupied by the additional image synthesized by the synthesizing means by the synthesizing means is smaller than a preset lower limit, the display changing means causes the display area of the converted image on the display screen to be enlarged. Next, the display area and the display position of the own vehicle by the vehicle display means are changed.

That is, when the additional image is not extracted or only a small amount is extracted, and most of the peripheral image is composed of only the converted image, the area for displaying the converted image is enlarged. As described above, according to the vehicle peripheral image display device of the present invention, the area where the peripheral image is not displayed at all is minimized, and the display screen is effectively used. Can be obtained via

Further, according to the vehicle peripheral image display apparatus of the present invention, the photographing means is installed so as to photograph obliquely downward, and the converting means converts the photographed image from the vertical direction to the ground. Since the image is converted into an image representing a bird's-eye view looking down on the top surface, the image displayed on the display means is easy to see without distortion unlike an image captured by the image capturing means.

Further, according to the vehicle peripheral image display device of the present invention, an image representing the position of the own vehicle is displayed on the display screen, so that a fixed object (for example, a parking frame or the like) indicated in the peripheral image is displayed. The positional relationship with the own vehicle becomes clear, and the operation of the vehicle becomes easy. In particular, when the photographing means is configured to photograph the rear of the vehicle, for example, as set forth in claim 2, it is preferably used when the vehicle is parked by moving backward in the parking frame. Can be.

In this case, since the additional image (the image outside the field of view of the photographing means) is not extracted at the start of the retreat, the area for displaying the converted image generated from the actual photographed image is displayed on the own vehicle display. Widely secured on the rear side of the vehicle.
Thereafter, as the vehicle retreats, the area closer to the vehicle deviates from the field of view of the photographing means, and is not included in the photographed image (hence, the converted image). However, by extracting the image of this portion as an additional image and synthesizing it with the converted image, the peripheral image is also displayed on the side of the own vehicle display, which was not initially displayed. As the display area of the peripheral image based on the additional image is enlarged, the display area of the peripheral image based on the converted image is reduced, and the display area of the peripheral image based on the additional image is sufficiently secured. in this way,
From the peripheral image displayed on the display screen, the situation around the vehicle as well as behind the vehicle can be clearly grasped.

Further, the image conversion means may be configured to extract an image indicating a lane to generate a converted image. In this case, if the captured image is binarized or the like, a lane or the like indicated by a white line can be extracted, and the load of image processing when converting to a bird's-eye view can be reduced.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an explanatory diagram illustrating a main configuration of the vehicle rear image display device of the present embodiment. As shown in FIG. 1, a vehicle rear image processing apparatus according to the present embodiment includes a camera (for example, a CCD camera) 1 as a photographing unit disposed at the rear of an automobile so as to photograph obliquely from below and a dash. An on-board monitor (for example, a liquid crystal display) 3 as a display means arranged on a board, and a microcomputer are configured as main components, and are used for actual display based on photographed image data obtained by photographing with the camera 1. And an image processing unit 5 for generating display image data.

Incidentally, the image processing unit 5 may have an integral structure with either the camera 1 or the monitor 3. The monitor 3 may be provided as a dedicated device for the device, or may be a monitor already installed for another vehicle-mounted device such as a navigation device.

Here, the processing executed by the image processing unit 5 to generate the display image data will be described with reference to the flowchart shown in FIG. Note that this processing is started at a fixed cycle (for example, at intervals of 100 ms).
When the present process is started, first, it is determined whether or not the shift lever is set to the shift position indicating the reverse running (S
110). This shift position can be detected by a signal from a shift position detection sensor (not shown) or a signal from another electronic control unit.

If the shift lever is not at the shift position indicating backward running, the image is erased (S180) and this processing is terminated. On the other hand, if the shift lever is at the shift position indicating backward running, captured image data is taken from the camera 1. (S120), by performing coordinate transformation on the captured image data representing the image looking down on the ground surface from an obliquely upward direction, to generate converted image data representing a bird's eye view image looking down on the ground surface from a vertical direction ( S130).

The process of converting the image data captured by the camera 1 into converted image data can be realized by, for example, performing a process reverse to normal perspective transformation. Specifically, Japanese Patent Application Laid-Open No. H10-211849 Etc., and the description thereof is omitted here.

Next, the converted image data generated in S130 is compared with the peripheral image data corresponding to the peripheral image being displayed on the monitor 3 to extract an area outside the field of view of the camera 1. (S140), the extracted area (hereinafter referred to as “additional area”) is matched with the converted image and synthesized, and this is used as peripheral image data for updating the display on the monitor 3 (S150).

The extraction of the region outside the visual field is specifically performed as follows.
This can be done by using a well-known image matching method to specify a matching area between the converted image and the peripheral image and removing the matching area from the peripheral image. However, if there is no area that matches between the converted image and the peripheral image, it is determined that no additional area exists. And

Subsequently, a drawing mode corresponding to the size of the additional region extracted in S140 (the drawing region of the peripheral image and the drawing position of the image representing the own vehicle (hereinafter referred to as "own vehicle display") are defined). Is set (S160), the peripheral image data generated in S150 is supplied to the monitor 3 according to the set drawing mode, and the display of the own vehicle and the display indicating the viewing angle of the camera 1 are combined. (S1
70) The display corresponding to the set drawing mode is displayed on the monitor 3.
And terminates the present processing.

In this embodiment, three types of drawing modes are provided, a mode in which the entire vehicle is displayed in the own vehicle display mode, a mode in which approximately half of the rear side of the vehicle is displayed, and
Mode 3 is displayed in which about 1/4 of the rear side of the vehicle is displayed. As the mode 1 shifts to the mode 3, the camera 1
The area where the real-time image (converted image) is displayed is reduced, and conversely, the area where the image outside the visual field is displayed is enlarged.

Then, the ratio of the actually displayed out-of-view area to the entire area secured for display of the out-of-view area in the mode 1 is determined by the first threshold TH1 (for example, 20).
%), The mode 3 is selected. If the value is not less than the first threshold value TH1 and less than the second threshold value TH2 (for example, 50%), the mode 2 is selected. Have been.

S130 is an image conversion means, S14
0 and S150 correspond to the combining means, S170 corresponds to the vehicle display means, the display control means, and S160 corresponds to the display change means. In the vehicle periphery image display device of the present embodiment configured as described above, for example, when the vehicle is moved backward and enters the parking frame, when the shift lever is operated to the shift position indicating the backward running, the vehicle 3 is displayed on the monitor 3. An image of the parking frame drawn on the rear road surface is displayed.

At this time, since the parking frame has a depth when viewed from the position of the camera 1, the parking frame shown in the photographed image is distorted from its original shape (FIG. 3A, left diagram). reference). However, in the present embodiment, since the captured image is converted into a bird's-eye view image, the monitor 3 displays a parking frame W having an original shape without distortion, and also displays the host vehicle display C. (Refer to the right figure in FIG. 3A).

Also, as shown in FIG. 3A, only the image (converted image) SI behind the vehicle based on the real-time photographed image from the camera 1 is displayed on the monitor 3 immediately after the vehicle starts retreating. However, as the vehicle retreats, the images SO of the additional area outside the field of view of the camera 1 are also sequentially synthesized and displayed. Further, as the proportion of the image SO in the additional area increases, the display position of the host vehicle display C is changed stepwise so as to be closer to the center of the screen. The area is enlarged (see FIGS. 3B and 3C). Therefore, finally, as shown in FIG. 3C, the positional relationship between the parking frame W and the host vehicle C is determined not only by the rear part within the field of view of the camera 1 but also outside the field of view of the camera 1. The side portions will be clearly displayed.

As described above, in the vehicle peripheral image display device of the present embodiment, the monitor 3 for displaying the peripheral image
In addition to displaying the image of the own vehicle display C and the image of the viewing angle display A, an additional area outside the field of view of the camera 1 due to the movement of the vehicle is extracted from the past image, combined and displayed. Therefore, the positional relationship between the vehicle and the parking frame can be clearly understood.

Further, by changing the display position of the image representing the own vehicle position in accordance with the proportion of the image representing the additional region, the display region of the image based on the image captured by the camera 1 and the region outside the field of view are represented. The ratio of the image to the display area is changed so that the image information that can be provided is displayed to the maximum, so that the display of the peripheral image is easy to see, and necessary information from the peripheral image (for example, camera 1) (Such as the positional relationship with the parking frame on the side of the vehicle outside the field of view).

As a result, there is a remarkable effect that the operation of reversing the vehicle and parking the vehicle becomes extremely easy.
In the above-described embodiment, the display is switched in three stages according to the ratio of the displayed area outside the visual field.
You may comprise so that it may switch to a step or four or more steps. Further, the display may be continuously and continuously switched so that the area in which nothing is displayed becomes as small as possible.

Further, in the above embodiment, the photographed image from the camera 1 is directly converted into a bird's-eye view image. However, an image serving as a mark for vehicle operation such as a parking frame is extracted from the photographed image, and this extraction is performed. A configuration may be adopted in which only the converted image is subjected to coordinate conversion and this is displayed on the monitor 3.

Further, an image serving as a mark for operation of a vehicle such as a parking frame is configured not only to convert a photographed image into coordinates but to display the image, but also to highlight and display the image with colors and shades of color. You may.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram illustrating a main configuration of a vehicle rear image display device according to an embodiment.

FIG. 2 is a flowchart illustrating the content of a process executed by an image processing unit.

FIG. 3 is an explanatory diagram illustrating a change in display on a monitor.

FIG. 4 is an explanatory diagram showing a problem of the conventional device.

FIG. 5 is an explanatory diagram showing an operation of the conventional device and a problem.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Camera 3 ... In-vehicle monitor 5 ... Image processing unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 628 B60R 21/00 628D G06T 1/00 330 G06T 1/00 330A (72) Inventor Tetsuri Ishikawa 1-1-1, Showa-cho, Kariya-shi, Aichi, Japan DENSO Corporation (72) Katsuyuki Imanishi 14, Iwatani, Shimowasumi-machi, Nishio-shi, Aichi F-term in Japan Auto Parts Research Institute, Inc. CA16 CB08 CB12 CB16 CD01 CD05 CE08 CE09 CH18 DA08 DA16 DB02 DB09 5C054 CC05 CE02 FD03 FE14 FF03 HA30

Claims (3)

[Claims] 1. A display means for displaying various images, a photographing means attached to a vehicle so as to photograph obliquely downward, and a photographed image obtained by photographing by the photographing means is vertically displayed. Image conversion means for converting into an image representing a bird's-eye view looking down on the ground surface from a direction, and an area corresponding between the converted image generated by the image conversion means and the peripheral image being displayed on the display means In the case, a region that does not match the converted image is extracted from the peripheral image, and the extracted region is synthesized as an additional image with the converted image. A vehicle display unit for displaying a vehicle, and a display for updating a peripheral image displayed by the display unit with a converted image generated by the image conversion unit or a converted image obtained by synthesizing an additional image by the synthesizing unit. A vehicle surrounding image display device comprising: an area occupied by an additional image to be synthesized with the converted image by the synthesizing means, wherein the area occupied by the additional image is smaller than a preset lower limit. A vehicle periphery image display device, further comprising: a display change unit that changes a display area of the own vehicle by the display area and the vehicle display unit so that an image display area is enlarged. 2. The apparatus according to claim 1, wherein the photographing unit photographs an image behind the vehicle. 3. The image conversion unit according to claim 1, wherein the image conversion unit generates a converted image by extracting an image indicating a lane.
Or the vehicle peripheral image processing device according to 2.