JP2004356869A - Vehicle surrounding image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suitably provide a user with information for supporting the driving. <P>SOLUTION: In the vehicle surrounding image processing system 1, an illuminator 4 illuminates a photographing area covered by a rear camera 3, when the brightness of a picture taken by the rear camera 3 is not over a specified level. Even at night or in an indoor parking area with poor illumination, the illuminator 4 illuminates the photographing area covered by the rear camera 3 to raise the brightness of the picture taken by the rear camera 3, resulting in that it exceeds the specified level thereafter. This is enough to suitably generate and display a composite bird's-eye view picture, thus suitably providing a user with information for supporting the driving. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention is configured to generate a bird's-eye view image by performing coordinate transformation on an image in which a periphery of a vehicle is photographed, and to generate and display a composite bird's-eye view image by combining a past bird's-eye view image and a current bird's-eye view image. The present invention relates to a vehicle peripheral image processing system.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a situation behind a vehicle is photographed by a rear camera installed at a rear part of the vehicle, a coordinate of the photographed image is converted to generate a bird's-eye view (a plan view viewed from the sky), and a past bird's-eye view image is generated. There is one that combines a current bird's-eye view image to generate and display a combined bird's-eye view image (for example, see Patent Document 1).
[0003]
[Patent Document 1]
JP-A-2002-87160
[Problems to be solved by the invention]
However, at night or in an indoor parking lot with dark lighting, the brightness of the image captured by the rear camera becomes low (darkens), so that it is difficult to appropriately generate and display a composite bird's-eye view image, and to assist driving. There is a problem that information for performing the operation cannot be appropriately provided to the user. Further, when a bird's-eye view image generated by performing coordinate conversion on an image captured when the brake lamp is lit and a bird's-eye view image generated by performing coordinate conversion on an image captured when the brake lamp is not lit are combined, Since a bird's-eye view image with high luminance and a bird's-eye view image with low luminance are combined, a stripe pattern occurs due to a difference in luminance in the composite bird's-eye view image obtained by combining these bird's-eye view images, and the composite bird's-eye view image is In this case, there is a problem that information for assisting driving cannot be appropriately provided to the user.
[0005]
The present invention has been made in view of the above circumstances, and an object of the present invention is to generate a bird's-eye view image by performing coordinate transformation on an image in which a periphery of a vehicle is photographed, and to combine a past bird's-eye view image with a current bird's-eye view image. It is an object of the present invention to provide a vehicle peripheral image processing system which can appropriately provide a user with information for assisting driving in generating and displaying a composite bird's-eye view image.
[0006]
[Means for Solving the Problems]
According to the vehicle periphery image processing system described in claim 1, when the photographing means photographs the periphery of the vehicle, the bird's-eye view image generating means generates a bird's-eye view image by performing coordinate transformation on the image photographed by the photographing means. The storage unit stores the bird's-eye view image generated by the bird's-eye view image generation unit. When the vehicle moving state detecting means detects the moving state of the vehicle, the bird's-eye view image moving means detects the past bird's-eye view stored in the bird's-eye view image storing means based on the moving state of the vehicle detected by the vehicle moving state detecting means. The image is rotated or translated, and the combined bird's-eye view image generating means is coordinate-transformed by the bird's-eye view image generating means into a past bird's-eye view image rotated or translated by the bird's-eye view image moving means and an image taken by the photographing means. And a current bird's-eye view image generated as described above to generate a composite bird's-eye view image. At this time, the luminance determining unit determines whether the luminance of the image photographed by the photographing unit is equal to or higher than a predetermined level, and when it is detected that the luminance of the image photographed by the photographing unit is not higher than the predetermined level. The lighting control means operates the lighting means to illuminate a shooting area shot by the shooting means.
[0007]
Thus, even at night or in an indoor parking lot where the lighting is dark, the photographing area photographed by the photographing means is illuminated by the lighting means. As described above, unlike the related art, the synthesized bird's-eye view image can be appropriately generated and displayed, and information for assisting driving can be appropriately provided to the user.
[0008]
Further, since the photographing area photographed by the photographing means is illuminated by the illuminating means, even when the brake lamp is not lit, the brightness of the photographed image can be reduced when the brake lamp is lit. Since the bird's-eye view images having no difference in luminance will be combined thereafter, unlike the conventional one, the difference in luminance in the combined bird's-eye view image in which these bird's-eye view images are combined is different from the conventional one. As a result, a stripe pattern does not occur, and information for assisting driving can be appropriately provided to the user.
[0009]
According to the vehicle peripheral image processing system described in claim 2, the lighting control means operates the lighting means by changing the lighting level according to the luminance of the image photographed by the photographing means. Thus, by changing the illumination level so that the luminance of the image photographed by the photographing means is always constant, it is possible to display an easy-to-view synthesized bird's-eye view image with no more difference in luminance.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, FIG. 2 schematically shows a configuration of the vehicle periphery image processing system. The vehicle peripheral image processing system 1 includes a rear camera 3 (photographing means in the present invention) for photographing a situation behind the vehicle 2 and an illumination device 4 (illumination in the present invention) for illuminating a photographing area photographed by the rear camera 3. It is configured to include an illumination unit), an in-vehicle monitor 6 (display unit according to the present invention) arranged on a dashboard, for example, in a vehicle cabin 5, and an image processing unit 7 for performing image processing. In this case, an illumination area illuminated by the illumination device 4 (an area indicated by a two-dot chain line "P" in FIG. 2) is an imaging area to be imaged by the rear camera 3 (an area indicated by a two-dot chain line "Q" in FIG. 2). ) Or slightly larger.
[0011]
FIG. 1 shows a functional block diagram of the vehicle peripheral image processing system 1. The vehicle peripheral image processing system 1 has, as functional blocks, a CPU 8 that controls the operation of the entire system (vehicle moving state detecting means, bird's-eye view image moving means, synthetic bird's-eye view image generating means, luminance determining means, and lighting control means). A rear camera 3, an illuminating device 4, a bird's-eye view image generating circuit 9 (a bird's-eye view image generating means according to the present invention), a first image memory 10 (a bird's-eye view image storing means according to the present invention), and a second Image memory 11, display image memory 12, in-vehicle monitor 6, steering wheel signal input circuit 13, vehicle speed signal input circuit 14, yaw rate signal input circuit 15, ultrasonic sensor signal input circuit 16, shift position And a signal input circuit 17.
[0012]
When an image is input from the rear camera 3, the bird's-eye view image generation circuit 9 generates a bird's-eye view image by performing coordinate conversion on the input image, and outputs the generated bird's-eye view image to the first image memory 10. When a bird's-eye view image is input from the bird's-eye view image generation circuit 9, the first image memory 10 temporarily stores the input bird's-eye view image as a current bird's-eye view image.
[0013]
When the bird's-eye view image is transferred from the first image memory 10, the second image memory 11 stores the transferred bird's-eye view image as a past bird's-eye view image. The display image memory 12 displays a combined bird's-eye view image in which a current bird's-eye view image stored in the first image memory 10 and a past bird's-eye view image stored in the second image memory 11 are combined. To be stored. The in-vehicle monitor 6 displays a composite bird's-eye view image stored in the display image memory 12 as a display image.
[0014]
The handle angle signal input circuit 13 causes the CPU 8 to input data relating to the user's handle operation. The vehicle speed signal input circuit 14 causes the CPU 8 to input data relating to the vehicle speed. The yaw rate signal input circuit 15 causes the CPU 8 to input data relating to the turning state of the vehicle 2. The ultrasonic sensor signal input circuit 16 detects the presence or absence of an obstacle existing around the vehicle 2 and causes the CPU 8 to input data on the presence or absence of the obstacle and data on the distance between the obstacle and the vehicle 2. The shift position signal input circuit 17 causes the CPU 8 to input data relating to the operation of the shift lever by the user.
[0015]
In the above configuration, the CPU 8, the bird's-eye view image generation circuit 9, the first image memory 10, the second image memory 11, the display image memory 12, the steering wheel angle signal input circuit 13, the vehicle speed signal input circuit 14, the yaw rate signal The input circuit 15, the ultrasonic sensor signal input circuit 16, and the shift position signal input circuit 17 can be configured as one LSI by integrating all or a part of them. Further, instead of including the bird's-eye view image generation circuit 9 as a dedicated image processing circuit for generating a bird's-eye view image, the CPU 8 can generate the bird's-eye view image by software. Further, the in-vehicle monitor 6 can also be used as a display of a navigation device.
[0016]
Here, a procedure in which the bird's-eye view image generation circuit 9 generates a bird's-eye view image will be briefly described. As a precondition, it is assumed that the rear camera 3 has an autofocus function, and its mounting height and mounting angle are set in advance. In addition, the ground is assumed to be a plane as a calculation precondition. Here, the image taken by the rear camera 3 is decomposed in pixel units, the distance to each pixel and the width direction position are calculated using the rear end of the vehicle 2 as a reference line, and the position coordinates of each pixel after bird's-eye conversion are calculated. calculate. Then, each pixel is rearranged based on the calculated position coordinates. Thereby, a bird's-eye view image can be generated from an image captured by the rear camera 3.
[0017]
By performing the bird's-eye view conversion in this manner, in the bird's-eye view image, it is assumed that a gap may occur between pixels or the pixels may overlap with each other. At times, it is assumed that the image displayed on the in-vehicle monitor 6 is an image that is somewhat thinned out, and the operability is lower than the original image (raw image) captured by the rear camera 3 is displayed. Good thing.
[0018]
Next, the operation of the above configuration will be described with reference to FIG.
When the CPU 8 detects that an image has been input from the rear camera 3 in response to the image being captured by the rear camera 3 (“YES” in step S1), the brightness of the image captured by the rear camera 3 is reduced to a predetermined level. It is determined whether or not this is the case (step S2). In this case, the predetermined level is an index indicating whether or not the luminance is suitable (sufficient) for generating a bird's-eye view image thereafter, and may be a value stored in the CPU 8 in advance. It may be a value that is arbitrarily set by the user according to his / her preference (in accordance with the level requested by the user). In this case, the CPU 8 may determine the brightness of the entire image at once, or may average and determine the brightness of a plurality of locations of the image.
[0019]
When the CPU 8 detects that the brightness of the image captured by the rear camera 3 is equal to or higher than a predetermined level ("YES" in step S2), the first image memory 10, the second image memory 11, and the display The initialization image memory 12 and various parameters are initialized (step S3).
[0020]
On the other hand, when the CPU 8 detects that the brightness of the image captured by the rear camera 3 is not higher than the predetermined level (“NO” in step S2), the CPU 8 activates the illumination device 4 and captures the image by the rear camera 3. The illumination range is illuminated by the illumination device 4. In this case, the CPU 8 may change the illumination level in accordance with the luminance of the image captured by the rear camera 3 to perform the illumination, or may make the illumination level constantly constant to perform the illumination. After illuminating the photographing range photographed by the rear camera 3 with the illuminating device 4 (step S4), the CPU 8 stores the first image memory 10, the second image memory 11, and the display image memory. 12 and various parameters are initialized (step S3).
[0021]
Next, the CPU 8 detects that the image captured by the rear camera 3 is coordinate-transformed by the bird's-eye view image generation circuit 9 to generate a bird's-eye view image, and that the bird's-eye view image is stored in the first image memory 10 (step S5). Is "YES"), it is determined whether or not the bird's-eye view image is stored in the second image memory 11 (step S6).
[0022]
Here, when the CPU 8 detects that the bird's-eye view image is stored in the second image memory 11 (“YES” in step S6), the data input from the steering wheel angle signal input circuit 13 and the vehicle speed signal input It is determined whether the vehicle 2 is moving based on the data input from the circuit 14, the data input from the yaw rate signal input circuit 15, and the data input from the ultrasonic sensor signal input circuit 16 ( Step S7).
[0023]
Next, when detecting that the vehicle 2 is moving (“YES” in step S7), the CPU 8 reads the bird's-eye view image stored as the past bird's-eye view image from the second image memory 11 (step S8). Then, image processing for rotating or translating the read bird's-eye view image based on the moving state of the vehicle 2 is performed (step S9). At this time, if the vehicle 2 is moving in a curve, the CPU 8 performs image processing for rotating and moving the bird's-eye view image stored in the second image memory 11, and the vehicle 2 moves straight and moves. If so, image processing is performed to translate the bird's-eye view image stored in the second image memory 11 in parallel.
[0024]
Next, the CPU 8 reads the bird's-eye view image stored as the current bird's-eye view image from the first image memory 10 (step S10), and reads the past bird's-eye view image read from the second image memory 11 and subjected to image processing. And the current bird's-eye view image read from the first image memory 10 to generate a combined bird's-eye view image (step S11).
[0025]
Then, the CPU 8 outputs the synthesized bird's-eye view image generated in this way to the display image memory 12 and displays it on the in-vehicle monitor 6 (step S12). The synthesized bird's-eye view image is stored in the second image memory 11 in the past. The image is stored as a bird's-eye view image (step S13), the process returns to step S5, and the processes after step S5 are repeated.
[0026]
Further, when detecting that the bird's-eye view image is not stored in the second image memory 11 (“NO” in step S6), the CPU 8 stores the bird's-eye view image stored in the first image memory 10 as the current bird's-eye view image. The image is transferred to the second image memory 11 (step S14), the process returns to step S5, and the processes after step S5 are repeated. When detecting that the vehicle 2 is not moving (“NO” in step S7), the CPU 8 reads the bird's-eye view image read from the second image memory 11 and subjected to the image processing, and the first image The bird's-eye view image stored in the first image memory 10 as the current bird's-eye view image is directly displayed in the display area behind the vehicle figure without generating a combined bird's-eye view image by combining the bird's-eye view image read from the memory 10. It is displayed (step S15), the process returns to step S5, and the processes after step S5 are repeated.
[0027]
As described above, according to the present embodiment, in the vehicle peripheral image processing system 1, when the brightness of the image captured by the rear camera 3 is not equal to or higher than the predetermined level, the photographic area captured by the rear camera 3 is illuminated. Since the illumination device 4 is used to illuminate, the illuminating device 4 illuminates the shooting area captured by the rear camera 3 even at night or in an indoor parking lot where the illumination is dark. As a result, the brightness of the captured image becomes equal to or higher than a predetermined level, the synthesized bird's-eye view image can be appropriately generated and displayed, and information for assisting driving can be appropriately provided to the user.
[0028]
In addition, since the photographing area photographed by the rear camera 3 is illuminated by the illumination device 4 as described above, even when the brake lamp is not lit, the luminance of the photographed image is photographed when the brake lamp is lit. After that, bird's-eye view images having no difference in luminance are combined, and a stripe pattern is generated due to the difference in luminance in the combined bird's-eye view image obtained by combining these bird's-eye view images. Therefore, information for assisting driving can be appropriately provided to the user.
[0029]
The present invention is not limited to the above-described embodiment, but can be modified or expanded as follows.
When the brake lamp and the lighting device are configured to be directly linked to each other and the display function of the composite bird's-eye view image is used, the lighting device automatically stops operating when the brake lamp is turned on, and is turned off when the brake lamp is not turned on. The device may be configured to operate automatically.
[Brief description of the drawings]
FIG. 1 is a functional block diagram showing an embodiment of the present invention. FIG. 2 is a diagram schematically showing the entire configuration of a system. FIG. 3 is a flowchart.
In the drawings, 1 is a vehicle peripheral image processing system, 2 is a vehicle, 3 is a rear camera (photographing means), 4 is an illuminating device (illuminating means), 6 is an on-vehicle monitor (display means), and 8 is a CPU (vehicle moving state detection). Means, bird's-eye view image moving means, synthetic bird's-eye view image generating means, luminance determining means, lighting control means), 9 is a bird's-eye view image generating circuit (bird's-eye view image generating means), and 10 is a first image memory (bird's-eye view image storing means). .

Claims (2)

Photographing means for photographing around the vehicle;
Bird's-eye view image generating means for generating a bird's-eye view image by performing coordinate conversion on an image shot by the shooting means,
Bird's-eye view image storage means for storing a bird's-eye view image generated by the bird's-eye view image generation means,
Vehicle moving state detecting means for detecting a moving state of the vehicle;
A bird's-eye view image moving means for rotating or translating a past bird's-eye view image stored in the bird's-eye view image storage means based on the moving state of the vehicle detected by the vehicle moving state detecting means,
A past bird's-eye view image rotated or translated by the bird's-eye view image moving means is combined with a current bird's-eye view image generated by performing coordinate transformation on the image photographed by the photographing means by the bird's-eye view image generating means. A vehicle-surrounding image processing system comprising a synthetic bird's-eye view image generating means for generating a synthetic bird's-eye view image,
Brightness determination means for determining whether the brightness of the image captured by the imaging means is equal to or higher than a predetermined level,
Illuminating means for illuminating a photographing area photographed by the photographing means;
And a lighting control means for operating the lighting means when the brightness determination means detects that the brightness of the image shot by the shooting means is not higher than a predetermined level. system. The vehicle peripheral image processing system according to claim 1,
The vehicle peripheral image processing system according to claim 1, wherein the lighting control means operates the lighting means by changing a lighting level in accordance with a luminance of an image photographed by the photographing means.

Images