JP3941926B2 - Vehicle periphery monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a vehicle periphery monitoring device mounted on a vehicle.
[0002]
[Prior art]
Conventionally, a system has been proposed in which a camera is mounted on a vehicle, the surroundings are photographed, the obtained image is processed to recognize white lines on a road, and the recognition result is displayed on a monitor. For example, Japanese Patent Laid-Open No. 10-76272 discloses a vehicle periphery monitoring device that increases the sharpness of an image by applying edge enhancement processing to a captured image of a camera that can capture even near-infrared light, and presents this to the driver. Are listed.
[0003]
In order to increase the sharpness, that is, the sharpness of an image, for example, the Laplacian value that is the second derivative of the luminance value may be subtracted from the luminance value of each part of the image. In reality, since only discrete brightness of pixels can be obtained, the brightness value of each pixel after conversion is obtained using the brightness value of adjacent pixels.
g (i, j) = f (i, j) −Δf (i, j)
Can be obtained as
[0004]
Here, the position of the pixel is represented by a two-dimensional coordinate of a horizontal position i and a vertical position j. Δ is a digital Laplacian that is a second derivative of the input image f (i, j), and Δf (i, j) = f (i + 1, j) + f (i−1, j) + f (i, j + 1). + F (i, j-1) -4f (i, j)
By substituting this into Equation 1,
[0005]
g (i, j) = 5f (i, j)-[f (i + 1, j) + f (i-1, j) + f (i, j + 1) + f (i, j-1)]
It becomes. Such conversion can enhance the edge of the entire image and increase the sharpness of the image.
[0006]
By the way, applying edge enhancement using, for example, first-order differentiation or Laplacian to the image, or sharpening is the “Image Processing Engineering Basics” published by Keiji Taniguchi and Kyoritsu Publishing Co., Ltd. p.64, p.70. -77, which is a known technique.
[0007]
[Problems to be solved by the invention]
In vehicle periphery monitoring, the part that the driver wants to watch is often a part of the screen, and only that part is highlighted and displayed in an easy-to-understand manner, and the remaining part is left as it is, or conversely inconspicuous The driver is more effective because it makes it easier for the driver to find obstacles. For example, when the driver looks forward through the monitor screen while driving, what he really wants to watch is the situation of the traveling path of the vehicle, and the priority of information such as headlights and oncoming vehicles in diagonally opposite lanes is In most cases, it is quite low or rather noise and you want to eliminate it.
[0008]
However, in the above prior art, the same sharpening filter processing is performed on the entire image, so that the sharpening effect is obtained on the entire screen, and the driver's attention is distributed over the entire screen. In other words, the conspicuous filtering process emphasizes the parts corresponding to noise and the parts that do not need to be watched. There is a problem that it will be presented.
[0009]
An object of the present invention is to solve the above-described problems of the prior art, and it is possible to present a monitor screen that easily emphasizes only the point to be watched and makes it easy to see and draws attention to the driver. The object is to provide a vehicle periphery monitoring device.
[0010]
[Means for Solving the Problems]
The vehicle periphery monitoring device that achieves the above-described object provides a sharp image display in a partial area of the captured image and other areas excluding the partial area when displaying a captured image of the vehicle periphery by an in-vehicle camera on a monitor screen. It is characterized by comprising control means for varying the degree. With this configuration, the driver's attention looking at the monitor screen can be directed to images with high sharpness relative to the image display in some areas and other areas, and the driver's attention can be concentrated on images with high sharpness. It becomes possible.
[0011]
In addition, the vehicle periphery monitoring device that achieves the above object sharpens a part of the captured image and displays it on the monitor screen when the captured image around the vehicle by the in-vehicle camera is displayed on the monitor screen. Control means for unsharpening and displaying other areas excluding the partial area on the monitor screen. With this configuration, in the same manner as described above, the driver who watches the monitor screen can be directed to a partial area where the driver's attention is sharpened regardless of day or night. The partial area where the captured image is sharpened is determined according to a predetermined rule or according to the current road surrounding situation and vehicle situation.
[0012]
Preferably, the part of the captured image in which the sharpening is performed is an area that falls within a required range with respect to a vehicle traveling prediction direction of the captured image displayed on the monitor screen. . With this configuration, the driver's attention can always be directed in the vehicle traveling direction.
[0013]
Preferably, the partial area is widened to a far viewable range as the vehicle speed increases. With this configuration, when driving at a high speed, it is possible to gaze at an image sharpened far away.
[0014]
More preferably, the display position of the partial area on the monitor screen changes according to the vehicle travel prediction direction. With this configuration, even when the vehicle traveling direction changes, the driver's attention can always be directed to the vehicle traveling direction.
[0015]
More preferably, the vehicle travel prediction direction is predicted by a car navigation system or is predicted from a detection value of a steering angle sensor attached to a steering wheel. With this configuration, the vehicle traveling direction can be predicted with high accuracy, and the driver's attention range can be accurately directed toward the vehicle traveling direction.
[0016]
More preferably, when a corresponding portion of the captured image is displayed in the partial area, the sensitivity of the pixel data is increased or the display color is emphasized. With this configuration, it is possible to further pay attention to the gaze target in the sharpened image.
[0017]
More preferably, the unsharpened image to be displayed is an image of the captured image as it is or a blurred image of the captured image. With this configuration, the driver's attention to the unsharpened image is reduced, and the driver's attention can be directed to the sharpened image.
[0018]
More preferably, the sharpening process is performed by subtracting a Laplacian value that is a second derivative of the luminance value from the luminance values of the adjacent pixels in the horizontal direction of the individual pixels of the captured image and the peripheral pixels. It is characterized by. With this configuration, the filter means can be easily configured, and the calculation speed can be improved.
[0019]
More preferably, the sharpening is performed by filtering each of the RGB signals of the captured image. This configuration further facilitates the configuration of the filter means and enables high-speed processing.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0021]
(First embodiment)
FIG. 1 is a block configuration diagram of a vehicle periphery monitoring device according to a first embodiment of the present invention. This vehicle periphery monitoring device is mounted on a vehicle such as a passenger car, and includes an imaging unit 1, an imaging data recording unit 2 that records imaging data of the imaging unit 1, an arithmetic control unit 9, and an arithmetic control unit. 9 includes image processing data recording means 5 for recording image data of the result of image processing according to 9, and display means 6 for displaying an image of the result of image processing.
[0022]
The arithmetic control unit 9 includes an image processing content determination unit 3 that determines the image processing content of the imaging data, and a filter unit 4 that performs image processing on the imaging data of the imaging unit 1 based on the processing content determined by the image processing content determination unit 3. It consists of. The image processing content determination unit 3 and the filter unit 4 may be configured by dedicated hardware, or may be configured by software processing by a CPU (Central Processing Unit) or DSP (Digital Signal Processor).
[0023]
The image pickup means 1 is a camera incorporating an image pickup device such as a CCD or a CMOS, for example, and can collect light from a visible light region to a near infrared light region and pick up a subject image. The captured image of the imaging unit 1 is digitized and then recorded in the imaging data recording unit 2. The imaging data recording means 2 is composed of, for example, a semiconductor memory, and temporarily stores imaging data so that subsequent processing is easy. If the subsequent processing is possible in real time, it can be omitted. is there.
[0024]
The image processing content determination means 3 determines which area of the acquired captured image is to be emphasized or sharpened in order to make the driver gaze (this area is hereinafter referred to as “A area”), and conversely which. The part is irrelevant to the driver and determines whether it is desired to keep away from consciousness or to prevent the influence of disturbance as much as possible (this area is hereinafter referred to as “B area”).
[0025]
FIG. 2 is an explanatory diagram illustrating an example of the A area and the B area. In this example, as shown in FIG. 2A, the host vehicle 10 travels straight on a straight road 11. On the monitor screen 6a (FIG. 2B) displayed on the display means 6 (FIG. 1), a picked-up image of the image pickup means 1, that is, a road image 11a extending to the horizontal line 14 in front of the host vehicle 10 is displayed. The The display image of this screen 6a is divided into A area and B area other than that, the image of A area is sharpened and displayed on the monitor screen 6a, and the image of B area remains as a captured image or is blurred. Displayed as an image.
[0026]
In the example of FIG. 2, first, a range 12 of “vehicle width + 1 m” in the traveling direction of the host vehicle 10 shown in FIG. Then, a range obtained by combining the captured image (trapezoid range) 12a on the monitor screen 6a in the range 12 and the spatial range 12c obtained by lifting the trapezoid range 12a in the vertical direction by 2 m is hatched in the A area (FIG. 2C). ), That is, a required range for sharpening processing, and the other area is defined as area B. As a result, the image of a pedestrian walking in the range of “vehicle width + 1 m” in the straight direction of the host vehicle 10 is sharpened, and when the pedestrian is in the vehicle traveling direction, the driver's attention is concentrated on the pedestrian. Is possible.
[0027]
The filter means 4 of FIG. 1 determines an appropriate filter or filter coefficient for each according to the content specified by the image processing content determination means 3, and processes the captured image. For example, if the area A is to be sharpened, first, a Gaussian filter or a low-pass filter is passed to remove the influence of image noise. For example, if it is a low-pass filter,
d (i, j) = 0.5 * f (i, j) + 0.125 * [f (i + 1, j) + f (i-1, j) + f (i, j + 1) + f (i, j-1) ]
The following calculation is performed on each pixel. However, the values of 0.5 and 0.125 are adjusted depending on the degree of the low-pass filter. After that, sharpening treatment is performed as in the conventional example,
g (i, j) = d (i, j) −α * Δd (i, j)
The luminance value of each pixel in the A area is obtained Here, Δ is a digital Laplacian that is a second derivative of the input image d (i, j), and α is an appropriate weighting factor.
[0028]
In the case of pixel data of area B that does not need to be watched, it may be blurred only through a Gaussian filter or a low-pass filter, or simply reduced in luminance. For example,
g (i, j) = 0.5 * f (i, j)
As described above, the luminance value of each pixel in area B is obtained. Note that the coefficients of the above-described filters may be common to the areas, or may be changed for each pixel or for each block that is a set of several pixels.
[0029]
The result of filtering the individual pixels by the filter unit 4 is recorded in the image processing data recording unit 5. The image processing data recording means 5 is composed of, for example, a semiconductor memory. Of course, the image processing data recording means 5 may be shared with the imaging data recording means 2. The display means 6 is installed in a place where the driver can safely view the display unit, and provides the driver with the image of the image processing data recording means 5. Note that the image processing data recording means 5 can be omitted if the data can be sent to the display means 6 in real time from the filter processing.
[0030]
FIG. 3 is a diagram showing an actual image example to which the first embodiment of the present invention is applied. FIG. 3A shows the captured image (original image) of the imaging means 1 as it is. A pedestrian is walking in the traveling direction of the host vehicle, and a street lamp is shown on the upper right side of the screen. FIG. 3B shows an image obtained by sharpening this original image on the entire screen as in the prior art. In this conventional image, the pedestrian image is sharpened, but the image of the street light is also sharpened at the same time, and the driver's attention is distributed not only to the pedestrian but also to the street light.
[0031]
Therefore, in the present embodiment, the screen is divided into the A area and the other B areas, and the image of the A area is sharpened and the image of the B area is displayed on the screen as a blurred image. FIG. 3C shows an area division range. In the A area described with reference to FIG. 2, the upper edge of the A area is a range obtained by lifting the range 12 of the vehicle width + 1 m by 2 m. However, the image pickup means 1 is fixed to the vehicle, and can be set in advance in general, rather than obtaining the upper edge of the A area by calculation each time, and the calculation speed becomes faster.
[0032]
Therefore, in this embodiment, as shown in FIG. 3C, the boundary K between the A area and the B area is determined in advance. Then, as shown in FIG. 3D, only the image of the A area is sharpened, and the image of the B area is displayed in a blurred manner. As a result, the streetlight image is blurred and the pedestrian image is sharpened, so that the driver's attention can be concentrated on the pedestrian.
[0033]
As described above, according to the present embodiment, it is possible to effectively and clearly present only the image of the area A that the driver wants to watch, and to provide an excellent vehicle periphery monitoring device.
[0034]
(Second Embodiment)
Next, a second embodiment of the present invention will be described. Since the configuration of the vehicle periphery monitoring device in the present embodiment is the same as that in FIG. 1, the description thereof is omitted. In the present embodiment, the imaging sensitivity of the imaging means can be changed for each pixel. For example, when an imaging pixel is a CMOS sensor, each pixel data is photoelectrically converted by a corresponding photodiode and amplified by a subsequent transistor circuit. However, the amplification factor is increased in the above noted area (A area). In other areas (B area), it is lowered.
[0035]
Subsequent processing is the same as in the first embodiment. With this configuration, it is possible to provide an excellent vehicle periphery monitoring apparatus that can change not only the filter but also the sensitivity of image capturing and present only the A area to be noticed to the driver more clearly and effectively. Become.
[0036]
(Third embodiment)
Next, a third embodiment of the present invention will be described. Although the vehicle periphery monitoring apparatus of this embodiment is the same as the structure shown in FIG. 1, the image processing content determination means 3 of this embodiment determines which part of the image shows the traveling path in advance when the host vehicle goes straight ahead. Keep track of. Thereby, the processing content can be determined so as to sharpen the above-mentioned portion that should be the expected course of the host vehicle and the vicinity thereof as an A area to be watched.
[0037]
For example, in the example of FIG. 3, the predicted straight course of the host vehicle is displayed at the center of the screen, so the center of the screen is sharpened, but the driving lane of the host vehicle is displayed on the left half of the screen. When monitor adjustment is performed, an A area is set around the left half of the screen, and the image is sharpened.
[0038]
Even on the same traveling path, if the speed of the host vehicle is high, it is necessary to look farther, so the vehicle speed is estimated from the speed sensor or the optical flow of the image, and the higher the speed, the sharper the distance. To do. That is, the boundary K in FIG. 3C is moved upward, and the A area is further extended. Thereby, the image range to be sharpened can be changed according to the vehicle speed. Thus, for example, an image in a range up to 200 m ahead is sharpened while traveling at 60 km, and an image in a range up to 500 m forward is sharpened while traveling 100 km on a highway.
[0039]
The above example is an example in which the imaging means for monitoring the front of the vehicle is mounted on the vehicle. However, when the vehicle for monitoring the rear of the vehicle is mounted on the vehicle and the vehicle moves backward, By applying the form, the same effect as described above can be obtained.
[0040]
(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. FIG. 4 is a block configuration diagram of the vehicle periphery monitoring device according to the present embodiment. The vehicle periphery monitoring device according to the present embodiment includes a navigation system 7 in addition to the configuration shown in FIG. 1, and the image processing content determination means 3 determines the processing content, particularly the range of area A based on the signal from the navigation system 7. It is characterized by determining.
[0041]
The car navigation system 7 according to the present embodiment estimates a current position and a traveling road from position detection means such as GPS and map data, and what kind of road shape place will be passed in the future based on the road information. Predict. Based on this prediction, the image processing content determination means 3 determines the range of the A area based on the map data in the future progress prediction direction.
[0042]
For example, as shown in FIG. 5 (a), when it is estimated by the map data that the future progress predicted road is the straight road 16 with respect to the own vehicle position 15, as shown in FIG. An image center direction overlapping the straight path image 16a is set as an object to be watched, that is, a sharpening process target (A area).
[0043]
Further, as shown in FIG. 6 (a), when it is estimated from the map data that the future progress predicted road is the curved road 17 with respect to the own vehicle position 15, the curved road as shown in FIG. 6 (b). The A area to be watched is shifted from the center of the screen in the direction of bending according to the direction of the image 17a and the road curvature.
[0044]
As described above, according to the present embodiment, it is possible to provide an excellent vehicle periphery monitoring device that can make a driver gaze only in a necessary direction while estimating a change in terrain in advance.
[0045]
(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. FIG. 7 is a block configuration diagram of the vehicle periphery monitoring device according to the present embodiment. The vehicle periphery monitoring apparatus according to the present embodiment includes a steering rudder angle detection unit 8 in addition to the configuration of FIG. 4, and the image processing content determination unit 3 includes not only the car navigation system 7 but also the steering rudder angle detection unit 8. The processing content, in particular, the range of the A area is determined according to the detection value. The steering rudder angle detection means 8 detects the rotation angle of the steering by using, for example, a hall element, and the image processing content determination means 3 estimates the amount of bending of the tire based on this detection value.
[0046]
Since the image processing content determination means 3 can know the direction in which the vehicle will travel from the current time based on the detection value of the steering angle detection means 8, the area to be watched, that is, the A to be sharpened according to the direction. The area can be shifted in the direction of turning from the center. This makes it possible to always sharpen the image of the direction of travel of the vehicle and to blur the rest, allowing the driver to turn off the steering wheel and watch only the direction he wants to proceed in the future. An apparatus can be provided.
[0047]
(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. The configuration of the vehicle periphery monitoring device according to the present embodiment is the same as the configuration of any one of FIGS. 1, 4 and 7, and the filter means 4 of the present embodiment is only required to sharpen the area A to be noticed. In addition, processing for emphasizing a color or tone specified in advance is also performed. For example, the white line and the person on the road shown in FIG. 3 appear white. However, it is better to display the yellow color near the display color “white” on the screen, or to increase the brightness. It is easy to attract attention.
[0048]
Alternatively, only the red color is emphasized in the color image that projects the front. Specifically, when R is an RGB value and R is larger than G and B by a certain amount, it is determined that the color is reddish, and the R value is increased to further increase the redness of the color. As a result, only the stop lamp of the car running ahead and the red color of the traffic light are emphasized over the other colors.
[0049]
As described above, according to the present embodiment, it is possible to provide an excellent vehicle periphery monitoring apparatus that not only sharpens the captured image but also enhances the color and enhances the gaze effect.
[0050]
(Seventh embodiment)
Next, a seventh embodiment of the present invention will be described. The configuration of the vehicle periphery monitoring device of the present embodiment is also the same as the configuration of any of FIGS. 1, 4, and 7, and the filter means 4 of the present embodiment is adjacent to the target pixel in the horizontal direction. , And peripheral pixels constitute a filter. For example,
g (i, j) = f (i, j) -h (i, j)
In order to obtain simultaneously the effect of noise removal by the low-pass filter and the effect of brightness sharpening by the Laplacian filter, the value of h corresponding to a filter having both functions of the low-pass filter and the Laplacian filter is obtained.
h (i.j) = f (i−2, j) + 2f (i−1, j) −6f (i, j) + 2f (i + 1, j) + f (i + 2, j)
And only the pixels in the horizontal direction.
[0051]
Even with this filter, vertical edges on the screen, for example, the edges of people or white lines, can be captured well, and the amount of calculation can be saved by not using vertical pixel information. It can be simplified.
[0052]
Thereby, in this embodiment, desired sharpening can be realized with a small amount of calculation and simple hardware.
[0053]
(Eighth embodiment)
Next, an eighth embodiment of the present invention will be described. The configuration of the vehicle periphery monitoring device of the present embodiment is also the same as the configuration of any one of FIGS. In the embodiment described above, the filter unit 4 performs the above-described filter calculation on the luminance value of the captured image. For example, if the captured image is a color image, YUV conversion of RGB three values is performed to obtain a YUV signal standardized by color television, and the luminance information Y value is filtered. However, the present embodiment is characterized in that the sharpening process is performed by filtering each value of RGB without obtaining the luminance information Y.
[0054]
The filter processing in the filter means 4 for sharpening the A area is aimed at emphasis and sharpening for the driver to pay attention. For this reason, since a slight change in color can be ignored, the filter unit 4 of this embodiment performs a filtering process for each color of RGB and obtains the same sharpening effect as in the first to sixth embodiments. In addition, since the filter means can be configured for each color, the configuration of the filter for performing color enhancement is also inexpensive.
[0055]
That is, in the present embodiment, for example, the filter calculation processing of Equation 4 to Equation 6 is performed for each of the RGB values obtained in each pixel. Thus, an enhancement filter can be configured using RGB values without bothering to obtain an amplitude value (luminance value), and a simple and effective vehicle periphery monitoring device can be provided.
[0056]
In each of the embodiments described above, the example of sharpening the portion to be watched has been described.However, even if the portion to be watched is displayed as a captured image and the portion that is not to be watched is blurred and displayed, Needless to say, the same effect can be obtained. In short, it is only necessary to display the part that is desired to be watched and the part that is not so different.
[0057]
Moreover, not only the sharpening area and the other areas but also a region having an intermediate sharpness level may be present. The sharpening type is not limited to the example described in the embodiment, in which the brightness is increased, the white color is displayed in yellow, or the red color is emphasized. The range that the driver should pay attention to (see above) Compared to other areas in the monitor screen (area where the driver does not have to pay attention: area B above) so that the driver's attention concentrates on the image display in area A) It is sufficient that can be visually recognized more clearly. Therefore, the same applies to blurring, and is not limited to display with reduced brightness or reduced image resolution. The image is completely blurred so that the driver's attention is not suitable for B area image display. It is also possible not to see.
[0058]
【The invention's effect】
Advantageous Effects of Invention According to the present invention, a vehicle that can display an image of an area that a driver wants to watch from a captured image is sharper than other areas, and can present a screen that is easy to call the driver. A peripheral monitoring device can be provided.
[Brief description of the drawings]
FIG. 1 is a block configuration diagram of a vehicle periphery monitoring device according to a first embodiment of the present invention. FIG. 2 is an explanation of a sharpening area (A area) in the vehicle periphery monitoring device according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating a display image in the vehicle periphery monitoring device according to the first embodiment of the present invention. FIG. 4 is a block configuration diagram of the vehicle periphery monitoring device according to the fourth embodiment of the present invention. 5 is an explanatory view of a sharpening area when a vehicle traveling path is predicted to be a straight traveling path in a vehicle periphery monitoring apparatus according to a fourth embodiment of the present invention. FIG. 6 is a view of a vehicle periphery according to a fourth embodiment of the present invention. FIG. 7 is an explanatory diagram of a sharpening area when a vehicle traveling path is predicted to be a curved path in the monitoring apparatus. FIG. 7 is a block diagram of a vehicle periphery monitoring apparatus according to a fifth embodiment of the present invention.
DESCRIPTION OF SYMBOLS 1 Imaging means 2 Imaging data recording means 3 Image processing content determination means 4 Filter means 5 Image processing data recording means 6 Display means 7 Car navigation system 8 Steering angle detection means 9 Calculation control means

Claims (5)

When displaying a captured image of surroundings of the vehicle by the vehicle-mounted camera on a monitor screen, a control means for varying the sharpness of the image displayed on the other area except a part area of the portion area of the captured image, the one The vehicle periphery monitoring device , wherein the partial area is an area within a required range in the vehicle travel prediction direction displayed on the monitor screen . When a captured image of a vehicle periphery by an in-vehicle camera is displayed on a monitor screen, a part of the captured image is sharpened and displayed on the monitor screen, and other areas excluding the partial area from the captured image are not displayed. A vehicle periphery monitoring device comprising control means for sharpening and displaying on the monitor screen , wherein the partial area is an area within a required range of a vehicle travel prediction direction displayed on the monitor screen . 3. The vehicle periphery monitoring device according to claim 1, wherein the partial area is expanded to a range that can be viewed far away as the vehicle speed increases. 4. The display position on the monitor screen some areas, the vehicle surroundings monitoring apparatus according to any one of claims 1 to 3, characterized in that changes in accordance with the vehicle traveling prediction direction. The vehicle traveling prediction direction is predicted by a car navigation system or the vehicle periphery according to any one of claims 1 to 4, characterized in that predicted from the detected value of the steering angle sensor attached to a steering Monitoring device.

Images