JP2009206939A - Image display system for vehicle, image display device, and image enhancement method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly enhance a specific region where any object requiring special attention in a display image exists, and to validly enhance the cognitiveness of the object requiring special attention which should be originally enhanced. <P>SOLUTION: Among display images obtained by enhancing a region where an object requiring special attention, such as, another vehicle approaching a self-vehicle exists from among images picked up by an in-vehicle camera 1, and displaying it on a display 2, a display image generating part 12 of an image processing unit 10 controls the space-time frequency component of the display image to enhance the region where the object requiring special attention exists. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image display system for a vehicle, an image display apparatus, and an image enhancement method for emphasizing and displaying a target object requiring attention from a driver among images around a vehicle imaged by an imaging means. .

2. Description of the Related Art Conventionally, in a vehicle image display system that displays an image around a vehicle captured by an in-vehicle camera on a display device in a vehicle interior, an object to be watched by a driver is detected from the image captured by the in-vehicle camera. Technology that makes it easy to recognize objects of interest in images by performing enhancement processing such as edge emphasis, saturation emphasis, frame attachment, etc. on the display object and displaying them on a display device. (For example, refer to Patent Document 1).
JP-A-9-48282

  However, in the conventional technique as described in Patent Document 1, an enhancement process such as edge enhancement, saturation enhancement, and frame attachment is performed on a target object in an image and displayed on a display device. For this reason, the emphasis process itself is too conspicuous, and the cognition of the target object that should be emphasized may be lowered.

  The present invention has been made in view of the above-described problems of the prior art, and properly emphasizes a specific area where a target object or the like in a display image is present, and emphasizes the original area. It is an object of the present invention to provide an image display system for a vehicle, an image display device, and an image enhancement method that can effectively enhance the recognition of an object to be watched.

  In the present invention, when emphasizing a specific region in which a target object or the like in the display image is present, the region is emphasized by controlling the spatio-temporal frequency composition of the display image. Solve the problem.

  According to the present invention, it is possible to appropriately emphasize a specific area to which an observer should pay attention without making the emphasis processing itself excessively conspicuous, and it is essential to emphasize an object requiring attention in the area. Effectively enhance the cognition of the subject.

  Hereinafter, as an embodiment of the present invention, an example in which the present invention is applied to a vehicle image display system that displays an image around a vehicle captured by an in-vehicle camera on a display device in a vehicle interior will be described in detail with reference to the drawings. .

  FIG. 1 is a block diagram showing a configuration of a vehicle image display system to which the present invention is applied. As shown in FIG. 1, the vehicle image display system includes an in-vehicle camera 1 connected to the input side of the image processing unit 10 and a display 2 connected to the output side of the image processing unit 10.

  The in-vehicle camera 1 is installed at a predetermined position of a vehicle (hereinafter referred to as a host vehicle) on which the vehicle image display system of the present embodiment is mounted, for example, a front grill or a roof spoiler of the host vehicle. An image of a predetermined range around the host vehicle such as the rear of the vehicle is taken. This in-vehicle camera 1 has an optical system composed of a lens and a light receiving element. Light received through the lens is received by a light receiving element such as a CCD or a CMOS and converted into an electrical signal. Output as a digitized image signal. An image around the host vehicle captured by the in-vehicle camera 1 is input to the image processing unit 10 as needed.

  The display 2 is installed at a position that is easy for the driver to see, such as a center console in the passenger compartment of the host vehicle, and displays an image of the surroundings of the host vehicle captured by the in-vehicle camera 1 based on an image signal output from the image processing unit 10. To do. Here, there is a cautionary object that the driver of the own vehicle should pay attention to when performing safe driving, such as another vehicle approaching the own vehicle, around the own vehicle. When the image is taken by the camera 1, the display 2 displays an image that emphasizes the area where the target object needs attention.

  The image processing unit 10 includes an information processing device such as a microcomputer in which a predetermined processing program is stored, and the host vehicle captured by the in-vehicle camera 1 by executing this program with an arithmetic device such as a CPU. The surrounding image is processed to generate a display image, and the generated display image is displayed on the display 2. When the configuration of the image processing unit 10 is understood functionally, as shown in FIG. 1, the camera image input unit 11, the target object detection unit 12, the enhancement region specifying unit 13, and the display image generation are performed. The unit 14 and the display image output unit 15 are provided.

  The camera image input unit 11 includes an input buffer corresponding to the frame size of an image captured by the in-vehicle camera 1. Data of an image around the host vehicle photographed by the in-vehicle camera 1 is stored in the input buffer of the camera image input unit 11 while being updated every frame.

  The object-of-interest detection unit 12 is a known object such as an edge detection process or a template matching process for an image around the host vehicle captured by the in-vehicle camera 1 and stored in the input buffer of the camera image input unit 11. Detection processing is performed, and an object requiring attention such as another vehicle approaching the host vehicle is detected from an image around the host vehicle captured by the in-vehicle camera 1. For example, a method for determining whether an object detected on an image is a target object is to determine whether it is an approaching object by obtaining an optical flow of an object that is a candidate for the target object from a temporal change in the image. For example, a known method may be used.

  Based on the detection result of the cautionary object detection unit 12, the emphasis region specifying unit 13 automatically captures the image of the surroundings of the host vehicle captured by the in-vehicle camera 1 and stored in the input buffer of the camera image input unit 11. An area in which an object requiring attention such as another vehicle approaching the vehicle exists is specified, and this area is determined as an area to be emphasized.

  The display image generation unit 14 processes an image around the host vehicle captured by the in-vehicle camera 1 and stored in the input buffer of the camera image input unit 11 to generate a display image to be displayed on the display 2. That is, in the display image generation unit 14, a target object around the host vehicle is detected by the target object detection unit 12, and a target object in the captured image of the in-vehicle camera 1 exists by the enhancement region specifying unit 13. When the area to be identified is specified, the image captured by the in-vehicle camera 1 is processed so that the area where the target object is present is emphasized, and a display image to be displayed on the display 2 is generated. At this time, the display image generation unit 14 particularly emphasizes a region where a target object in the captured image of the in-vehicle camera 1 exists by controlling the spatiotemporal frequency composition of the display image, as will be described in detail later. I have to.

  The display image output unit 15 includes an output buffer corresponding to the frame size of the display image displayed on the display 2. The display image data generated by the display image generation unit 14 is stored in this output buffer. The display image output unit 15 outputs the display image data stored in the output buffer to the display 2 as needed. As a result, the display 2 displays an image around the host vehicle imaged by the in-vehicle camera 1, and a target object such as another vehicle approaching the host vehicle exists in the imaging range of the in-vehicle camera 1. In this case, an image in which the region where the target object needs attention is emphasized in the captured image of the in-vehicle camera 1 is displayed.

  Here, the display image enhancement processing by the display image generation unit 14 which is characteristic in the vehicle image display system of the present embodiment will be described in more detail.

  As a technique for emphasizing a specific area that should be particularly noticed by the observer in the display image displayed on the display 2, a technique of surrounding the area with a frame image having an edge is widely adopted. This has the following two problems.

  First, when a specific area is surrounded by a frame image having an edge, the visibility of an object that is adjacent to the edge, that is, in this case, a specific area surrounded by the frame image There is a problem that decreases. This is called spatial frequency masking and is a phenomenon that occurs even if the frame image itself does not occlude the object, and its effect is known to be a function of distance (Funakawa Masami “impacts readability. "Experimental research on spatial factors", Journal of Lighting Society, 84, 11, 793-798, 2000). Further, even if the frame image or the object in the region to be emphasized is blinked temporally with rectangular wave luminance modulation, the visibility of the surrounding objects is similarly reduced. This is called time frequency masking.

  Second, in a situation where the display image displayed on the display 2 is within the field of view but the observer is not gazing at it, there is a frame image having a sharp edge in the display image, or the frame image If the object in the area surrounded by the frame image is blinking, the eye-catching property is high, which causes inappropriate gaze movement and interrupts the viewer to see what he / she originally should see. There is a problem. Further, even if the observer's line of sight does not actually move, an annoying impression is given.

  Therefore, in the vehicle image display system of the present embodiment, the display image generation unit 12 of the image processing unit 10 controls the spatio-temporal frequency composition of the display image when emphasizing a specific region in the display image. By emphasizing a specific area, it is possible to suppress a reduction in the visibility of a target object in the specific area and to guide the line of sight to the emphasis process itself. It is designed to properly guide attention to things.

  Specifically, the display image generation unit 14 of the image processing unit 10, for example, a frame image obtained by performing a low-pass filtering process on a region in which a target object is present in a captured image of the in-vehicle camera 1. The region where the target object needs attention is emphasized. That is, the display image generation unit 14 generates a display image by drawing a frame image surrounding an area where a target object needs attention with respect to a captured image of the in-vehicle camera 1. Apply a spatio-temporal low-pass filtering process to the frame image so that the region where the target object is present is surrounded by the blurred frame image without edges to emphasize the region where the target object exists. Let

  A frame image that has been subjected to low-pass filtering in spatio-temporal manner does not mask high spatio-temporal frequencies required for recognizing a target object in a region surrounded by the frame image. Also, frame images that have undergone low-pass filtering in spatio-temporal space do not have spatio-temporal edge components that change their appearance depending on the field of view. Does not trigger. Furthermore, since the frame image without an edge does not cause the central vision, it is not troublesome. In this way, by enclosing the area where the object of interest is present in the in-vehicle camera 1 with a frame image using a band that does not overlap with the spatio-temporal frequency band necessary for recognition of the object of interest within the area. By emphasizing the area, avoiding the degradation of visibility due to spatio-temporal frequency masking or shielding to the target object due to the frame image itself, and inappropriate gaze movement induction due to the high attraction of the frame image itself Can do.

  2 to 4 are diagrams illustrating specific examples of display images generated by the display image generation unit 14. The image example shown in FIG. 2 is a display image P when the in-vehicle camera 1 captures an image in front of the host vehicle and a bicycle traveling on the shoulder on the left front of the host vehicle is detected as the object T of interest. In the image example shown in FIG. 3, the in-vehicle camera 1 captures an image in front of the host vehicle, and the oncoming vehicle waiting for a right turn in front of the host vehicle is detected as the target object T to be watched for. It is an example of the display image P in the case of. Further, in the image example shown in FIG. 4, the in-vehicle camera 1 captures an image of the rear side of the host vehicle, and a two-wheeled vehicle that is about to pass the host vehicle from the right rear side of the host vehicle is detected as the attention target T. This is an example of the display image P.

  As shown in the image examples of FIGS. 2 to 4, a frame image F obtained by applying a low-pass filtering process in time and space to an area where a target object T is present in a captured image of the in-vehicle camera 1, that is, an edge By enclosing the image with the blurred frame image F having no frame, the region where the target object T needs to be present can be appropriately emphasized without making the frame image F excessively conspicuous, and the frame image F is included in this region. It is possible to effectively suppress the inconvenience of masking spatio-temporal frequency components that are important in recognizing the target object T requiring attention, and to enhance the recognition of the target object T requiring attention.

  It should be noted that the color of the frame image surrounding the area where the target object needs attention is preferably changed according to the potential level of attention required for the target object present in the area. For example, as shown in the image example of FIG. 2, when a bicycle running on the shoulder is detected as the object T requiring attention, the bicycle may change its course or fall down to the roadway side. Since the level is high, the frame image F is set to a red display color. Further, as shown in the image example of FIG. 3, when an oncoming vehicle waiting for a right turn is detected as the object T requiring attention, this oncoming vehicle is a target to be recognized, but is a stopped vehicle and Since the stop position is also appropriate and the level of caution is not so high, the frame image F is set to a yellow display color. As described above, if the display color of the frame image is changed according to the level of attention of the target object requiring attention, it is possible to more appropriately alert the driver of the host vehicle.

  Further, when the display image generation unit 14 performs the low-pass filtering process on the frame image surrounding the area where the attention target object exists, the display image generation unit 14 sets the cutoff frequency of the low-pass filtering process to the area where the attention object exists. It is desirable to control in accordance with the size of.

  When the area where the target object needs attention is a relatively small area, the spatial frequency required for recognizing the target object within that area is relatively high. Conversely, the area where the target object exists is present. In the case of a relatively small area, in the case of a large object, the spatial frequency required for recognizing a target object requiring attention in that area is relatively low. Therefore, the region where the target object exists is controlled by controlling the cutoff frequency of the low-pass filtering process for the frame image surrounding the region where the target object exists, according to the size of the region where the target object exists. If the area is small, the spatial frequency component of the frame image surrounding the area is increased, and if the area where the target object is present is large, the spatial frequency component of the frame image surrounding the area is decreased. By doing so, it becomes possible to more effectively suppress the shielding of the object requiring attention by the frame image. In addition, the relationship between the appearance of the frame image and the object to be watched, that is, the relationship in which the area where the object to be watched exists is surrounded by the frame image using a band lower than the spatial frequency band necessary for the recognition. It can be kept constant, and the inconvenience that the frame image becomes more conspicuous than the object requiring attention can be more effectively suppressed when the object requiring attention is watched.

  In addition, when the display image generation unit 14 performs the low-pass filtering process on the frame image surrounding the area where the target object is present, the frame image is matched with the change in the distance from the own vehicle to the target object. It is desirable to change the cut-off frequency of the low-pass filtering process for.

  Even if there is no change in the size of the target object itself, if the distance from the own vehicle to the target object changes with time as the host vehicle travels, the target object in the captured image of the in-vehicle camera 1 is changed. The size of the existing area changes. Therefore, the cutoff frequency of the low-pass filtering process for the frame image surrounding the area where the target object exists is changed in accordance with the change in the distance from the host vehicle to the target object. By reducing the spatial frequency component of the frame image surrounding the area where the target object exists as the distance to the target area becomes smaller, it is more effective to shield the target object with the frame image as in the above example. Can be suppressed. In addition, the relationship between the appearance of the frame image and the object to be watched, that is, the relationship in which the area where the object to be watched exists is surrounded by the frame image using a band lower than the spatial frequency band necessary for the recognition. It can be kept constant, and the inconvenience that the frame image becomes more conspicuous than the object requiring attention can be more effectively suppressed when the object requiring attention is watched.

  In addition, when the display image generation unit 14 generates a display image that surrounds an area where a target object needs attention with a frame image subjected to low-pass filtering, the frame image blinks on the display image displayed on the display 2. It is desirable to generate the display image so that it is displayed.

  Flashing a low spatio-temporal frequency pattern, which is a frame image that has been subjected to low-pass filtering in spatiotemporal space, results in a gentle luminance modulation close to a sine wave rather than a rectangular wave luminance modulation in time. In addition to the improvement in the performance, the possibility of inducing inappropriate gaze movement, which is the negative effect, can be reduced. Therefore, by displaying on the display 2 a display image in which the frame image subjected to the low-pass filtering process is displayed blinking, attention should be paid while effectively avoiding inappropriate gaze movement guidance due to the high attractiveness of the frame image itself. The cognition of the object can be increased more effectively.

  By the way, as described above, as an example of emphasizing a specific region by controlling the spatio-temporal frequency composition of the display image, the region where the target object is present is surrounded by a frame image that has been subjected to low-pass filtering in space and time. In addition to the above, for example, the high-pass filtering process is performed on the area where the target object needs to be present in the captured image of the in-vehicle camera 1, and the other image areas are also processed. The method of emphasizing the region where the target object needs attention is also effective by applying a low-pass filtering process. In other words, in this example, the spatio-temporal frequency composition of the entire display image is controlled, and the attractiveness of the region itself is positively increased by performing a high-pass filtering process on the region where the target object needs to be present. In addition, the image area other than the area where the target object needs attention is subjected to a low-pass filtering process so that the image is blurred and inconspicuous, thereby improving the recognition of the target object. Yes.

  FIG. 5 shows that when the in-vehicle camera 1 captures an image of the rear side of the host vehicle and a two-wheeled vehicle that is going to pass the host vehicle from the right rear side of the host vehicle is detected as the target object T requiring attention. This is an example of a display image P when a high-pass filtering process is performed on a region S in which a target object T is present in a captured image and a low-pass filtering process is performed on other image regions. .

  As shown in the image example of FIG. 5, the display image generation unit 14 performs a high-pass filtering process on a region S where the target object T is present in the captured image of the in-vehicle camera 1 and requires attention. An image area other than the area S where the object T exists is subjected to a low-pass filtering process to generate a display image, so that the area S where the attention object T in the display image exists is relatively conspicuous The region S can be emphasized, and the recognition of the object T requiring attention can be enhanced.

  In this case, the display image generation unit 14 may control the cutoff frequency of the high-pass filtering process for the region S where the target object T is present to gradually increase toward the center of the region S. desirable. In this way, by controlling the cut-off frequency of the high-pass filtering process for the region S where the target object T is present, the spatio-temporal frequency composition of the entire image changes smoothly, and the display is less uncomfortable as an image. An image can be generated.

  In addition, the display image generation unit 14 relatively increases the luminance level of the region S where the target object T is present or the target object T exists together with the control of the spatio-temporal frequency composition as described above. The display area may be generated by performing image processing such as coloring the area S to be displayed with a display color different from that of the other areas. In this case, the recognition of the object T requiring attention is further increased. It becomes possible.

  As described above in detail with specific examples, according to the vehicle image display system of the present embodiment, in the image captured by the in-vehicle camera 1, for example, another vehicle approaching the host vehicle. The display image of the image processing unit 10 is generated when a display image highlighting a region where a target object to be watched is present and displayed on the display 2 when the driver of the vehicle performs safe driving. Since the generation unit 12 controls the spatio-temporal frequency composition of the display image so as to emphasize a region where the target object is present, the visibility of the target object is reduced due to the emphasis process itself being too conspicuous. In addition, it is possible to effectively increase the cognition of the target object of interest by suppressing the line-of-sight guidance to the emphasis processing itself and appropriately guiding the target object of interest.

  The vehicle image display system described above is an example of an application of the present invention, and the technical scope of the present invention is not intended to be limited to the contents described in the above embodiments. Absent. That is, the technical scope of the present invention is not limited to the specific technical matters disclosed in the above embodiments, but includes various modifications, changes, alternative techniques, and the like that can be easily derived from this disclosure.

  For example, in the vehicle image display system described above, the cautionary object detection unit 12 of the image processing unit 10 performs image processing on the captured image of the in-vehicle camera 1 to detect the cautionary object present around the host vehicle. However, when an object detection device such as a laser radar device is mounted on the own vehicle, the object detection device may be used to detect a target object around the own vehicle. Good. In this case, since various parameters such as the installation position and the installation angle of the in-vehicle camera 1 and the object detection device with respect to the host vehicle are known in advance, the emphasis region specifying unit 13 of the image processing unit 10 uses the geometry based on these installation parameters. It is only necessary to specify a region in which an object requiring attention detected by the object detection device is present in the image captured by the in-vehicle camera 1 by scientific calculation.

  Moreover, although embodiment described above is an example which applied this invention with respect to the image display system for vehicles which displays the image around the own vehicle imaged with the vehicle-mounted camera 1 on the display 2 in a vehicle interior, this invention is The present invention can be widely applied to various image display apparatuses that display an image by performing an emphasis process so that a specific region in the display image is emphasized. For example, the present invention is also effective in applications such as emphasizing specific numbers and characters on an image display device that displays a display image including information such as numbers and characters.

  FIG. 6 shows an example of emphasizing numbers by surrounding them with frame images. In FIG. 6, the numbers at the bottom are examples in which the present invention is applied to perform enhancement processing (examples in which enhancement processing is performed by surrounding with a frame image subjected to low-pass filtering processing). An example in which enhancement processing is performed by enclosing numerals with a frame image having a simple edge is shown as a comparative example.

  As shown in FIG. 6, when a number to be emphasized is surrounded by a frame image having edges, the visibility of the number to be emphasized is reduced by spatial frequency masking, but the frame image surrounding the number to be emphasized is reduced. By applying low-pass filtering to the blurred frame image without edges, spatial frequency masking by the frame image can be avoided, and the number is appropriately emphasized while suppressing a decrease in the visibility of the number by the frame image. Can be made.

It is a block diagram which shows the structure of the image display system for vehicles to which this invention is applied. This is a specific example of a display image that is emphasized by enclosing the area where the target object exists in a frame image that has been subjected to low-pass filtering in spatio-temporal space, and requires a bicycle traveling on the shoulder to the left front of the vehicle. It is an example of a display image at the time of being detected as an attention object. It is a specific example of a display image emphasized by surrounding a region where a target object exists with a frame image subjected to low-pass filtering in space and time, and an oncoming vehicle waiting for a right turn in front of the host vehicle is It is an example of a display image at the time of being detected as a target object requiring attention. This is a specific example of a display image that is emphasized by surrounding a region where a target object exists with a frame image that has been subjected to low-pass filtering in spatiotemporal space. It is an example of a display image when is detected as an object requiring attention. This is a specific example of a display image when a high-pass filtering process is performed on an area where an object of interest is present and a low-pass filtering process is performed on other image areas. It is an example of a display image when a two-wheeled vehicle is detected as an object requiring attention. It is a figure which shows the example which emphasizes a number by enclosing a number with the frame image which performed the low-pass filtering process spatiotemporally.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Car-mounted camera 2 Display 10 Image processing unit 11 Camera image input part 12 Critical object detection part 13 Highlight area specific | specification part 14 Display image generation part 15 Display image output part P Display image T Critical object F Frame image S Caution Area where the object exists

Claims (9)

An imaging means for capturing an image around the vehicle;
A cautionary object detection means for detecting a cautionary object present in the imaging range of the imaging means;
Area specifying means for specifying an area in which the object to be watched is present in an image picked up by the image pickup means;
Display image generating means for processing the image captured by the imaging means to generate a display image so that the area specified by the area specifying means is emphasized;
Display means for displaying the display image generated by the display image generation means,
The vehicle image display system characterized in that the display image generation unit emphasizes the region specified by the region specifying unit by controlling a spatio-temporal frequency composition of a display image to be generated.   2. The vehicle according to claim 1, wherein the display image generating unit emphasizes the region by surrounding the region specified by the region specifying unit with a frame image subjected to a low-pass filtering process in space and time. Image display system.   The vehicle image according to claim 2, wherein the display image generation unit controls a cut-off frequency of a low-pass filtering process for the frame image according to a size of the region specified by the region specifying unit. Display system.   The display image generating means changes a cut-off frequency of low-pass filtering processing for the frame image in accordance with a change in distance to the attention object detected by the attention object detection means. Item 4. The vehicle image display system according to Item 2 or 3.   5. The vehicular image display system according to claim 2, wherein the display image generation unit generates a display image in which the frame image subjected to the low-pass filtering process is displayed in a blinking manner. 6.   The display image generating means performs the high-pass filtering process on the area specified by the area specifying means and performs the low-pass filtering process on the other image areas, thereby specifying the area specified by the area specifying means. 2. The vehicle image display system according to claim 1, wherein the region is emphasized.   The said display image generation means controls the cutoff frequency of the high-pass filtering process with respect to the area | region specified by the said area | region specification means so that it may become high gradually as it goes to the center of the said area | region. Vehicle image display system. An image display device that displays an image so that a specific region in the display image is emphasized,
An image display device characterized by emphasizing the specific region by controlling a spatio-temporal frequency composition of the display image. A method for highlighting a specific area in a display image,
An image enhancement method, wherein the specific region is enhanced by controlling a spatio-temporal frequency composition of the display image.