JP2007156807A - Vehicle operation support device and operation support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To keep excellent visibility of a border line superimposed on a pick-up image and displayed. <P>SOLUTION: A pick-up image including a vehicle area where a part of a vehicle is imaged, and a peripheral area where a periphery of the vehicle is imaged is imaged with a camera 1, and a border line which indicates a boundary between the vehicle area and the peripheral area is superimposed and drawn to the pickup image by an image processor 2, and a processed image processed by the image processor 2 is displayed on a display unit 3. The image processor 2 decides a drawing mode of the border line which is superimposed and drawn on the boundary part of the vehicle area and the peripheral area in a photographed image of the camera 1 based on the pick-up image of the camera 1 to which the border line is superimposed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a vehicle driving support device and a driving support method that display an image around a vehicle and assist a driver's driving operation.

  Conventionally, an image pickup device such as a CCD camera mounted on a vehicle picks up an image around the vehicle, and the driver picks up the image picked up by the image pickup device on a display device such as a liquid crystal display installed in the passenger compartment. Technologies that support driving operations have been proposed. When displaying the situation around the vehicle as an image, it is important that the driver can appropriately recognize the position of the object or the like appearing in the displayed image with respect to the vehicle. From such a viewpoint, normally, an image of an area around the vehicle is captured so that a part of the vehicle is reflected by the imaging device, and an image including a part of the vehicle is displayed on the display device.

Further, when displaying an image including a part of the vehicle on the display device in this way, it is desirable that the boundary between the vehicle portion and the vehicle surrounding region is clear. From such a viewpoint, the image is captured by the imaging device. Attempts have also been made to draw a boundary line superimposed on the boundary between the vehicle portion and the vehicle surrounding area in the image and display the image on which the boundary line is superimposed on a display device (for example, see Patent Document 1).
JP 2000-227999 A

  However, in the technique described in Patent Document 1, the boundary line that overlaps the boundary between the vehicle portion and the vehicle surrounding area is drawn in a predetermined drawing form. Has a problem that the visibility of the boundary line deteriorates. That is, the environment around the vehicle changes variously, and the color and brightness of the vehicle surrounding area in the image captured by the imaging device are not constant. Also, depending on the lighting conditions around the vehicle, the color of the vehicle in the image captured by the imaging device may change, and the blue sky or clouds above the vehicle or the ceiling of the parking lot may be reflected in the vehicle. In some cases, the color and brightness of the vehicle in the image may change variously due to the effect of the occurrence of. For this reason, depending on the environment around the vehicle, the boundary line drawn superimposed on the image is assimilated with the vehicle and the surrounding area, and the visibility deteriorates, which is an obstacle to providing appropriate driving support. There was a problem.

  The present invention was devised in view of the conventional situation as described above, and it is possible to maintain good visibility of the boundary line displayed superimposed on the image regardless of the environment around the vehicle. It is an object of the present invention to provide a vehicle driving support device and a driving support method.

  In order to achieve the above-mentioned object, the present invention provides a drawing form of the boundary line when the boundary line is drawn on the boundary part between the vehicle area and the peripheral area in the image photographed by the photographing means. The determination is made based on the image on which the line is superimposed.

  According to the present invention, since the drawing form of the boundary line drawn superimposed on the image photographed by the photographing means is determined based on the image on which the boundary line is superimposed, the environment around the vehicle The boundary line can be drawn in an optimum drawing form corresponding to the above, and the visibility of the boundary line can be improved.

  Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a vehicle driving support apparatus to which the present invention is applied. As shown in FIG. 1, the vehicle driving support device according to the present embodiment is installed in a vehicle and captures an image of an area around the vehicle, and performs various image processing on the image captured by the camera 1. The image processing apparatus 2 to be applied and the display apparatus 3 that is installed in the vehicle cabin of the vehicle and displays the processed image output from the image processing apparatus 2 are configured.

  The camera 1 includes, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) type element as an image pickup device, picks up an image of a region around the vehicle, converts it into an electric signal, and converts it into an electric signal, and an image processor Output to 2. In the present embodiment, as illustrated in FIG. 2, the camera 1 is described as being attached to the rear portion of the vehicle 100 and capturing an image of the region at the rear of the vehicle 100. It may be arbitrarily set according to the position of the vehicle peripheral area that is desired to be displayed, and a plurality of cameras 1 may be installed in accordance with a plurality of areas.

  FIG. 3 is a diagram illustrating an example of an image of the rear region of the vehicle that is captured by the camera 1 and displayed on the display device 3. In the driving support device of the present embodiment, as shown in FIG. 3, the camera 1 includes a vehicle region R1 in which a part of the vehicle (in this example, a part of the rear part of the vehicle) is imaged, and a peripheral part of the vehicle (vehicle An image including the peripheral region R2 in which the rear external region) is captured is captured. In this way, by capturing an image including the vehicle region R1 and the peripheral region R2 with the camera 1 and displaying the image on the display device 3, the driver can determine which object in the peripheral region R2 is on the vehicle. Whether such a positional relationship is present can be determined using the vehicle region R1 included in the image as a guideline, which is very effective as an image for supporting driving operation.

  The image processing device 2 superimposes a boundary line L (see FIG. 3) on the boundary portion between the vehicle region R1 and the peripheral region R2 on the image of the rear region of the vehicle that is captured by the camera 1 and input as a video signal. The image is drawn and output to the display device 3 as a processed image. For example, by a video processing circuit constituted by various electric / electronic elements, a digital image processing circuit constituted by a semiconductor chip, etc. It is configured.

  Particularly, as shown in FIG. 1, the image processing apparatus 2 includes an input frame buffer 11 that temporarily stores an image input as a video signal from the camera 1 for each frame, and an input frame as shown in FIG. Based on the color information measured by the color information measuring unit 12, the color information measuring unit 12 that measures color information about the captured image of the camera 1 held in the buffer 11, and the vehicle region R 1 in the captured image of the camera 1 A drawing form determining unit 13 that determines a drawing form of a boundary line that indicates the boundary with the peripheral region R2, and a drawing superimposing unit 14 that superimposes the boundary line on the captured image of the camera 1 according to the drawing form determined by the drawing form determining unit 13. And an output frame buffer 15 for temporarily storing the processed image in which the boundary line is drawn by the drawing superimposing unit 14 for each frame. And outputs the processed image stored in § 15 to the display device 3.

  The display device 3 includes a display panel such as a liquid crystal display panel or an organic EL panel, for example, and displays the processed image output from the image processing device 2. The display device 3 is disposed at a position that is visible from the driver seated in the driver's seat in the vehicle cabin. As the display device 3, a dedicated vehicle driving support device may be used. For example, when a navigation device or the like is mounted on the vehicle, a display device provided in the navigation device or the like is used. Then, an image around the vehicle may be displayed on this display device when necessary according to the driver's operation.

  A driver of a vehicle equipped with the vehicle driving support device as described above grasps the distance and direction between the vehicle and an object existing around the vehicle while referring to an image displayed on the display device 3. At this time, under a situation where the colors of the vehicle region R1 and the peripheral region R2 are similar, it becomes difficult to distinguish the boundary between the vehicle region R1 and the peripheral region R2, and as a result, the vehicle region R1 exists around the vehicle. It is conceivable that a situation may occur in which the positional relationship with the object becomes difficult to grasp. In addition, since the focus and exposure adjustment of the camera 1 are generally optimized for imaging the peripheral area R2, the outline of a part of the vehicle imaged as the vehicle area R1 is likely to be blurred in an optimal exposure state. In some cases, the image is not captured. Such a situation also becomes a factor that makes it difficult to grasp the positional relationship between the vehicle and objects around the vehicle.

  Therefore, in the vehicle driving support device of the present embodiment, when the image processing device 2 superimposes and draws the boundary line L on the captured image captured by the camera 1, the image processing device 2 uses the camera 1. A drawing form of the boundary line L is determined based on the captured image, and the boundary line L is drawn in a superimposed manner according to the determined drawing form. Thereby, in the vehicle driving support device of the present embodiment, the boundary line L can be drawn in a drawing form with high visibility according to the captured image of the camera 1 under any external environment. As a result, the visibility of the boundary between the vehicle region R1 and the peripheral region R2 in the image displayed on the display device 3 can be improved.

  Here, an example of image processing executed by the image processing device 2 in the vehicle driving support device of the present embodiment will be described with reference to a flowchart shown in FIG.

  When the operation of the vehicle driving support device starts, an image of the vehicle peripheral region (vehicle rear region in the present embodiment) is captured by the camera 1 and input to the image processing device 2 as a video signal (step S1). When the captured image of the camera 1 is input as a video signal, the image processing device 2 temporarily stores the captured image in the input frame buffer 11 for each frame.

  Next, the color information measuring unit 12 measures the color information about the captured image of the camera 1 held in the input frame buffer 11 (step S2). At this time, the color information measuring unit 12 measures the color information for each of the vehicle region R1 and the peripheral region R2 illustrated in FIG. 2, and obtains color information A and color information B for each region.

  The acquisition method of the color information A and the color information B is arbitrary. For example, the captured image of the camera 1 held in the input frame buffer 11 is a 24-bit R (color data) for each pixel (pixel). In the case of an image format having red, G (green), and B (blue) values, the color information A and B is calculated by calculating the average value of the color data of all the pixels included in each of the regions R1 and R2. Can be obtained.

  In addition, a predetermined number of characteristic pixels in each of the regions R1 and R2 may be extracted in advance as a processing target, and color information A and B may be obtained by calculating an average value of color data for the extracted pixels. . Specifically, for example, for each of the regions R1 and R2, 16 pixels of 4 pixels in the vertical direction and 4 pixels in the horizontal direction are divided in advance as one unit, and the color of the upper left point for each unit is set as color information representing each unit. Then, the color information A and B for each of the regions R1 and R2 may be obtained by calculating the average of the color information for each unit. As a result, the number of pixels processed by the color information measuring unit 12 is 1/16 of that for all pixels, so even if the image processing capability of the color information measuring unit 12 is relatively low, The color information A and B can be calculated with time.

  Further, as color information acquisition processing in the color information measurement unit 12, for example, as shown in FIG. 5, a predetermined region R3 to be acquired for color information is set in the vicinity of the boundary line L to be drawn, and this region is set. Color information calculation processing may be performed only for the pixels in R3. Specifically, as shown in FIG. 5, in the region R3, a portion that overlaps the vehicle region R1 is a region R1 ', and a portion that overlaps the peripheral region R2 is a region R2'. Then, the color information A and B are obtained by calculating the average of the color information for only the pixels in the regions R1 'and R2'.

  As described above, since the number of pixels to be processed by the color information measuring unit 12 can be reduced by previously limiting the range in which the color information calculation processing is performed, the image information processing capability of the color information measuring unit 12 is compared. The color information A and B can be calculated in a short time even when the target is low. In addition, since only the pixels in the vicinity of the boundary line L are subject to color information calculation, the drawing form of the boundary line L can be determined more effectively, and the visibility of the boundary line L is more effective. Can be improved.

  Next, the drawing form determination unit 13 determines the drawing form of the boundary line L based on the color information A and B about the vehicle region R1 and the peripheral region R2 acquired as described above.

  Here, the distance CL between the color information A and the color information B is defined by Expression 1 shown below.

CL = | R1-R2 | + | G1-G2 | + | B1-B2 | (Formula 1)
Note that the color information A and B measured by the color information measuring unit 12 has color information expressed as 24-bit RGB data. In addition, the R, G, and B values of the color information A are R1, G1, and B1, respectively, and the R, G, and B values of the color information B are R2, G2, and B2, respectively. The drawing form determination unit 13 first calculates a difference (distance) CL between the color information A and the color information B based on the above formula 1 (step S3).

  Next, the drawing form determination unit 13 determines the drawing color of the boundary line L based on the calculated difference CL between the color information A and the color information B (step S4). At this time, as the drawing color of the boundary line L, by determining the color that is at a distance from the color information A that is the representative color of the vehicle region R1 and the color information B that is the representative color of the peripheral region R2, Since the visibility of the boundary line L can be improved, a color that is separated from both the color information A and the color information B by a predetermined distance or more is determined as the drawing color of the boundary line L.

  The drawing color of the boundary line L may be dynamically determined for each frame of the captured image of the camera 1 based on, for example, the above formula 1, but several drawing colors are prepared in advance. Alternatively, among the prepared drawing colors, a color separated from the color information A and the color information B by a reference distance or more may be selectively determined. Thereby, the amount of calculation required for determining the drawing color of the boundary line L can be significantly reduced.

  In this embodiment, both the color information A and the color information B are used when determining the drawing color of the boundary line L. However, the color information A that is the representative color of the vehicle area R1 and the peripheral area The drawing color of the boundary line L may be determined based on only one of the color information B that is the representative color of R2. Further, when both the color information A and B are used, for example, it is determined in advance which of the color information A and B is to be given priority and the drawing color of the boundary line L is determined. When it is impossible to select a color that is more than the reference distance with respect to both the color information B and the color information B, a color that is separated from the priority color information by the reference distance may be selected.

  When the distance between the color information A and B is far, it is presumed that the visibility of the vehicle area R1 and the peripheral area R2 is originally high. There is no need to make it stand out. In this case, the reference distance may be calculated based on an equation that is inversely proportional to the difference between the color information A and the color information B, and the drawing color of the boundary line L may be determined based on the calculated reference distance. . In this case, when the distance between the color information A and the color information B is large, the reference distance is small. When the distance between the color information A and the color information B is close, the reference distance is large. Thus, an appropriate drawing color can be determined according to the captured image input to the image processing apparatus 2.

  Furthermore, in the present embodiment, it is assumed that the boundary line L is always drawn. For example, when the distance between the color information A and B is sufficiently large, the drawing form determination unit 13 performs the boundary. It may be determined that drawing of the line L is unnecessary and it is determined not to draw the boundary line L.

  When the drawing color of the boundary line L is determined as described above, the drawing form determination unit 13 next determines the drawing density of the boundary line L based on the difference CL between the color information A and the color information B. (Step S5). Specifically, the drawing density of the boundary line L is such that the density is lower as the difference CL between the color information A and the color information B is larger and the density is higher as the difference CL between the color information A and the color information B is smaller. To decide. Next, the drawing form determination unit 52 determines the drawing thickness of the boundary line L based on the difference CL between the color information A and the color information B (step S6). Specifically, the drawing thickness of the boundary line L is such that the larger the difference CL between the color information A and the color information B, the thinner the line, and the smaller the difference CL between the color information A and the color information B, the thicker the line. To decide.

  In the processing in step S5 and step S6, as in the processing in step S4 described above, among the color information A that is the representative color of the vehicle region R1 and the color information B that is the representative color of the peripheral region R2, The drawing density and drawing thickness of the boundary line L may be determined based on only one of them.

  When the drawing form of the boundary line L is determined by the drawing form determining unit 13 as described above, the drawing superimposing unit 14 then takes a captured image of the camera 1 according to the drawing form determined by the drawing form determining unit 13. The boundary line L is superimposed and drawn on the boundary portion between the vehicle region R1 and the peripheral region R2 (step S7). Note that the drawing process in step S7 can use a general technique in the field of image processing, and thus a detailed description thereof will be omitted. If the drawing form determination unit 13 determines that the boundary line L is not drawn, the processing from step S5 to step S7 is omitted.

  The processed image in which the boundary line L is drawn by the drawing superimposing unit 14 is sent to the output frame buffer 15 and temporarily held in the output frame buffer 15 for each frame. Then, the processed image temporarily stored in the output frame buffer 15 is output to the display device 3 (step S8), whereby the image as illustrated in FIG.

  As described above in detail with reference to specific examples, in the vehicle driving support device of the present embodiment, based on the captured image captured by the camera 1, the vehicle region R1 and the peripheral region R2 in the captured image. Is determined in accordance with the imaging color of the vehicle itself and the imaging color of the surrounding area of the vehicle, which changes according to the external environment of the vehicle. The boundary line L can be drawn in a simple drawing form. Therefore, according to the vehicle driving support apparatus of the present embodiment, the vehicle periphery in which the boundary line L with good visibility is superimposed on the boundary portion between the vehicle region R1 and the peripheral region R2 under any external environment. These images can be displayed on the display device 3, and the driving operation by the driver can be appropriately supported.

  In the embodiment described above, the drawing color, drawing density, and drawing thickness are determined as the drawing form of the boundary line L, respectively, and the boundary line is determined based on the determined drawing color, drawing density, and drawing thickness. L is drawn, but at least one of drawing color, drawing density, and drawing thickness is dynamically determined based on the color information A and B, and the rest is a boundary line using a predetermined value set in advance. L may be drawn. Further, the drawing form is not limited to the drawing color, drawing density, or drawing thickness. For example, when the boundary line L is displayed as an animation, the animation form, the speed of blinking display, etc. May be determined based on the captured image as a drawing form.

It is a block diagram which shows schematic structure of the driving assistance device for vehicles to which this invention is applied. It is a schematic diagram explaining an example of the attachment position of a camera. It is a figure which shows an example of the vehicle back image imaged with a camera and displayed on a display apparatus. It is a flowchart which shows an example of the image process performed in an image processing apparatus. It is a figure explaining the variation of the image processing method in an image processing apparatus, and is a figure explaining the example which makes the process target the area | region of the boundary line vicinity in a captured image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera 2 Image processing apparatus 3 Display apparatus 11 Input frame buffer 12 Color information measurement part 13 Drawing form determination part 14 Drawing superimposition part 15 Output frame buffer 100 Vehicle R1 Vehicle area R2 Peripheral area L Boundary line

Claims (8)

In a vehicle driving assistance device that displays an image around a vehicle and assists the driving operation of the driver,
Imaging means for imaging an image including a vehicle area in which a part of the vehicle is imaged and a peripheral area in which a peripheral portion of the vehicle is imaged;
Image processing means for superimposing and drawing a boundary line indicating a boundary between the vehicle area and the peripheral area on the image captured by the imaging means, and outputting a processed image;
Display means for displaying the processed image output from the image processing means,
The vehicle processing support apparatus, wherein the image processing unit determines a drawing form of the boundary line based on an image captured by the imaging unit.   The said image processing means determines the drawing form of the said boundary line based on at least any one of the color information of the said vehicle area | region, and the color information of the said surrounding area | region. Vehicle driving support device.   The image processing means compares the color information of the vehicle area and the color information of the peripheral area, and determines whether or not to draw the boundary line according to the magnitude of the difference between the color information. The driving support apparatus for a vehicle according to claim 2, wherein the driving support apparatus is for a vehicle.   The image processing means compares the color information of the vehicle area and the color information of the peripheral area, and determines the drawing color of the boundary line according to the magnitude of the difference between the color information. The vehicle driving support device according to claim 2.   The image processing means compares the color information of the vehicle area and the color information of the peripheral area, and determines the drawing density of the boundary line according to the difference in the color information. The vehicle driving support device according to claim 2.   The image processing means compares the color information of the vehicle area and the color information of the peripheral area, and determines the drawing thickness of the boundary line according to the difference in the color information. The vehicle driving support device according to claim 2.   The image processing means, when determining the drawing form of the boundary line, determines the drawing form based on color information of the vicinity of the boundary line in at least one of the vehicle area and the peripheral area. The driving support apparatus for a vehicle according to claim 2, wherein the driving support apparatus is for a vehicle. In a driving support method for supporting a driver's driving operation by displaying an image around the vehicle,
An imaging step of capturing an image including a vehicle region in which a part of the vehicle is imaged and a peripheral region in which a peripheral portion of the vehicle is imaged;
A drawing form determining step for determining a drawing form of a boundary line indicating a boundary between the vehicle area and the peripheral area based on the image captured in the imaging step;
A drawing step of superimposing and drawing the boundary line in the drawing mode determined in the drawing mode determining step on the image captured in the image capturing step;
And a display step for displaying the image in which the boundary line is drawn in the drawing step.

Images