JP5217860B2 - Blind spot image display device and blind spot image display method - Google Patents

Description

  The present invention relates to a blind spot image display apparatus and a blind spot image display method for presenting a blind spot image obtained by photographing a blind spot of an own vehicle to an occupant with an imaging device provided in the vehicle.

Conventionally, as this type of technology, for example, those described in the following documents are known (see Patent Document 1). In the technique described in this document, when a blind spot image is displayed on a monitor by operating a switch or the like by an occupant at an intersection with poor visibility, etc., the link number of this place is displayed. After memorizing the travel direction and the distance to the intersection, the vehicle travels this place after the next time, the link number of this travel path, the travel direction and the distance to the intersection match the previous stored information, and the vehicle speed is If it is below the predetermined value, the blind monitor is automatically activated.
JP-A-10-274534

  The conventional blind monitor has a problem that it is troublesome and requires an operation by a passenger at the first place where the blind spot image is displayed on the monitor.

  Therefore, the present invention has been made in view of the above, and an object of the present invention is to provide a blind spot image display device and a blind spot image display method capable of displaying a blind spot image without requiring an occupant operation. There is to do.

In order to achieve the above object, at least one of the front left side and front right side images of the host vehicle imaged by the first imaging means installed on the front grille and a second side installed on the side mirror The degree of coincidence between a part of the imaging area imaged by the first imaging unit and at least the front side image that has been imaged by the imaging unit is determined, and as a result of the determination, the degree of coincidence of both images is preset If it is determined to be lower than the degrees that are is characterized by displaying a blind spot image captured by the first imaging hand stage on the display means.

  According to the present invention, since a blind spot image is automatically displayed at an intersection with poor visibility, the blind spot image can be displayed without requiring an operation by a passenger.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS The best embodiment for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a blind spot image display device according to a first embodiment of the present invention. The blind spot image display device of Example 1 shown in FIG. 1 processes and controls display of a plurality of cameras 11 (11a, 11b) that capture the surroundings of the host vehicle and image data obtained by imaging with these cameras 11. The image processing control device 12 includes a monitor 13 that displays an image obtained by the image processing control device 12.

  The camera 11a is installed at the center of the front grille of the host vehicle, and as shown by reference numeral A in FIG. 2, the front and left / right sides of the host vehicle 20 on which the camera 11a is installed are symmetrical with respect to the center of the front grille. Image. This imaging range includes a blind spot area that is a blind spot that is difficult for the driver on the left and right in front of the host vehicle 20 to view. Therefore, the blind spot image display device displays a captured image including an image of the blind spot area captured by the camera 11a.

  The camera 11b is installed on the left and right door mirrors of the host vehicle 20, and the camera 11b installed on the left door mirror images the left front side of the host vehicle 20 and is installed on the right door mirror, as indicated by reference numeral B in FIG. Although not shown in the drawing, the camera 11b captures the right front side of the host vehicle 20 that is symmetrical with the imaging region of the camera 11b installed on the left door mirror around the center line before and after the host vehicle 20. In the following description, the imaging area of the camera 11a and the camera 11b installed on the left door mirror will be described, but the same applies to the imaging area of the camera 11a and the camera 11b installed on the right door mirror.

  Here, the overlapping imaging area in which the imaging area A of the camera 11a and the imaging area B of the camera 11b installed in the left door mirror overlap refers to an area indicated by reference numeral C in FIG. The specific area set in advance to determine whether or not the line of sight at the intersection described below is good is set to a predetermined area in the overlapping imaging area C, and is an area (denoted by symbol D in FIG. 2). This is set as an area indicated by crossed lines.

  These cameras 11 a and 11 b are constituted by, for example, a CCD camera or a CMOS camera, and image data obtained by imaging is given to the image processing control device 12.

  The image processing control device 12 receives image data obtained by imaging by each camera 11, performs image processing on this image data, and displays the obtained image on the monitor 13. The image processing control device 12 performs edge detection processing for extracting the contour of an object in an image among image processing technologies conventionally known in the field of image processing technology on image data obtained by imaging by the camera 11a. Then, the presence or absence of a guardrail on the road is determined. That is, the image processing control device 12 extracts an edge using a threshold value that is applied when performing edge extraction on a preset region in the image obtained by the camera 11a, and extracts the extracted edge. It is determined whether or not the previously set specific region is divided into two, and if it is divided into two, it is determined that a guardrail is installed on the traveling road. When the guardrail is installed, a screen such as route guidance is displayed on the screen of the monitor 13 by the conventionally known navigation system 14 without starting the blind monitor.

  The image processing control device 12 compares the image data obtained by the imaging of the camera 11a and the image data obtained by the imaging of the camera 11b, and calculates the degree of coincidence between the two image data. That is, the image processing control device 12 specifies in advance an overlapped imaging area C in which the imaging area A of the camera 11a shown in FIG. 2 and the imaging area B of the camera 11b installed on the left door mirror overlap. A statistic such as a standard deviation of luminance in the region D is calculated for the specific region D corresponding to each of the cameras 11a and 11b, and the statistic corresponding to the camera 11a and the statistic corresponding to the camera 11b are compared. It is determined whether or not the difference is equal to or greater than a predetermined determination value of the degree of coincidence, and the degree of coincidence of statistics such as standard deviation in the image data of the specific area D of each camera 11a, 11b is determined based on the determination result, Whether to display the blind spot image captured by the camera 11a on the monitor 13 is selected based on the degree of coincidence.

  For example, when the host vehicle 20 approaches an intersection as shown in FIG. 2, as shown in FIG. 2A, an overlapping imaging area C in which the imaging area A of the camera 11 a and the imaging area B of the camera 11 b overlap. For example, when the difference between the standard deviations calculated for each of the cameras 11a and 11b with respect to the specific region D set within is equal to or less than a preset value, an image obtained by imaging with the camera 11a It is determined that both images of the captured image obtained by the camera 11b are substantially the same (the degree of coincidence is high), it is determined that the view of the intersection is good when viewed from the driver, and the blind spot image is monitored on the monitor 13. The screen display up to that time of the monitor 13 is continued without being displayed.

  On the other hand, when the difference in standard deviation of the specific area D corresponding to each of the cameras 11a and 11b is equal to or larger than a preset value as shown in FIG. It is determined that the image and the captured image obtained by the camera 11b do not match (the degree of matching is low), and it is determined that the line of sight of the intersection is not good when viewed from the driver. A blind spot image in the blind spot direction is displayed for the driver of the host vehicle 20 obtained by the camera 11a. When there is a building at the corner of the intersection, the camera 11a does not capture the building, so the far range is directly imaged. However, the camera 11b captures the building, so the camera 11b is far from the building. The range cannot be captured by the camera 11b. Accordingly, the two specific areas D are different images, and the standard deviations are also different, and therefore the degree of matching is low.

  As a method for determining the degree of coincidence between the image data of the camera 11a and the camera 11b, in addition to the above, for example, a pattern matching technique that has been generally widely known in the technical field of image processing is applied. You may make it discriminate | determine the coincidence degree of both image data.

  The image processing control device 12 is an integrated circuit for an existing specific application such as a microcomputer, an ASIC, or an FPGA that includes a CPU, a storage device, an input / output device, and the like necessary for a computer that controls various operation processes based on a program. It is comprised including.

  The monitor 13 is installed at a position in the vehicle where the screen is easy to see, and when the navigation system 14 is activated, the monitor 13 displays an image provided from the navigation system 14 such as route guidance and presents it to the driver. On the other hand, when the blind monitor is activated, the blind spot image captured by the camera 11c is displayed in preference to the image given from the navigation system 14 and presented to the driver. The monitor 13 is composed of an existing display device such as a CRT or LCD.

  Next, an operation procedure of the blind spot image display device according to the first embodiment will be described with reference to the flowchart of FIG.

  Referring to FIG. 3, first, an edge is extracted by applying an edge extraction method of image processing technology and an extraction threshold value to a specific area D set in the imaging area A of the camera 11a (Step S1). S301). Thereafter, it is determined whether or not the extracted edge divides the previous specific region D into two (step S302). As a result of the discrimination, when the extracted edge divides the area into two, it is assumed that a guardrail exists in the imaging area, and is given from, for example, the navigation system 14 without displaying the blind spot image on the monitor 13. The contents that have been displayed so far are continuously displayed on the screen of the monitor 13 (step S303).

  On the other hand, if the extracted edge does not divide the area into two in the determination result, the standard deviation (Laσ) of the luminance in the image data of the specific area D of the image data of the camera 11a is subsequently calculated ( Step S304). Similarly, the luminance standard deviation (Lbσ) in the image data of the specific area D in the image data of the camera 11b is calculated (step S305).

  Next, it is determined whether or not the absolute value of the difference between the two standard deviations is greater than a preset determination value (α) for the degree of coincidence (step S306). If the result of determination is not large, it is determined that the two image data in the specific area D are substantially coincident and the degree of coincidence is high, and it is presumed that the driver has a good view of an intersection or the like. The screen display (for example, navigation screen) until then on the monitor 13 is continued without displaying the image (step S307).

  On the other hand, if the result of the determination is large, it is assumed that the driver has a poor view of the intersection or the like (the driver needs to display the blind spot image), and the screen display that has been displayed so far is displayed. Instead, a blind spot image obtained by imaging with the camera 11a is displayed on the monitor 13 (step S308).

  Whether or not the blind spot image is displayed on the monitor 13 may include a requirement for information on whether the vehicle speed of the host vehicle or whether the host vehicle is traveling near an intersection as shown in FIG. . That is, if the vehicle speed used in the navigation system 14 is equal to or lower than a preset speed (requirement 1), a blind spot image is displayed on the monitor 13, or in addition to the requirement 1, obtained from the map information provided in the navigation system 14. When the position of the subject vehicle is near the intersection (requirement 2), a blind spot image is displayed on the monitor 13, or when the requirement of the first embodiment is satisfied in addition to the requirement 1 and requirement 2, the monitor 13 is displayed. You may make it display a blind spot image.

  As described above, in the first embodiment, it is determined in advance that the driver is required to display the blind spot image by determining that the prospect of the intersection or the like is not good, and based on the determination result. Since the blind spot image is displayed on the monitor 13, the blind spot image can be accurately displayed on the monitor 13 without performing the activation operation by the occupant. Thereby, the display operation by the occupant and the registration work of registering in advance the place where the blind spot image is displayed on the monitor 13 become unnecessary, and the troublesomeness can be reduced.

  In addition, the display of the blind spot image on the monitor 13 at a place where the display of the originally blind spot image is unnecessary, such as an intersection with good visibility, is suppressed. This makes it possible to display and present information on the monitor 13 that is more necessary for the driver than displaying a blind spot image on the monitor 13 at the present time, such as route guidance given from the navigation system 14. Can be matched with the intention of the person.

  Next, a second embodiment of the present invention will be described. The feature of the second embodiment is that the degree of coincidence of the image data is calculated by the standard deviation of brightness and the pattern matching employed in the previous first embodiment using the overhead image. This is the same as in the first embodiment.

  In realizing the above characteristic technique, the image processing control device 12 generates an overhead image corresponding to each of the two images based on the image data acquired by the cameras 11a and 11b. This overhead image uses, for example, a conventionally known viewpoint conversion technique. This viewpoint conversion image processing technique is employed in, for example, a vehicle image display system described in Japanese Patent Application Laid-Open No. 2000-33901, and an actual image obtained by a camera that captures a predetermined area around the vehicle is displayed. This is a technique for converting into a predetermined virtual viewpoint, for example, a virtual image viewed from above the vehicle. Specifically, for example, a conversion address (mapping table) indicating a correspondence relationship between the position coordinates of the output pixel in the virtual image after the viewpoint conversion and the position coordinates of the input pixel in the actual image obtained by imaging with the camera is prepared in advance. Then, by changing the pixel position of the input pixel according to this mapping table and generating an output image, for example, real time conversion from a virtual viewpoint set above the vehicle to a bird's-eye view image is realized. .

  In addition, since the orientation of the overhead view image of both cameras differs because the installation positions of the camera 11a and the camera 11b are different, the image processing control device 12 makes the orientation of one overhead view image coincide with the orientation of the other overhead view image. After rotating one overhead image, both overhead images are compared by pattern matching.

  For example, as shown in FIG. 5A, when the area indicated by reference numeral 50 is set to the specific area D described in the first embodiment, for example, in the images taken by the cameras 11a and 11b, the image of this area is displayed. Is a distorted image as shown in FIG. 5A, and the degree of distortion differs between the two cameras. Therefore, if the degree of coincidence is determined using such an image, the determination accuracy may be lowered.

  On the other hand, as shown in FIG. 5B, by converting both real images of the cameras 11a and 11b into a bird's-eye view image using the viewpoint conversion technique, the shapes of the both bird's-eye view images match. .

  As described above, the second embodiment has the same effect as the first embodiment, and the degree of coincidence is determined using the overhead-converted image. The effect that the accuracy of determination once can be increased is obtained.

  Next, Embodiment 3 of the present invention will be described. FIG. 6 is a diagram illustrating the configuration of the blind spot image display device according to the third embodiment.

  In the third embodiment, the camera 11c is added to the configuration of the first embodiment or the second embodiment described above, and the image processing apparatus 12 uses both the image data of the camera 11a and the camera 11b described in the first embodiment. A function of displaying a blind spot image on the monitor 13 is added when priority is given to the determination result of the degree of coincidence and the driver's line-of-sight movement is detected based on an image obtained by imaging with the camera 11c.

  The camera 11c is installed at a position where the driver's face can be imaged, and functions as a detection unit that images the driver's face so that the movement of the driver's line of sight can be detected and detects the driver's line of sight movement. Image data obtained by imaging by the camera 11c is given to the image processing control device 12.

  The image processing control device 12 analyzes the image data given from the camera 11c and detects the movement of the driver's line of sight. That is, for example, as shown in FIG. 7 (a), the driver 72 moves the line of sight in the direction of the left and right A pillars 71 so that the driver can confirm the left and right safety when the vehicle approaches an intersection. When the movement of the line of sight is detected, the driver 72 then turns the line of sight in the direction in which the monitor 13 is installed so that the driver 72 can view the screen of the monitor 13 as shown in FIG. If it is determined whether or not the movement of the line of sight is detected, the blind spot image is displayed on the monitor 13 in preference to the determination result of the degree of coincidence between the image data of the camera 11a and the camera 11b.

  Next, an operation procedure in the third embodiment will be described with reference to the flowchart of FIG. In FIG. 8, first, based on the position of the host vehicle and the map data provided in the navigation system 14, it is determined whether or not the current position of the host vehicle is near an intersection (step S801). If it is determined that the host vehicle is traveling near the intersection, the speed of the host vehicle obtained from the function of the navigation system 14 is sufficient to display a blind spot image on the preset vehicle speed, for example, the monitor 13. It is determined whether or not the speed is low (step S802). As a result of the determination, if the vehicle speed is a set low speed, it is subsequently determined whether or not the driver's line of sight has moved in the direction of the left and right A pillars (step S803). If the driver's line of sight has moved in the direction of the A pillar as a result of the determination, it is then determined whether or not the driver's line of sight has moved to the screen of the monitor 13 (step S804). If the driver's line of sight moves to the screen of the monitor 13 as a result of the determination, a blind spot image is displayed on the monitor 13 (step S805). On the other hand, as described above, the third embodiment has the same effect as the first embodiment and also displays a blind spot image on the monitor 13 when it is detected that the driver has moved his / her line of sight to the left and right. Since it is made to display, it becomes possible to display a blind spot image exactly.

  Correspondence between the configuration requirements of the above-described embodiments and the configuration requirements in the claims is as follows. That is, the camera 11a constitutes a first imaging means, the camera 11b constitutes a second imaging means, the image processing device 12 constitutes a discrimination means and a display control means, and the monitor 13 constitutes a display means.

It is a figure which shows the structure of the blind spot image display apparatus which concerns on Example 1 of this invention. It is a figure which shows the relationship between the imaging area of cameras 11a, 11b, and the coincidence degree of an image. It is a flowchart which shows the procedure of the operation | movement process which concerns on Example 1 of this invention. It is a figure which shows the display requirement of a blind spot image. It is a figure which shows the bird's-eye view image conversion example of the image which concerns on Example 2 of this invention. It is a figure which shows the structure of the blind spot image display apparatus which concerns on Example 3 of this invention. It is a figure which shows the mode of a driver | operator's eyes | visual_axis movement which concerns on Example 3 of this invention. It is a flowchart which shows the procedure of the operation | movement process which concerns on Example 3 of this invention.

Explanation of symbols

11a to 11d ... Camera 12 ... Image processing control device 13 ... Monitor 14 ... Navigation system

Claims (5)

In a blind spot image display device that displays on a display means a blind spot image obtained by imaging the blind spot of the host vehicle.
A first imaging means installed on a front grill for imaging at least one of a front left side and a front right side of the host vehicle;
A second imaging unit that is installed on the door mirror and that images a front side that overlaps at least a part of an imaging region captured by the first imaging unit;
The image of the first imaging unit and the second imaging unit in an overlapping imaging region where the imaging region imaged by the first imaging unit and the imaging region imaged by the second imaging unit overlap. A discriminating means for discriminating the degree of coincidence with the image;
Display control means for displaying a blind spot image captured by the first imaging means on the display means when the discrimination means determines that the matching degree of both images is lower than a preset matching degree; A blind spot image display device comprising: The blind spot image display device according to claim 1, wherein the determination unit determines the degree of coincidence of the bird's-eye images obtained by bird's-eye conversion of the two images. A detecting means for detecting the movement of the line of sight of the driver of the own vehicle;
The display control means displays a blind spot image on the display means in preference to a determination result of the determination means when a movement of the driver's line of sight is detected by the detection means. 3. A blind spot image display device according to 1 or 2. The discriminating means calculates a standard deviation of luminance in an image of a specific area preset in the overlapping imaging area for each of the specific area of the first imaging means and the specific area of the second imaging means. 4. When the difference between the standard deviations of both luminances is larger than a preset discrimination value, it is judged that the degree of coincidence between the two images is low. The blind spot image display device described in 1. In a blind spot image display method for displaying on a display means a blind spot image obtained by imaging a blind spot of the host vehicle,
At least one image of the front left side and the front right side of the host vehicle imaged by the first imaging unit installed on the front grille and the second imaging unit installed on the side mirror, Determining the degree of coincidence between at least a part of the imaging region imaged by the first imaging means and the front side image overlapping ,
Result of the determination, when the degree of matching between the images is determined to be lower than the degrees that is set in advance, displaying a blind spot image captured by the first imaging hands stage the said display means A blind spot image display method as a feature.

Images