JP2018097541A - Driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driving support device capable of enabling a driver to recognize a low-visibility information display body with low visibility, by a virtual image that is displayed on a windscreen, and preventing a high-visibility information display body from unnecessarily becoming a virtual image.SOLUTION: A driving support device comprises: imaging means 4 which images a front side of a vehicle and acquires a vehicle front image; storage means 2 which stores map information; extraction means 11 for extracting a low-visibility information display body with low visibility, at the front side of the vehicle from the vehicle front image; class information acquisition means 13 which acquires a class of the low-visibility information display body from the map information; and display means by which a virtual image representing the class is overlapped with the low-visibility information display body and displayed on a windscreen.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a driving support device that performs driving support by displaying a virtual image representing a sign or the like on a windshield in a superimposed manner on the sign or the like.

In order to make it easier for the driver to recognize traffic lights, signs, etc., various support devices have been proposed in which a display showing the traffic lights, signs, etc. is displayed on the windshield so as to overlap the actual traffic lights, signs, etc. ( For example, Patent Documents 1 and 2).
Hereinafter, the reference numerals used in Patent Documents 1 and 2 are shown in parentheses for reference, and the techniques disclosed in Patent Documents 1 and 2 will be described.
Patent Document 1 states that “When a driver uses a mobile phone in a hands-free manner, the presence of a traffic light (60A) in front of the host vehicle is detected from map information stored in the navigation system (10). In addition, a driving vehicle apparatus that projects the display virtual image (90A) on the windshield (70) so as to substantially overlap the traffic light (60A) is disclosed.
Patent Document 2 states that “when a vehicle front image captured by the imaging device (115) is analyzed and it is determined that the field of view is poor, a predetermined type of road marking or road is determined from the vehicle front image. There is disclosed a “driving support device” that extracts a sign and projects the extracted road sign or road sign on a windshield at a position overlapping the actual road sign or road sign.

JP 2000-331289 A JP 2008-280026 A

However, in the driving vehicle device disclosed in Patent Document 1, the indication indicating the sign or the like is projected on the windshield so as to overlap the actual sign or the like. However, the displayed sign or the like is shifted from the actual sign or the like. May be projected.
That is, in the map information of the navigation system (10), there is a limit to the type and accuracy of information, information such as the size and height of the sign cannot be acquired, or the accuracy of the acquired position information of the sign is low. Therefore, based on such limited information, it is difficult to project a display indicating a sign or the like on the windshield so as to overlap the actual sign or the like.
If the displayed sign or the like is shifted from the actual sign or the like, the judgment of the driver may be affected.

  Further, the driving support device disclosed in Patent Document 2 extracts a road sign of a type set in advance by image analysis from a captured vehicle front image. Although it is desirable that road signs with low visibility (for example, dirty road signs, faint road signs, road signs with poor installation orientation) be displayed as images that are easy for the driver to recognize, Since the driving support device 2 extracts road signs and the like by image analysis, for example, even if it can be recognized that it is a road sign, it may not be possible to extract road signs of low visibility up to the type of road sign. There is.

  The present invention was devised in view of the above problems, and allows a driver to recognize a low-visibility information display body with poor visibility by a virtual image displayed on the windshield, and has good visibility. An object of the present invention is to provide a driving support device that can prevent an information display body from being virtualized unnecessarily.

  (1) In order to achieve the above object, the driving support apparatus according to the present invention includes an imaging unit that captures an image of the front of the vehicle and acquires a vehicle forward image, a storage unit that stores map information, and the vehicle forward image. An extraction means for extracting a low visibility information display body with poor visibility in front of the vehicle, a type information acquisition means for acquiring the type of the low visibility information display body from the map information, and the type It is characterized by comprising display means for displaying a virtual image on the windshield superimposed on the low visibility information display.

  (2) The extraction means analyzes the vehicle front image, and the low visibility information display body having a similarity with a preset information display body template image of a first threshold value or more and a second threshold value or less. It is preferable to have a first extraction function for extracting as

  (3) It further includes gaze detection means for detecting the gaze of the driver, the extraction means has a second extraction function, and the second extraction function analyzes the vehicle front image and stores the information display body in advance. A low visibility information display object candidate having a similarity with a template image equal to or higher than a first threshold is used as the candidate for the low visibility information display body, and the low eye movement is detected based on static eye movement detected by the eye detection means. It is preferable to select the visibility information display body.

  (4) The second extraction function may be configured to select the candidate ahead of the line of sight when a change in the direction of the line of sight detected by the line of sight detection unit continues for a predetermined time or more. It is preferable to select as the low visibility information display.

  (5) When the line of sight detected by the line-of-sight detection means is directed to the candidate body more than a predetermined number of times, the second extraction function displays the candidate body as the low-visibility information display body. It is preferable to sort as

According to the present invention, the low visibility information display body is extracted by the extraction means from the vehicle forward image captured by the vehicle forward imaging device, and the type information of the low visibility information display body is stored in the map information of the storage means. And a virtual image representing the type is superimposed on the low visibility information display body and displayed on the windshield.
Therefore, the low visibility information display body with poor visibility can be recognized by the driver by the virtual image displayed on the windshield in correspondence with the actual installation position. Furthermore, what is displayed as a virtual image is limited to the low-visibility information display body, and it can be prevented that the information display body with good visibility is unnecessarily virtualized.

It is a block diagram which shows the structure of the driving assistance apparatus of one Embodiment of this invention. It is a schematic diagram which shows the mode of assistance by the driving assistance apparatus of one Embodiment of this invention. It is a flowchart which shows an example of a control flow by the driving assistance device of one Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. Note that the embodiment described below is merely an example, and there is no intention to exclude various modifications and technical applications that are not explicitly described in the following embodiment. The configurations of the following embodiments can be implemented with various modifications without departing from the spirit thereof.

  A driving support apparatus 100 as one embodiment of the present invention will be described with reference to FIGS.

[1. Configuration of driving support device]

  As shown in FIG. 1, the driving support apparatus 100 includes a host vehicle position detector 1, a map information storage device (storage means) 2, a vehicle front photographing device (imaging means) 4, and a viewpoint related information detection device (line of sight). Detection means) 5, a head-up display 6, and an ECU 10 to which these devices are connected.

  In the present embodiment, the host vehicle position detector 1 includes a geomagnetic sensor, a gyroscope, a distance sensor that detects the travel distance of the host vehicle, and a sensor such as a GPS receiver for a global positioning system (GPS). ing. Each of these sensors has an error of a different nature or cannot acquire information depending on the situation, and is configured to be used while complementing each other.

The map information storage device 2 stores various types of map information such as the type of information display body in addition to the road shape. The ECU 10 refers to various map information in the map information storage device 2 as appropriate.
Here, the information display body provides appropriate information and instructions necessary for the driver as exemplified by road signs and traffic lights. The information related to the information display body is the position information and the type information of the information display body. The type information is a specific information such as “one-way” or “no overtaking” if the information display body is a road sign. If the information display body is a traffic light, it is information to that effect (indicating that it is a traffic signal).

  The vehicle front photographing device 4 is a stereo camera, and is provided at the front end of the host vehicle or the upper part of the vehicle interior. The vehicle front image capturing device 4 images a predetermined range in front of the vehicle in accordance with an instruction from the ECU 10 and outputs data of the captured image (hereinafter referred to as “vehicle front image”) to the ECU 10.

The viewpoint related information detection device 5 includes a stereo camera different from that constituting the vehicle front photographing device 4 and a processing device that analyzes an image captured by the stereo camera. The stereo camera of the viewpoint related information detection device 5 is provided at a position where the driver's face can be photographed when the driver is facing forward, for example, on the side of the rearview mirror, and includes the driver's face. Output the image data to the processor. The processing device analyzes the supplied image data by a known recognition processing method, and determines the position (eye point) of the driver's eyes and the direction of the line of sight.
Note that the processing device that analyzes the image may be a functional element of the ECU 10.

  The head-up display 6 is a device that projects an image on the windshield of the host vehicle, and the head-up display 6 of the present embodiment can display over almost the entire range of the windshield.

  The ECU 10 is a normal computer and includes a well-known CPU, ROM, RAM, I / O, and a bus line for connecting these components. In the ROM, a program to be executed by the ECU 10 is written, and a CPU or the like executes predetermined arithmetic processing according to the program.

  The ECU 10 includes an extraction unit (extraction unit) 11, an information display body position calculation unit 12, a type information acquisition unit (type information acquisition unit) 13, a display position setting unit 14, and a display control unit 15 as functional elements.

The extraction unit 11 selectively extracts a low visibility information display body. The low visibility information display body is an information display body with low visibility, that is, it is difficult for the driver to visually recognize, for example, a dirty road sign, a faint road sign, a bad road sign, snow or It is a road sign whose visibility has decreased due to rain or lack of illumination. The extraction unit 11 selects and extracts a low visibility information display body, and does not extract an information display body with good visibility.
The extraction unit 11 includes a first extraction function 11A and a second extraction function 11B.

The first extraction function 11A analyzes the vehicle front image and performs matching with a template image (information display body template image) of a plurality of types of information display bodies stored in advance. Extract the display.
Specifically, the first extraction function 11A quantifies the similarity R (the degree of similarity to the template image in the pattern matching process) of the image area to be analyzed (hereinafter referred to as “target area”) in the vehicle front image. Is obtained while sequentially shifting the target area, and this is performed for each of the template images. The first extraction function 11A has a similarity R with one of the template images equal to or higher than the first threshold R1 and equal to or lower than the second threshold R2 (R1 ≦ R ≦ R2, R1 <R2). Is extracted as a low-visibility information display body, and position information of the low-visibility information display body in the vehicle front image is output to the display position setting unit 14.

The second extraction function 11B first analyzes the vehicle front image, performs matching with the plurality of types of template images, and extracts candidates for the low visibility information display body from the vehicle front image.
Specifically, the second extraction function 11B obtains the similarity R of the target area in the vehicle front image while sequentially shifting the target area, and performs this for each of the template images. The target region whose similarity R is equal to or higher than the first threshold value R1 is set as a candidate.

The second extraction function 11B further selects a low visibility information display body from the candidates based on the static movement of the line of sight detected by the viewpoint related information detection device 5.
Specifically, the second extraction function 11 </ b> B extracts candidates that are ahead of the line of sight as low-visibility information display bodies when they correspond to the next case 1 or the case 2, respectively.
Case 1: When the driver's line-of-sight change Δe continues to be equal to or smaller than a predetermined minute amount, that is, when the time T for staring at the candidate exceeds the predetermined time T1, the driver selects the type. Since it is hard to see, it is guessed that the candidate is staring, so the candidate ahead of the line of sight is extracted as a low visibility information display.
Case 2: The number of times the driver's line of sight is directed to the candidate, that is, the number N of reviewing the candidate exceeds the predetermined number N1 (the driver's line of sight is directed to the candidate beyond the predetermined number N1. )), It is assumed that the driver has reviewed the road sign many times because it is difficult for the driver to visually recognize the type. Therefore, the candidate ahead of the line of sight is extracted as the low visibility information display body.

  The predetermined time T1 and the predetermined number of times N1 are constants in the present embodiment, but as the vehicle speed v detected by the vehicle speed sensor increases, the predetermined time T1 may be shortened or the predetermined number of times N1 may be decreased. May be. This is because as the vehicle speed v increases, the period from when the driver recognizes the candidate until the host vehicle passes the candidate, that is, the period during which the driver can try to recognize the candidate is shortened. Even if it is difficult for the person to visually recognize the information display body, the time for staring at the candidate body is shortened, and the number of times of reviewing the candidate body is reduced, so this effect is eliminated.

In the present embodiment, the extraction unit 11 includes both the first extraction function 11A and the second extraction function 11B. However, the extraction unit 11 includes only one of the first extraction function 11A and the second extraction function 11B. You may make it provide. When the second extraction function 11B is provided, the second extraction function 11B may be configured to extract only one case of the case 1 and the case 2 as a low visibility information display body.
In addition, the function for obtaining the similarity R may be shared between the first extraction function 11A and the second extraction function 11B.

  The information display body position calculation unit 12 calculates a representative position of the actual low visibility information display body with the host vehicle as a reference point. The vehicle front photographing device 4 is configured by a stereo camera as described above, and the information display body position calculation unit 12 has the same low-visibility information display body in the vehicle front images respectively captured by these two stereo cameras. The actual position of the low-visibility information display body with respect to the own vehicle as a reference point is calculated from the difference in position (parallax information) from which is extracted.

The type information acquisition unit 13 acquires the type information of the low visibility information display body from the map information in the map information storage device 2.
Specifically, the type information acquisition unit 13 is based on the position of the host vehicle detected by the host vehicle position detector 1 and the host vehicle of the low visibility information display body calculated by the information display body position calculation unit 12. The absolute position of the low visibility information display body is estimated from the position as a point. Then, the type information acquisition unit 13 in the map information of the map information storage device 2 is an information display body within a certain distance from the estimated position of the low visibility information display body (in the case where there are a plurality of information display bodies, the information display nearest to the information display body). The body) is the low visibility information display body, the type information of the information display body is acquired from the map information as the type information of the low visibility information display body.

As shown in FIG. 2, the display position setting unit 14 is arranged on the windshield 30 so that the virtual image 20 ′ of the low visibility information display body overlaps the real object 20 of the low visibility information display body as viewed from the driver 50. The display position P ′ of the virtual image 20 ′ is set.
Specifically, the display position setting unit 14 uses the position P of the actual object 20 of the low visibility information display body calculated by the information display body position calculation unit 12 and the viewpoint P0 detected by the viewpoint related information detection device 5. A point where the connecting straight line L intersects the windshield 30 is set as the display position P ′. Reference numeral 25 in FIG. 2 is an information display body 25 that has good visibility and can be recognized by the driver 50 with his / her own vision. The information display body 25 with good visibility is a virtual image on the windshield 30. Is not displayed.

  The display control unit 15 acquires the type information of the low visibility information display body from the type information acquisition unit 13 and the display position P ′ from the display position setting unit 14. Then, the display control unit 15 controls the head-up display 6 so that a virtual image 20 ′ set in advance according to the type of the low visibility information display body is displayed at the display position P ′ on the windshield 30. The head-up display 6 is controlled.

  The head-up display 6, the display position setting unit 14, and the display control unit 15 constitute display means of the present invention.

[1-2. Control flow]
Hereinafter, with reference to FIG. 3, the control flow of the driving assistance apparatus as one embodiment of the present invention will be described. This control flow is repeatedly executed at a predetermined cycle.
First, in step S10, a vehicle front image is captured by the vehicle front photographing device 4, and in step S20, the extraction unit 11 extracts a low visibility information display body from the vehicle front image using template matching. If the low visibility information display body is extracted, the process proceeds to step S30. If the low visibility information display body is not extracted, the process returns. In S20, a plurality of low visibility information display bodies may be extracted.
In step S30, the type of the low visibility information display body extracted in step S20 is determined by the type information acquisition unit 13.
In step S40, the display position setting unit 14 sets the display position P ′ based on the position P of the actual low visibility information display 20 and the viewpoint P0 of the driver 50. In step S50, the display control unit 15 Under the control, the virtual image 20 ′ of the low visibility information display body is projected from the head-up display 6 to the display position P ′, and the process returns.
When a plurality of low-visibility information display bodies are extracted in step S20, a type is acquired for each low-visibility information display body in steps S30, S40, and S50, a display position is set, and a virtual image is displayed. Is displayed.

[1-3. Action / Effect]
Referring to FIG. 2, according to the driving support device as one embodiment of the present invention, the low visibility information display body is extracted by the extraction unit 11 from the vehicle front image captured by the vehicle front image capturing device 4. At the same time, the type information of the low-visibility information display body is acquired from the map information in the map information storage device 2, and a virtual image 20 'representing the low-visibility information display body of this type is obtained from the low-visibility information display body. It is superimposed on the actual object 20 and displayed on the windshield 30.
Therefore, the low visibility information display body with poor visibility can be recognized by the driver by the virtual image 20 ′ displayed on the windshield 30 in correspondence with the actual installation position P. Furthermore, what is displayed as a virtual image is limited to the low-visibility information display body, and it is possible to prevent unnecessary display of the virtual image up to the information display body 25 with good visibility.

  In the second extraction function 11B, since the low visibility information display body is selected based on the movement of the driver's line of sight, the low visibility information display body can be extracted based on the actual behavior of the driver. it can.

2 Map information storage device (storage means)
4 Vehicle front photographing device (imaging means)
5 Viewpoint related information detection device (line of sight detection means)
6 Head-up display 10 ECU
DESCRIPTION OF SYMBOLS 11 Extraction part 11A 1st extraction function 11B 2nd extraction function 12 Information display body position calculation part 13 Type information acquisition part (type information acquisition means)
DESCRIPTION OF SYMBOLS 14 Display position setting part 15 Display control part 20 Real thing of low visibility information display body 20 'Virtual image of low visibility information display body 25 Real thing of information display body with good visibility 50 Driver | operator 100 Driving assistance device P Low visibility information The position of the display object 20 P 'The position of the virtual image 20' of the low visibility information display body P0 The viewpoint of the driver 50

Claims (5)

Imaging means for capturing the vehicle front and acquiring the vehicle front image;
Storage means for storing map information;
Extraction means for extracting a low visibility information display body with poor visibility in front of the vehicle from the vehicle front image;
Type information acquisition means for acquiring the type of the low visibility information display body from the map information;
A driving support device, comprising: a display unit configured to display a virtual image representing the type on a windshield in a superimposed manner on the low visibility information display body. The extraction means comprises:
A first extraction function that analyzes the vehicle front image and extracts, as the low-visibility information display body, a similarity with a preset information display body template image that is greater than or equal to a first threshold value and less than or equal to a second threshold value; The driving support apparatus according to claim 1, further comprising: It further comprises line-of-sight detection means for detecting the driver's line of sight,
The extraction means has a second extraction function;
The second extraction function is:
The vehicle front image is analyzed, and the similarity with the information display body template image stored in advance is set as a candidate for the low visibility information display body with the first threshold value or more, and the line of sight is selected from the candidates. The driving support apparatus according to claim 1 or 2, wherein the low visibility information display body is selected based on static movement of the line of sight detected by the detecting means. The second extraction function is configured to detect the candidate object at the tip of the line of sight when the change in the direction of the line of sight detected by the line-of-sight detection unit continues for a predetermined time or longer. The driving support device according to claim 3, wherein the driving support device is selected as a sex information display body. The second extraction function selects the candidate as the low-visibility information display when the line of sight detected by the line-of-sight detection unit is directed to the candidate more than a predetermined number of times. The driving support apparatus according to claim 3 or 4, wherein

Images