WO2020183652A1 - Driving assistance device - Google Patents

Abstract

A driving assistance device (100) is provided with: a road traffic information detection unit (21) which detects, in a forward image (I1), a region (A) corresponding to a road traffic information sign (S), and also detects the content of the road traffic information sign (S); a viewpoint detection unit (22) which uses a vehicle interior image (I2) to detect, in the forward image (I1), a position (P) corresponding to the viewpoint of a driver; a viewpoint distance coefficient calculation unit (23) which calculates a viewpoint distance coefficient (C) corresponding to the distance (L) of the region (A) with respect to the position (P); a viewpoint distance coefficient correction unit (24) which corrects the viewpoint distance coefficient (C) on the basis of the content of the road traffic information sign (S); and an awareness level calculation unit (25) which calculates the driver's level of awareness (D2) of the road traffic information sign (S) by integrating corrected viewpoint distance coefficients (C').

Description

Driving support device

The present invention relates to a driving support device.

Patent Document 1 discloses a technique for calculating the visibility of a safety confirmation object by the driver based on the distance of the safety confirmation object with respect to the driver's line-of-sight direction. As a result, not only the visibility of the safety confirmation object being watched by the driver is calculated, but also the visibility of the safety confirmation object included in the so-called "peripheral vision" is calculated.

International Publication No. 2008/029802

Conventionally, signs, display boards, signboards, etc. (hereinafter collectively referred to as "road information display") showing various information about roads have been installed in various places. In addition, signs, display boards, signboards, etc. (hereinafter collectively referred to as "traffic information display") showing various information related to traffic are also installed in various places. Hereinafter, the road information display and the traffic information display are collectively referred to as "road traffic information display".

As described above, the technique described in Patent Document 1 calculates the visibility of the safety confirmation object by the driver. By diverting the technology described in Patent Document 1, the degree of recognition of the road traffic information display by the driver instead of calculating the visibility of the safety confirmation object by the driver (hereinafter referred to as "recognition degree"). It is conceivable to calculate (hereinafter referred to as "diversification technology").

Here, the degree of recognition of the road traffic information display by the driver is not only different depending on the distance of the road traffic information display with respect to the line-of-sight direction, but also different depending on the content of the road traffic information display. .. Therefore, when calculating the degree of recognition of the road traffic information display by the driver, it is preferable to consider not only the distance of the road traffic information display with respect to the line-of-sight direction but also the content of the road traffic information display. is there.

The technique described in Patent Document 1 considers the distance of the safety confirmation object with respect to the line-of-sight direction when calculating the visibility of the safety confirmation object by the driver, but does not consider the content of the safety confirmation object. .. Therefore, the diversion technology does not consider the content of the road traffic information display when calculating the recognition degree of the road traffic information display by the driver. As a result, there is a problem that the accuracy of determining whether or not the road traffic information display is recognized by the driver is low.

The present invention has been made to solve the above problems, and an object of the present invention is to improve the accuracy of determining whether or not the road traffic information display has been recognized by the driver.

The driving support device of the present invention uses a road traffic information detection unit that detects an area corresponding to the road traffic information display in the front image and detects the content of the road traffic information display, and a vehicle interior image to drive in the front image. The viewpoint distance coefficient is corrected based on the viewpoint detection unit that detects the position corresponding to the viewpoint of the person, the viewpoint distance coefficient calculation unit that calculates the viewpoint distance coefficient according to the distance of the area to the position, and the content of the road traffic information display. It is provided with a viewpoint distance coefficient correction unit for calculating a viewpoint distance coefficient, and a recognition degree calculation unit for calculating the recognition degree of the road traffic information display by the driver by integrating the corrected viewpoint distance coefficient.

According to the present invention, since it is configured as described above, it is possible to improve the accuracy of determining whether or not the road traffic information display is recognized by the driver.

It is a block diagram which shows the main part of the driving support system including the driving support device which concerns on Embodiment 1. FIG. It is explanatory drawing which shows the example of the front image. It is explanatory drawing which shows the area corresponding to the road traffic information display in the front image shown in FIG. 2A. It is explanatory drawing which shows the position corresponding to the viewpoint of the driver in the front image shown in FIG. 2A. It is explanatory drawing which shows the distance of the said region with respect to the said position in the front image shown in FIG. 2A. It is explanatory drawing which shows the other example of the front image. It is explanatory drawing which shows the area corresponding to the road traffic information display in the front image shown in FIG. 3A. It is explanatory drawing which shows the position corresponding to the viewpoint of the driver in the front image shown in FIG. 3A. It is explanatory drawing which shows the distance of the said region with respect to the said position in the front image shown in FIG. 3A. It is explanatory drawing which shows the hardware configuration of the driving support apparatus which concerns on Embodiment 1. FIG. It is explanatory drawing which shows the other hardware configuration of the driving support apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows the operation of the driving support apparatus which concerns on Embodiment 1. FIG. It is a flowchart which shows other operation of the driving support apparatus which concerns on Embodiment 1. FIG. It is explanatory drawing which shows the other example of the front image. It is explanatory drawing which shows the area corresponding to the road traffic information display in the front image shown in FIG. 7A. It is explanatory drawing which shows the position corresponding to the viewpoint of the driver in the front image shown in FIG. 7A. It is explanatory drawing which shows the distance of the said region with respect to the said position in the front image shown in FIG. 7A. It is a block diagram which shows the main part of the driving support system including the other driving support device which concerns on Embodiment 1. FIG. It is a block diagram which shows the main part of the driving support system including the driving support device which concerns on Embodiment 2. It is a flowchart which shows the operation of the driving support apparatus which concerns on Embodiment 2. It is a block diagram which shows the main part of the driving support system including the other driving support device which concerns on Embodiment 2. It is a block diagram which shows the main part of the driving support system including the other driving support device which concerns on Embodiment 2. It is a block diagram which shows the main part of the driving support system including the driving support device which concerns on Embodiment 3. It is a flowchart which shows the operation of the driving support apparatus which concerns on Embodiment 3. It is a block diagram which shows the main part of the driving support system including the driving support device which concerns on Embodiment 4. It is a block diagram which shows the main part of the driving support system including the other driving support device which concerns on Embodiment 4.

Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described with reference to the accompanying drawings.

Embodiment 1.
FIG. 1 is a block diagram showing a main part of a driving support system including a driving support device according to the first embodiment. The operation support device 100 of the first embodiment will be described with reference to FIG. Further, the driving support system 200 including the driving support device 100 will be described.

As shown in FIG. 1, the vehicle 1 is provided with a control device 2, a first image pickup device 3, a second image pickup device 4, and a display device 5. The main part of the driving support system 200 is composed of the control device 2, the first image pickup device 3, the second image pickup device 4, and the display device 5.

The first image pickup device 3 takes an image of the front of the vehicle 1 at predetermined time intervals, and outputs an image signal indicating the captured image (hereinafter referred to as "front image") I1. The first imaging device 3 is composed of, for example, an infrared camera or a visible light camera. The first imaging device 3 is provided, for example, on the front end of the vehicle 1, on the dashboard of the vehicle 1, or on the ceiling of the front of the vehicle interior of the vehicle 1.

The second imaging device 4 captures the interior of the vehicle 1 at predetermined time intervals and outputs an image signal indicating the captured image (hereinafter referred to as "in-vehicle image") I2. The second imaging device 4 is composed of, for example, an infrared camera or a visible light camera. The second imaging device 4 is provided, for example, on the dashboard of the vehicle 1.

The display device 5 is composed of, for example, a liquid crystal display or an organic EL (Electro Luminescence) display, and is provided on the dashboard of the vehicle 1. Alternatively, for example, the display device 5 is configured by a HUD (Head-Up Display).

The control device 2 is composed of, for example, an ECU (Electronic Control Unit). The driving support device 100 is provided in the control device 2. Hereinafter, the driving support device 100 will be described.

The road traffic information detection unit 21 acquires the image signal output by the first imaging device 3. The road traffic information detection unit 21 detects the road traffic information display S included in the front image I1 by executing an image recognition process for the front image I1 using the acquired image signal. More specifically, the road traffic information detection unit 21 detects the area A corresponding to the road traffic information display S in the front image I1 and also detects the content of the road traffic information display S. Hereinafter, the process in which the road traffic information detection unit 21 detects the contents of the area A and the road traffic information display S is referred to as “road traffic information detection process”.

The viewpoint detection unit 22 acquires the image signal output by the second imaging device 4. The viewpoint detection unit 22 uses the acquired image signal to perform image recognition processing on the in-vehicle image I2, thereby performing an image recognition process on the vehicle interior image I2, thereby corresponding to the driver's viewpoint in the front image I1 (hereinafter referred to as “viewpoint position”). It detects P. Hereinafter, the process in which the viewpoint detection unit 22 detects the viewpoint position P is referred to as “viewpoint detection process”.

For example, the viewpoint detection unit 22 detects the position of the driver's head (more specifically, the position of the driver's eyes) by executing the image recognition process for the in-vehicle image I2, and also detects the driver's line of sight. Detect the vector. The viewpoint detection unit 22 stores in advance a table showing the correspondence between the position of the driver's eyes, the driver's line-of-sight vector, and the viewpoint position P in the front image I1. The viewpoint detection unit 22 detects the viewpoint position P in the front image I1 by using the stored table.

The viewpoint distance coefficient calculation unit 23 acquires the detection result by the road traffic information detection unit 21 and the detection result by the viewpoint detection unit 22. The viewpoint distance coefficient calculation unit 23 calculates the distance L of the region A with respect to the viewpoint position P by using the acquired detection result. Further, the viewpoint distance coefficient calculation unit 23 calculates a coefficient (hereinafter referred to as “viewpoint distance coefficient”) C corresponding to the calculated distance L. That is, the viewpoint distance coefficient C is a value that gradually decreases as the distance L increases. In other words, the viewpoint distance coefficient C is a value that gradually increases as the distance L decreases.

For example, the viewpoint distance coefficient calculation unit 23 has a coordinate value indicating the viewpoint position P in the front image I1 (hereinafter referred to as “first coordinate value”) and a predetermined portion (for example, the central portion or the upper left) of the region A in the front image I1. The distance L in pixel units is calculated based on the difference value from the coordinate value (hereinafter referred to as "second coordinate value") indicating the position of the end portion). The viewpoint distance coefficient calculation unit 23 calculates the viewpoint distance coefficient C corresponding to the calculated distance L in pixel units.

Alternatively, for example, the viewpoint distance coefficient calculation unit 23 calculates the distance L in pixel units based on the difference value between the first coordinate value and the second coordinate value. The viewpoint distance coefficient calculation unit 23 stores in advance a table showing the correspondence between the first coordinate value, the second coordinate value, and the coefficient for converting the distance L in pixel units into the distance L in meters. There is. The viewpoint distance coefficient calculation unit 23 calculates the distance L in meters by converting the calculated distance L in pixels into the distance L in meters using the stored table. The viewpoint distance coefficient calculation unit 23 calculates the viewpoint distance coefficient C corresponding to the calculated distance L in meters.

FIG. 2A shows an example of the front image I1. The front image I1 shown in FIG. 2A includes a road sign (more specifically, a regulation sign) having the meaning of “no vehicle intrusion”, that is, a road traffic information display S. The regulation sign is composed of one symbol (hereinafter referred to as "mark") and does not include a character string.

In this case, the road traffic information detection unit 21 detects the area A corresponding to the road traffic information display S (see FIG. 2B). Further, the road traffic information detection unit 21 detects the content of the road traffic information display S. More specifically, the road traffic information detection unit 21 detects that the road traffic information display S is a regulatory sign having the meaning of "no vehicle entry". At this time, it is also detected that the road traffic information display S is composed of one mark and that the road traffic information display S does not include a character string. Next, the viewpoint detection unit 22 detects the viewpoint position P in the front image I1 (see FIG. 2C). Next, the viewpoint distance coefficient calculation unit 23 calculates the distance L of the region A with respect to the viewpoint position P (see FIG. 2D), and calculates the viewpoint distance coefficient C corresponding to the calculated distance L.

FIG. 3A shows another example of the front image I1. The front image I1 shown in FIG. 3A includes a signboard including the character string “future / closed”, that is, the road traffic information display S.

In this case, the road traffic information detection unit 21 detects the area A corresponding to the road traffic information display S (see FIG. 3B). Further, the road traffic information detection unit 21 detects the content of the road traffic information display S. More specifically, the road traffic information detection unit 21 detects that the road traffic information display S is a signboard including the character string "future / closed". Next, the viewpoint detection unit 22 detects the viewpoint position P in the front image I1 (see FIG. 3C). Next, the viewpoint distance coefficient calculation unit 23 calculates the distance L of the region A with respect to the viewpoint position P (see FIG. 3D), and calculates the viewpoint distance coefficient C corresponding to the calculated distance L.

The viewpoint distance coefficient correction unit 24 acquires the viewpoint distance coefficient C calculated by the viewpoint distance coefficient calculation unit 23. Further, the viewpoint distance coefficient correction unit 24 acquires the detection result by the road traffic information detection unit 21. The viewpoint distance coefficient correction unit 24 corrects the acquired viewpoint distance coefficient C based on the content of the road traffic information display S by using the acquired detection result.

That is, the viewpoint distance coefficient correction unit 24 is based on the number of marks included in the road traffic information display S, the presence or absence of a character string in the road traffic information display S, the number of characters in the character string included in the road traffic information display S, and the like. The difficulty level D1 of recognition of the road traffic information display S is calculated. The difficulty level D1 is also an index indicating the complexity of the road traffic information display S.

For example, the road traffic information display S in the front image I1 shown in FIG. 2 is composed of one mark and does not include a character string. On the other hand, the road traffic information display S in the front image I1 shown in FIG. 3 includes a character string. In this case, the difficulty level D1 of the former (FIG. 2) is calculated to be smaller than the difficulty level D1 of the latter (FIG. 3). In other words, the difficulty level D1 of the latter (FIG. 3) is calculated to be larger than the difficulty level D1 of the former (FIG. 2).

The viewpoint distance coefficient correction unit 24 corrects the viewpoint distance coefficient C so that when the difficulty level D1 is large, the value becomes smaller than when the difficulty level D1 is small. In other words, the viewpoint distance coefficient correction unit 24 corrects the viewpoint distance coefficient C so that when the difficulty level D1 is small, the value becomes larger than when the difficulty level D1 is large.

As described above, the forward image I1 is imaged at predetermined time intervals. On the other hand, the road traffic information detection unit 21 executes the road traffic information detection process at predetermined time intervals. Therefore, when the road traffic information display S is included in the front image I1, the viewpoint distance coefficient C is calculated at a predetermined time interval, and the calculated viewpoint distance coefficient C is sequentially corrected. The recognition degree calculation unit 25 calculates the recognition degree D2 of the road traffic information display S by the driver by temporally integrating the corrected viewpoint distance coefficient C'. That is, the recognition degree D2 is updated every time the viewpoint distance coefficient C is calculated and the calculated viewpoint distance coefficient C is corrected.

The recognition degree determination unit 26 acquires the detection result by the road traffic information detection unit 21. In FIG. 1, the connection line between the road traffic information detection unit 21 and the recognition degree determination unit 26 is not shown. Normally, the road traffic information display S in the front image I1 goes out of the front image I1 due to the traveling of the vehicle 1 (that is, due to the passage of time during traveling). The recognition degree determination unit 26 determines whether or not the road traffic information display S has gone out of the front image I1 by using the acquired detection result.

The recognition degree determination unit 26 acquires the recognition degree D2 calculated by the recognition degree calculation unit 25 when it is determined that the road traffic information display S has gone out of the front image I1. The recognition degree determination unit 26 determines whether or not the road traffic information display S is recognized by the driver by determining whether or not the acquired recognition degree D2 is equal to or higher than a predetermined threshold value Dth. is there. That is, the threshold value Dth is set to a value that can determine whether or not the road traffic information display S is recognized by the driver.

When the recognition degree D2 is determined by the recognition degree determination unit 26 to be less than the threshold value Dth, the display control unit 27 controls the display device 5 to display the image I3 corresponding to the road traffic information display S (hereinafter, "first"). It is called "display control"). By visually recognizing the image I3, the driver can confirm the road traffic information display S that he / she missed.

The driving support device 100 is required by the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27. The part is composed.

Next, the hardware configuration of the main part of the driving support device 100 will be described with reference to FIG.

As shown in FIG. 4A, the driving support device 100 has a processor 51 and a memory 52. The memory 52 has functions of a road traffic information detection unit 21, a viewpoint detection unit 22, a viewpoint distance coefficient calculation unit 23, a viewpoint distance coefficient correction unit 24, a recognition degree calculation unit 25, a recognition degree determination unit 26, and a display control unit 27. The program to realize it is stored. When the processor 51 reads out and executes the stored program, the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, and recognition The functions of the coefficient determination unit 26 and the display control unit 27 are realized.

Alternatively, as shown in FIG. 4B, the driving support device 100 has a processing circuit 53. In this case, the functions of the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27 are dedicated. It is realized by the processing circuit 53.

Alternatively, the driving support device 100 has a processor 51, a memory 52, and a processing circuit 53 (not shown). In this case, among the functions of the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27. Some functions are realized by the processor 51 and the memory 52, and the remaining functions are realized by the dedicated processing circuit 53.

The processor 51 uses, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, and a DSP (Digital Signal Processor).

The memory 52 is composed of a non-volatile memory or a non-volatile memory and a volatile memory. The volatile memory in the memory 52 is, for example, one using RAM (Random Access Memory). The non-volatile memory among the memory 52 is, for example, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Memory) or a EEPROM (Electrically Erasable Memory) MemoryRead At least one of (Hard Disc Drive) is used.

The processing circuit 53 is composed of a digital circuit or a digital circuit and an analog circuit. The processing circuit 53 includes, for example, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), an FPGA (Field-Programmable Gate Array), a System-System (System) System, and an ASIC (System) System. At least one of them is used.

Next, with reference to the flowchart of FIG. 5, regarding the operation of the driving support device 100, the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, and the recognition degree calculation unit The operation of 25 will be mainly described.

It is assumed that the power supply for the driving support system 200 (for example, the accessory power supply or the ignition power supply in the vehicle 1) is turned on. The first image pickup apparatus 3 continuously executes a process of capturing an image of the front of the vehicle 1 at predetermined time intervals and outputting an image signal indicating the front image I1. The second imaging device 4 continuously executes a process of capturing an image of the interior of the vehicle 1 at predetermined time intervals and outputting an image signal indicating the in-vehicle image I2.

First, in step ST1, the road traffic information detection unit 21 executes the road traffic information detection process. When the road traffic information display S is included in the forward image I1 (step ST2 “YES”), the viewpoint detection unit 22 executes the viewpoint detection process in step ST3. Next, in step ST4, the viewpoint distance coefficient calculation unit 23 calculates the viewpoint distance coefficient C. Next, in step ST5, the viewpoint distance coefficient correction unit 24 corrects the viewpoint distance coefficient C. Next, in step ST6, the recognition degree calculation unit 25 calculates the recognition degree D2.

When the road traffic information display S is not included in the front image I1 (step ST2 “NO”), the process of the driving support device 100 returns to step ST1. Further, following step ST6, the processing of the driving support device 100 returns to step ST1. That is, as described above, the road traffic information detection process is executed at predetermined time intervals.

Next, with reference to the flowchart of FIG. 6, the operation of the driving support device 100 will be described focusing on the operations of the recognition degree determination unit 26 and the display control unit 27.

As described above, the recognition degree determination unit 26 determines whether or not the road traffic information display S has gone out of the front image I1 by using the detection result by the road traffic information detection unit 21. When it is determined that the road traffic information display S has gone out of the front image I1, the recognition degree determination unit 26 executes the process of step ST11.

First, in step ST11, the recognition degree determination unit 26 acquires the recognition degree D2 calculated by the recognition degree calculation unit 25, and determines whether or not the acquired recognition degree D2 is equal to or greater than the threshold value Dth. .. When it is determined that the recognition degree D2 is less than the threshold value Dth (step ST11 “NO”), the display control unit 27 executes the first display control in step ST12. On the other hand, when it is determined that the recognition degree D2 is equal to or higher than the threshold value Dth (step ST11 “YES”), the process of step ST12 is skipped.

As described above, the driving support device 100 includes the viewpoint distance coefficient correction unit 24 that corrects the viewpoint distance coefficient C based on the content of the road traffic information display S. By using the corrected viewpoint distance coefficient C'for the calculation of the recognition degree D2, the viewpoint distance coefficient C before the correction is used for the calculation of the recognition degree D2 (that is, compared with the conventional driving support device). The accuracy of determining whether or not the road traffic information display S is recognized by the driver can be improved.

Next, a modified example of the driving support device 100 will be described with reference to FIG. 7.

When the front image I1 includes a plurality of road traffic information displays S, the driving support device 100 may execute the processes of ST4 to ST6 for each of the plurality of road traffic information displays S. (See FIG. 7). Further, in this case, the driving support device 100 may execute the processes of steps ST11 and ST12 for each of the plurality of road traffic information display S.

Next, another modification of the driving support device 100 will be described with reference to FIG.

As shown in FIG. 8, the recognition degree determination unit 26 and the display control unit 27 may be provided outside the driving support device 100. That is, the main part of the driving support device 100 is composed of the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, and the recognition degree calculation unit 25. You may.

As described above, the driving support device 100 of the first embodiment detects the region A corresponding to the road traffic information display S in the front image I1 and also detects the content of the road traffic information display S. Using 21 and the in-vehicle image I2, the viewpoint detection unit 22 that detects the position P corresponding to the driver's viewpoint in the front image I1 and the viewpoint distance coefficient C corresponding to the distance L of the region A with respect to the position P are calculated. By integrating the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24 that corrects the viewpoint distance coefficient C based on the contents of the road traffic information display S, and the corrected viewpoint distance coefficient C'by the driver. A recognition degree calculation unit 25 for calculating the recognition degree D2 of the road traffic information display S is provided. By using the corrected viewpoint distance coefficient C'for calculating the recognition degree D2, it is possible to improve the accuracy of determining whether or not the road traffic information display S is recognized by the driver.

Further, the driving support device 100 determines whether or not the recognition degree D2 is equal to or higher than the threshold value Dth, thereby determining whether or not the road traffic information display S is recognized by the driver, and the recognition degree determination unit 26. When it is determined that the recognition degree D2 is less than the threshold value Dth, the display control unit 27 that executes the control (first display control) to display the image I3 corresponding to the road traffic information display S on the display device 5 is provided. .. As a result, the driver can confirm the road traffic information display S that he / she missed.

Further, the recognition degree determination unit 26 determines whether or not the recognition degree D2 is equal to or greater than the threshold value Dth when the road traffic information display S goes out of the front image I1. As a result, when the vehicle 1 passes the installation position of the road traffic information display S, it is possible to determine whether or not the recognition degree D2 is equal to or higher than the threshold value Dth. That is, it is possible to determine whether or not the recognition degree D2 is equal to or higher than the threshold value Dth for the road traffic information display S that the driver may have missed.

Embodiment 2.
FIG. 9 is a block diagram showing a main part of the driving support system including the driving support device according to the second embodiment. The driving support device 100a of the second embodiment will be described with reference to FIG. Moreover, the driving support system 200a including the driving support device 100a will be described. In FIG. 9, the same blocks as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 9, the control device 2a is provided in the vehicle 1. The control device 2a is composed of, for example, an ECU.

Also, the vehicle 1 is provided with sensors 6. The sensors 6 include various sensors. For example, the sensors 6 include a wheel speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a wiper sensor.

Further, the vehicle 1 is provided with a wireless communication device 7. The wireless communication device 7 is composed of a transmitter and a receiver for wireless communication. The wireless communication device 7 is free to communicate with the server device 8 outside the vehicle 1.

Further, the storage device 9 is provided in the vehicle 1. The storage device 9 is composed of a non-volatile memory. The storage device 9 may be integrally configured with the memory (that is, the memory 52) of the control device 2a.

Further, the vehicle 1 is provided with an operation input device 10. The operation input device 10 is composed of, for example, at least one of a touch panel, a hardware key, and a microphone for voice input. The operation input device 10 is provided on the dashboard of the vehicle 1, for example. The operation input device 10 may be provided integrally with the display device 5, or may be provided adjacent to the display device 5.

The main part of the driving support system 200a is composed of the control device 2a, the first imaging device 3, the second imaging device 4, the display device 5, the sensors 6, the wireless communication device 7, the storage device 9, and the operation input device 10. There is. The driving support device 100a is provided in the control device 2a.

The vehicle information acquisition unit 31 acquires information indicating the running state of the vehicle 1 (hereinafter referred to as "vehicle information"). The vehicle information includes, for example, information indicating the traveling speed of the vehicle 1 and information indicating the turning amount of the vehicle 1.

For example, the vehicle information acquisition unit 31 calculates the traveling speed of the vehicle 1 by using the output signal from the wheel speed sensor of the sensors 6. As a result, information indicating the traveling speed of the vehicle 1 is acquired.

Further, for example, the vehicle information acquisition unit 31 calculates the turning amount of the vehicle 1 by using the output signal from the acceleration sensor, the gyro sensor, or the steering sensor of the sensors 6. As a result, information indicating the turning amount of the vehicle 1 is acquired.

The environmental information acquisition unit 32 acquires information indicating the environment outside the vehicle 1 (hereinafter referred to as "environmental information"). The environmental information includes, for example, information indicating the brightness around the vehicle 1 and information indicating the weather around the vehicle 1.

For example, the environmental information acquisition unit 32 detects the brightness around the vehicle 1 by executing an image recognition process for the front image I1. As a result, information indicating the brightness of the surroundings of the vehicle 1 is acquired.

Further, for example, the environmental information acquisition unit 32 determines the weather around the vehicle 1 by executing the image recognition process for the front image I1. As a result, information indicating the weather around the vehicle 1 is acquired.

Alternatively, for example, the environmental information acquisition unit 32 determines whether or not the wiper of the vehicle 1 is operating by using the output signal from the wiper sensor of the sensors 6. When the wiper of the vehicle 1 is operating, the environmental information acquisition unit 32 determines that the weather around the vehicle 1 is rain or snow. Further, the environmental information acquisition unit 32 determines that the weather around the vehicle 1 is sunny or cloudy when the wiper of the vehicle 1 is stopped. As a result, information indicating the weather around the vehicle 1 is acquired.

Alternatively, for example, the server device 8 stores information indicating the weather in each area. By communicating with the server device 8, the wireless communication device 7 receives information indicating the weather in the area including the current position of the vehicle 1. The environment information acquisition unit 32 acquires the received information. As a result, information indicating the weather around the vehicle 1 is acquired.

The driver information acquisition unit 33 acquires information about the driver of the vehicle 1 (hereinafter referred to as "driver information"). The driver information includes, for example, information indicating the driver's eyesight.

For example, vehicle 1 is used by a plurality of users. The storage device 9 stores a database in which information indicating the visual acuity of each user and information indicating the face image of each user are associated with each other. The information indicating the visual acuity of each user is, for example, input in advance by each user using the operation input device 10. The facial image of each user is, for example, pre-imaged by the second imaging device 4.

The driver information acquisition unit 33 acquires the face image of the driver of the vehicle 1 by executing the image recognition process for the in-vehicle image I2. The driver information acquisition unit 33 determines which of the plurality of users the driver of the vehicle 1 is by matching the acquired face image with the face image in the database. The driver information acquisition unit 33 acquires information indicating the user's visual acuity corresponding to the result of the determination from the database. As a result, information indicating the eyesight of the driver of the vehicle 1 is acquired.

Alternatively, for example, the storage device 9 stores the following trained model. That is, a trained model is stored in which a value indicating the driver's visual acuity is output when a value indicating the result of the image recognition process for the in-vehicle image I2 is input. The trained model is pre-generated by machine learning. The driver information acquisition unit 33 executes an image recognition process for the in-vehicle image I2, and inputs a value indicating the result of the image recognition process into the trained model. As a result, information indicating the driver's eyesight is acquired.

Hereinafter, vehicle information, environmental information, and driver information are collectively referred to as "correction information". The vehicle information acquisition unit 31, the environment information acquisition unit 32, and the driver information acquisition unit 33 constitute the correction information acquisition unit 34.

The viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the content of the road traffic information display S. That is, the viewpoint distance coefficient correction unit 24a executes the same correction processing as the viewpoint distance coefficient correction unit 24. In addition to this, the viewpoint distance coefficient correction unit 24a executes at least one of the following correction processes (1) to (6).

(1) Correction processing based on the size of the area A The viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the size of the area A is large, the value becomes larger than when the size of the area A is small. To do. In other words, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the size of the area A is small, the value is smaller than when the size of the area A is large. The detection result by the road traffic information detection unit 21 is used for such correction processing.

Normally, when the size of the area A is large, the size of the road traffic information display S is large, or the distance of the road traffic information display S to the vehicle 1 is small. In such a case, it is unlikely that the driver will miss the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a larger value.

(2) Correction processing based on the traveling speed of the vehicle 1 The viewpoint distance coefficient correction unit 24a has a viewpoint distance coefficient such that when the traveling speed of the vehicle 1 is high, the value is smaller than when the traveling speed of the vehicle 1 is low. Correct C. In other words, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the traveling speed of the vehicle 1 is low, the value becomes larger than when the traveling speed of the vehicle 1 is high. Vehicle information acquired by the vehicle information acquisition unit 31 is used for such correction processing.

Normally, when the traveling speed of the vehicle 1 is high, the visible time of the road traffic information display S by the driver is short. In such a case, it is highly likely that the driver misses the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a smaller value.

(3) Correction processing based on the turning amount of the vehicle 1 The viewpoint distance coefficient correction unit 24a has a viewpoint distance coefficient so that when the turning amount of the vehicle 1 is large, the value becomes smaller than when the turning amount of the vehicle 1 is small. Correct C. In other words, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the turning amount of the vehicle 1 is large, the value becomes smaller than when the turning amount of the vehicle 1 is small. Vehicle information acquired by the vehicle information acquisition unit 31 is used for such correction processing.

Normally, when the amount of turning of the vehicle 1 when turning left is large and the road traffic information display S is arranged on the right side of the vehicle 1, the visible time of the road traffic information display S by the driver is short. Similarly, when the amount of turning of the vehicle 1 when turning right is large and the road traffic information display S is arranged on the left side of the vehicle 1, the visible time of the road traffic information display S by the driver is short. In such a case, it is highly likely that the driver misses the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a smaller value.

(4) Correction processing based on the brightness of the surroundings of the vehicle 1 The viewpoint distance coefficient correction unit 24a has a viewpoint distance coefficient so that when the surroundings of the vehicle 1 are dark, the value is smaller than when the surroundings of the vehicle 1 are bright. Correct C. In other words, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the surroundings of the vehicle 1 are bright, the value becomes larger than when the surroundings of the vehicle 1 are dark. The environmental information acquired by the environmental information acquisition unit 32 is used for the correction process.

Normally, when the surroundings of the vehicle 1 are dark, the visibility of the road traffic information display S by the driver is reduced. In such a case, it is highly likely that the driver misses the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a smaller value.

(5) Correction processing based on the weather around the vehicle 1 The viewpoint distance coefficient correction unit 24a compares when the weather around the vehicle 1 is rain or snow and the weather around the vehicle 1 is sunny or cloudy. The viewpoint distance coefficient C is corrected so that the value becomes small. In other words, the viewpoint distance coefficient correction unit 24a makes a large value when the weather around the vehicle 1 is sunny or cloudy as compared with when the weather around the vehicle 1 is rain or snow. The distance coefficient C is corrected. The environmental information acquired by the environmental information acquisition unit 32 is used for the correction process.

Normally, when the weather around the vehicle 1 is rain or snow, the visibility of the road traffic information display S by the driver is reduced. In such a case, it is highly likely that the driver misses the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a smaller value.

(6) Correction processing based on the driver's visual acuity The viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the driver's visual acuity is low, the value becomes smaller than when the driver's visual acuity is high. To do. In other words, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C so that when the driver's visual acuity is high, the value becomes larger than when the driver's visual acuity is low. The driver information acquired by the driver information acquisition unit 33 is used for such correction processing.

Normally, when the driver's eyesight is low, it is highly likely that the driver will miss the road traffic information display S. Therefore, the viewpoint distance coefficient C is corrected so as to have a smaller value.

By road traffic information detection unit 21, viewpoint detection unit 22, viewpoint distance coefficient calculation unit 23, viewpoint distance coefficient correction unit 24a, recognition degree calculation unit 25, recognition degree determination unit 26, display control unit 27, and correction information acquisition unit 34. , The main part of the driving support device 100a is configured.

Since the hardware configuration of the main part of the driving support device 100a is the same as that described with reference to FIG. 4 in the first embodiment, the illustration and description will be omitted. That is, the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24a, the recognition degree calculation unit 25, the recognition degree determination unit 26, the display control unit 27, and the correction information acquisition unit. Each function of 34 may be realized by the processor 51 and the memory 52, or may be realized by the dedicated processing circuit 53.

Next, with reference to the flowchart of FIG. 10, regarding the operation of the driving support device 100a, the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the correction information acquisition unit 34, and the viewpoint distance coefficient correction The operation of the unit 24a and the recognition degree calculation unit 25 will be mainly described. In FIG. 10, the same steps as those shown in FIG. 5 are designated by the same reference numerals, and the description thereof will be omitted.

First, in step ST1, the road traffic information detection unit 21 executes the road traffic information detection process. When the road traffic information display S is included in the forward image I1 (step ST2 “YES”), the viewpoint detection unit 22 executes the viewpoint detection process in step ST3. Next, in step ST4, the viewpoint distance coefficient calculation unit 23 calculates the viewpoint distance coefficient C.

Next, in step ST7, the correction information acquisition unit 34 acquires the correction information. Next, in step ST5a, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C. At this time, the viewpoint distance coefficient correction unit 24a executes at least one of the above correction processes (1) to (6) in addition to the correction process based on the content of the road traffic information display S. When the viewpoint distance coefficient correction unit 24a executes only the correction process (1) among the correction processes (1) to (6) above, the correction information is unnecessary. In this case, the process of step ST7 may be skipped.

Next, in step ST6, the recognition degree calculation unit 25 calculates the recognition degree D2.

As described above, the viewpoint distance coefficient correction unit 24a executes at least one of the above correction processes (1) to (6) in addition to the correction process based on the content of the road traffic information display S. By using the corrected viewpoint distance coefficient C'for calculating the recognition degree D2, it is possible to further improve the accuracy of determining whether or not the road traffic information display S is recognized by the driver.

Next, a modified example of the driving support device 100a will be described with reference to FIG.

As shown in FIG. 11, the correction information acquisition unit 34 may be provided outside the driving support device 100a. That is, the driving support device 100a is provided by the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24a, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27. The main part of the above may be configured.

Next, with reference to FIG. 12, another modification of the driving support device 100a will be described.

As shown in FIG. 12, the recognition degree determination unit 26 and the display control unit 27 may be provided outside the driving support device 100a. That is, the main part of the driving support device 100a is composed of the road traffic information detection unit 21, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24a, and the recognition degree calculation unit 25. You may.

In addition, as the driving support device 100a, various modifications similar to those described in the first embodiment can be adopted.

As described above, in the driving support device 100a of the second embodiment, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the content of the road traffic information display S and the viewpoint distance based on the size of the area A. Correct the coefficient C. As a result, the accuracy of determining whether or not the road traffic information display S is recognized by the driver can be further improved.

Further, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the content of the road traffic information display S, and corrects the viewpoint distance coefficient C based on the traveling state of the vehicle 1. More specifically, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the traveling speed of the vehicle 1 or the turning amount of the vehicle 1. As a result, the accuracy of determining whether or not the road traffic information display S is recognized by the driver can be further improved.

Further, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the content of the road traffic information display S, and corrects the viewpoint distance coefficient C based on the environment outside the vehicle 1. More specifically, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the brightness around the vehicle 1 or the weather around the vehicle 1. As a result, the accuracy of determining whether or not the road traffic information display S is recognized by the driver can be further improved.

Further, the viewpoint distance coefficient correction unit 24a corrects the viewpoint distance coefficient C based on the content of the road traffic information display S, and corrects the viewpoint distance coefficient C based on the driver's visual acuity. As a result, the accuracy of determining whether or not the road traffic information display S is recognized by the driver can be further improved.

Embodiment 3.
FIG. 13 is a block diagram showing a main part of the driving support system including the driving support device according to the third embodiment. The driving support device 100b of the third embodiment will be described with reference to FIG. Moreover, the driving support system 200b including the driving support device 100b will be described. In FIG. 13, the same blocks as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 13, the control device 2b is provided in the vehicle 1. The control device 2b is composed of, for example, an ECU.

Further, the vehicle 1 is provided with the voice output device 11. The audio output device 11 is composed of, for example, a speaker.

The main part of the driving support system 200b is composed of the control device 2b, the first imaging device 3, the second imaging device 4, the display device 5, and the audio output device 11. The driving support device 100b is provided in the control device 2b.

The importance determination unit 41 acquires the detection result by the road traffic information detection unit 21. The importance determination unit 41 determines the importance D3 of the road traffic information display S based on the content of the road traffic information display S by using the acquired detection result.

The importance D3 is, for example, one of three values (that is, the first value, the second value, and the third value) according to the magnitude of the influence of the content of the road traffic information display S on the driving of the vehicle 1. It is determined that it is one of the values. The larger the value of the importance D3, the greater the influence that the content of the road traffic information display S has on the driving of the vehicle 1.

For example, the importance determination unit 41 indicates that the content of the road traffic information display S requests the stop of the vehicle 1 (for example, the road traffic information display S is a regulation sign meaning "closed" or "closed". If), it is determined that the importance D3 is the third value. Further, the importance determination unit 41 determines that the importance D3 is the second value when the content of the road traffic information display S requests the deceleration or lane change of the vehicle 1. In addition, the importance determination unit 41 determines that the importance D3 is the first value in other cases.

When the recognition level D2 is determined by the recognition level determination unit 26 to be less than the threshold value Dth, the voice output control unit 42 sets the importance level D3 to a predetermined reference value Dref (for example, a second value) by the importance level determination unit 41. When it is determined that the above is the case, the control for outputting the voice corresponding to the road traffic information display S to the voice output device 11 (hereinafter referred to as “voice output control”) is executed. The voice is, for example, a voice that reads out a sentence corresponding to the content of the road traffic information display S.

The output control unit 43 is composed of the display control unit 27 and the audio output control unit 42. In addition, road traffic information detection unit 21, viewpoint detection unit 22, viewpoint distance coefficient calculation unit 23, viewpoint distance coefficient correction unit 24, recognition degree calculation unit 25, recognition degree determination unit 26, display control unit 27, importance determination unit 41. And the voice output control unit 42 constitutes the main part of the driving support device 100b.

Since the hardware configuration of the main part of the driving support device 100b is the same as that described with reference to FIG. 4 in the first embodiment, the illustration and description will be omitted. That is, road traffic information detection unit 21, viewpoint detection unit 22, viewpoint distance coefficient calculation unit 23, viewpoint distance coefficient correction unit 24, recognition degree calculation unit 25, recognition degree determination unit 26, display control unit 27, importance determination unit 41. Each function of the voice output control unit 42 and the voice output control unit 42 may be realized by the processor 51 and the memory 52, or may be realized by the dedicated processing circuit 53.

Next, with reference to the flowchart of FIG. 14, the operation of the driving support device 100b will be described focusing on the operations of the recognition degree determination unit 26, the importance determination unit 41, and the output control unit 43. In FIG. 14, the same steps as those shown in FIG. 6 are designated by the same reference numerals, and the description thereof will be omitted.

First, in step ST11, the recognition degree determination unit 26 acquires the recognition degree D2 calculated by the recognition degree calculation unit 25, and determines whether or not the acquired recognition degree D2 is equal to or greater than the threshold value Dth. .. When it is determined that the recognition degree D2 is less than the threshold value Dth (step ST11 “NO”), the importance degree determination unit 41 determines the importance degree D3 of the road traffic information display S in step ST13. When it is determined that the importance D3 is less than the reference value Dref (step ST13 “NO”), the output control unit 43 executes the first display control in step ST12. On the other hand, when it is determined that the importance D3 is equal to or higher than the reference value Dreff (step ST13 “YES”), the output control unit 43 executes the first display control and the audio output control in step ST14.

In this way, when it is determined that the recognition degree D2 is less than the threshold value Dth and the importance degree D3 is determined to be equal to or higher than the reference value Dref, the display control unit 27 executes the first display control. In addition, the voice output control unit 42 executes voice output control. As a result, the driver can more reliably confirm the important road traffic information display S among the road traffic information display S that he / she overlooked.

Next, a modified example of the driving support device 100b will be described.

The driving support device 100b may have the same viewpoint distance coefficient correction unit 24a as the driving support device 100a instead of the viewpoint distance coefficient correction unit 24. Further, the control device 2b may have the same correction information acquisition unit 34 as the control device 2a. In this case, the driving support system 200b may include sensors 6, a wireless communication device 7, a storage device 9, and an operation input device 10 similar to the driving support system 200a.

In addition, the driving support device 100b can employ various modifications similar to those described in the first and second embodiments.

As described above, the driving support device 100b of the third embodiment has the importance of determining whether or not the importance D3 of the road traffic information display S is equal to or higher than the reference value Dref based on the content of the road traffic information display S. When the determination unit 41 determines that the recognition degree D2 is less than the threshold value Dth and the importance degree D3 is determined to be equal to or higher than the reference value Dref, the voice output device corresponds to the road traffic information display S. A voice output control unit 42 that executes control to output to 11 is provided. As a result, the driver can more reliably confirm the important road traffic information display S among the road traffic information display S that he / she overlooked.

Embodiment 4.
FIG. 15 is a block diagram showing a main part of the driving support system including the driving support device according to the fourth embodiment. FIG. 15 is calculated, and the operation support device 100c according to the fourth embodiment will be described later. Moreover, the driving support system 200c including the driving support device 100c will be described. In FIG. 15, the same blocks as those shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted. Further, in FIG. 15, the same blocks as those shown in FIG. 9 are designated by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 15, the control device 2c is provided in the vehicle 1. The control device 2c is composed of, for example, an ECU.

The main part of the driving support system 200c is composed of the control device 2c, the first imaging device 3, the second imaging device 4, the display device 5, the storage device 9, and the operation input device 10. The driving support device 100c is provided in the control device 2c.

The road traffic information detection unit 21a executes the same road traffic information detection process as the road traffic information detection unit 21. The road traffic information detection unit 21a stores the data indicating the road traffic information display S detected by the road traffic information detection process in the storage device 9. The storage device 9 holds the stored data for a predetermined time.

The display control unit 27a executes the first display control similar to the display control unit 27. In addition to this, when the operation for instructing the display of the image I3 corresponding to the road traffic information display S indicated by the data stored in the storage device 9 is input to the operation input device 10, the display control unit 27a is concerned. The control for displaying the image I3 corresponding to the road traffic information display S indicated by the stored data on the display device 5 (hereinafter referred to as “second display control”) is executed.

By the second display control, the driver can confirm the content of the road traffic information display S that the vehicle 1 has passed at any timing regardless of whether or not he / she overlooked it.

The driving support device 100c is required by the road traffic information detection unit 21a, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27a. The part is composed.

Since the hardware configuration of the main part of the driving support device 100c is the same as that described with reference to FIG. 4 in the first embodiment, the illustration and description will be omitted. That is, the functions of the road traffic information detection unit 21a, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, the recognition degree calculation unit 25, the recognition degree determination unit 26, and the display control unit 27a are It may be realized by the processor 51 and the memory 52, or may be realized by the dedicated processing circuit 53.

Next, a modified example of the driving support device 100c will be described with reference to FIG.

As shown in FIG. 16, the recognition degree determination unit 26 may be provided outside the driving support device 100c. That is, the main part of the driving support device 100c is composed of the road traffic information detection unit 21a, the viewpoint detection unit 22, the viewpoint distance coefficient calculation unit 23, the viewpoint distance coefficient correction unit 24, and the recognition degree calculation unit 25. You may.

Next, another modification of the driving support device 100c will be described.

The driving support device 100c may have the same viewpoint distance coefficient correction unit 24a as the driving support device 100a instead of the viewpoint distance coefficient correction unit 24. Further, the control device 2c may have the same correction information acquisition unit 34 as the control device 2a. In this case, the driving support system 200c may include the same sensors 6 and the wireless communication device 7 as the driving support system 200a.

Further, the driving support device 100c may have the same importance determination unit 41 and voice output control unit 42 as the driving support device 100b. In this case, the driving support system 200c may include an audio output device 11 similar to the driving support system 200b.

In addition, as the driving support device 100c, various modifications similar to those described in the first to third embodiments can be adopted.

As described above, the driving support device 100c of the fourth embodiment is controlled to display the image I3 corresponding to the road traffic information display S on the display device 5 in response to the operation input to the operation input device 10 (second display control). ) Is included in the display control unit 27a. As a result, the driver can confirm the content of the road traffic information display S that the vehicle 1 has passed at any timing regardless of whether or not he / she overlooked it.

It should be noted that, within the scope of the invention, the invention of the present application is capable of freely combining the respective embodiments, modifying any constituent element of each embodiment, or omitting any constituent element in each embodiment. ..

The driving support device of the present invention can be used for driving support of a vehicle.

1 vehicle, 2, 2a, 2b, 2c control device, 3 first imaging device, 4 second imaging device, 5 display device, 6 sensors, 7 wireless communication device, 8 server device, 9 storage device, 10 operation input device , 11 voice output device, 21,21a road traffic information detection unit, 22 viewpoint detection unit, 23 viewpoint distance coefficient calculation unit, 24, 24a viewpoint distance coefficient correction unit, 25 recognition degree calculation unit, 26 recognition degree determination unit, 27, 27a Display control unit, 31 Vehicle information acquisition unit, 32 Environmental information acquisition unit, 33 Driver information acquisition unit, 34 Correction information acquisition unit, 41 Importance determination unit, 42 Voice output control unit, 43 Output control unit, 51 Processor , 52 memory, 53 processing circuit, 100, 100a, 100b, 100c operation support device, 200, 200a, 200b, 200c operation support system.

Claims (9)

1. A road traffic information detection unit that detects an area corresponding to the road traffic information display in the front image and detects the content of the road traffic information display,
   A viewpoint detection unit that detects a position corresponding to the driver's viewpoint in the front image using an in-vehicle image, and a viewpoint detection unit.
   A viewpoint distance coefficient calculation unit that calculates a viewpoint distance coefficient according to the distance of the region to the position,
   A viewpoint distance coefficient correction unit that corrects the viewpoint distance coefficient based on the above contents,
   A recognition degree calculation unit that calculates the recognition degree of the road traffic information display by the driver by integrating the corrected viewpoint distance coefficient.
   A driving support device for vehicles equipped with.
2. A recognition degree determination unit that determines whether or not the road traffic information display has been recognized by the driver by determining whether or not the recognition degree is equal to or higher than a threshold value.
   When it is determined that the recognition degree is less than the threshold value, a display control unit that executes control to display an image corresponding to the road traffic information display on the display device, and
   The driving support device according to claim 1, further comprising.
3. The driving support device according to claim 2, wherein the recognition degree determination unit determines whether or not the recognition degree is equal to or higher than the threshold value when the road traffic information display goes out of the front image. ..
4. Any of claims 1 to 3, wherein the viewpoint distance coefficient correction unit corrects the viewpoint distance coefficient based on the content and also corrects the viewpoint distance coefficient based on the size of the region. The driving support device according to item 1.
5. Of claims 1 to 3, the viewpoint distance coefficient correction unit corrects the viewpoint distance coefficient based on the content and also corrects the viewpoint distance coefficient based on the traveling state of the vehicle. The driving support device according to any one of the above.
6. Of claims 1 to 3, the viewpoint distance coefficient correction unit corrects the viewpoint distance coefficient based on the contents and also corrects the viewpoint distance coefficient based on the environment outside the vehicle. The driving support device according to any one of the above.
7. Of claims 1 to 3, the viewpoint distance coefficient correction unit corrects the viewpoint distance coefficient based on the contents and also corrects the viewpoint distance coefficient based on the visual acuity of the driver. The driving support device according to any one of the above.
8. Based on the above contents, the importance determination unit for determining whether or not the importance of the road traffic information display is equal to or higher than the reference value, and
   When it is determined that the recognition level is less than the threshold value and the importance level is determined to be equal to or higher than the reference value, control is performed to output the voice corresponding to the road traffic information display to the voice output device. The audio output control unit to be executed and
   The driving support device according to claim 2 or 3, wherein the driving support device is provided.
9. The driving support device according to claim 1, further comprising a display control unit that executes control for displaying an image corresponding to the road traffic information display on the display device in response to an operation input to the operation input device.

Images