JP2018151904A - Driver monitoring device, driver monitoring method and program for driver monitoring - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a driver to maintain or promote his/her recognition of the road environment where a vehicle travels in an automatic operation mode.SOLUTION: A driver monitoring device includes: an operation mode determination unit for determining whether or not an operation mode of a vehicle is an automatic operation mode; an object detection unit for detecting an object present within a predetermined range of the vehicle from a road environment where the vehicle travels when the operation mode is the automatic operation mode; a request signal generation unit for generating a request signal requesting to output a question about the object to a driver; and a request signal output unit for outputting the request signal.SELECTED DRAWING: Figure 3

Description

  The present invention relates to, for example, a driver monitoring device that monitors a driver of a vehicle, a driver monitoring method, and a program for driver monitoring.

In recent years, as a driving mode of a vehicle, in addition to a manual driving mode in which the vehicle is driven based on a driving operation of the driver, an automatic driving mode in which the vehicle is driven along a preset route regardless of the driving operation of the driver Development is underway.
Development of a technique for presenting an index at a position where safety should be confirmed in a vehicle is also underway (see Patent Document 1).

JP2015-184975A

  Even if the driving mode of the vehicle is the automatic driving mode, the driver is required to maintain concentration on driving. Even if the driver does not perform the driving operation, the driver needs to watch the road environment in which the vehicle travels, such as in front of the vehicle. Therefore, there is a demand for a technology that maintains or promotes recognition of the road environment in which the vehicle travels with respect to the driver.

  The present invention has been made paying attention to the above circumstances, and a driver monitoring device, a driver monitoring method, and a driver monitoring device capable of maintaining or promoting recognition of a road environment in which a vehicle travels for a driver in an automatic driving mode. It is intended to provide a program for driver monitoring.

  In order to solve the above-mentioned problem, a first aspect of the present invention is an operation mode determination unit that determines whether or not the operation mode of the vehicle is an automatic operation mode, and the operation mode is the automatic operation mode. , A target detection unit for detecting a target existing within a predetermined range of the vehicle from a road environment where the vehicle travels, and a request signal for generating a request signal for requesting the driver to output a question regarding the target It is a driver | operator monitoring apparatus provided with the production | generation part and the request signal output part which outputs the said request signal.

  According to a second aspect of the present invention, in the driver monitoring device according to the first aspect, the degree of recognition for detecting the degree of recognition of the driver by the driver based on whether the driver answers the question. A detection unit is provided.

  According to a third aspect of the present invention, in the driver monitoring device according to the first aspect, the request signal requests the driver to visually recognize the object before outputting the question. Is.

  According to a fourth aspect of the present invention, in the driver monitoring device according to the second aspect, an instruction signal output unit that outputs an instruction signal instructing provision of assistance to the driver when the degree of recognition does not satisfy a standard. Is provided.

  According to a fifth aspect of the present invention, in the driver monitoring device of the second aspect, when the degree of recognition does not satisfy a standard, a speed control signal for stopping the vehicle or reducing the speed of the vehicle is output. A speed control signal output unit is provided.

  According to a sixth aspect of the present invention, in the driver monitoring device according to any one of the first to fifth aspects, the target detection unit detects a road sign, a road marking, or an electric display panel as the target. It is a thing.

  According to a seventh aspect of the present invention, there is provided an operation mode determination process for determining whether or not a vehicle operation mode is an automatic operation mode, and a road environment in which the vehicle travels when the operation mode is the automatic operation mode. An object detection process for detecting an object existing within a predetermined range of the vehicle, a request signal generation process for generating a request signal for requesting a driver to output a question about the object, and the request signal A driver monitoring method comprising a request signal output process for outputting.

  An eighth aspect of the present invention is a program for driver monitoring that causes a computer to function as each unit included in the driver monitoring device according to any one of the first to sixth aspects.

  According to the first aspect of the present invention, when the driving mode is the automatic driving mode, the driver monitoring device can request an output of a question regarding an object detected from a road environment where the vehicle travels. Thereby, the driver | operator can recognize the question regarding object. Even if the driving mode is the automatic driving mode, the driver can pay attention to the road environment in which the vehicle travels, such as in front of the vehicle, and can maintain a state where the vehicle can be safely driven.

  That is, according to the first aspect, the driver monitoring device can maintain or promote recognition of the road environment in which the vehicle travels for the driver in the automatic driving mode.

  According to the second aspect of the present invention, the driver monitoring device can appropriately detect the degree of recognition of the driver's road environment by referring to the answer having the driver's intention.

According to the third aspect of the present invention, the driver monitoring device can prompt the driver to visually recognize the object before outputting the question. As a result, the driver can increase the awareness of paying attention to the road environment in which the vehicle is traveling.
According to the fourth aspect of the present invention, the driver monitoring device can recover the driver's concentration by providing assistance to the driver. The driver can recognize that the driver is not in a state suitable for driving the vehicle in the automatic driving mode. Therefore, the driver can pay attention again to the road environment in which the vehicle travels, and can increase the degree of concentration.

  According to the fifth aspect of the present invention, the driver monitoring device can safely control the vehicle even when the driver cannot pay attention to the road environment.

  According to the sixth aspect of the present invention, the driver monitoring device can use a road sign, a road sign, or an electric display board that is preferably visually recognized by the driver. Thus, the driver can pay attention to an object that is preferably visually recognized in the automatic driving mode.

  According to the seventh aspect of the present invention, the driver monitoring method can obtain the same effects as those of the first aspect described above. That is, the driver monitoring method can maintain or promote recognition of the road environment in which the vehicle travels for the driver in the automatic driving mode.

  According to the eighth aspect of the present invention, the driver monitoring program can obtain the same effects as those of the first aspect described above. That is, the driver monitoring program can maintain or promote recognition of the road environment in which the vehicle travels for the driver in the automatic driving mode.

The figure which shows the whole structure of a vehicle provided with the driver | operator monitoring apparatus which concerns on the 1st Embodiment of this invention. The block diagram which shows the structure of the driver | operator monitoring apparatus which concerns on the 1st Embodiment of this invention. The flowchart which shows the procedure of the driver | operator monitoring by the driver | operator monitoring apparatus shown in FIG.

Hereinafter, some embodiments according to the present invention will be described with reference to the drawings.
[First Embodiment]
(Constitution)
FIG. 1 is a diagram showing an overall configuration of a vehicle 1 including a driver monitoring device 2 according to the first embodiment of the present invention. The driver monitoring device 2 is mounted on a vehicle 1 such as a passenger car. The configuration of the driver monitoring device 2 will be described later. The vehicle 1 may be, for example, any one of an automobile, a bus, a truck, a train, and the like, or a vehicle on which a driver (hereinafter also referred to as a driver) other than these rides.

  The vehicle 1 includes, as basic equipment, a power unit 3 including a power source and a transmission, a steering device 4 equipped with a steering wheel 5, and further includes a manual operation mode and an automatic operation mode as operation modes. . An engine and / or a motor is used as the power source.

  The manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example. In the manual driving mode, for example, an operation mode for driving the vehicle 1 based only on the driving operation of the driver and an operation for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driving operation of the driver. Mode is included.

  In the driving operation support control, for example, the steering torque is assisted so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve. The driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included. In manual steering, the vehicle 1 is steered mainly by the driver's operation of the steering wheel 5. In the manual speed adjustment, the speed of the vehicle 1 is adjusted mainly by the driver's accelerator operation or brake operation.

  Note that the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and causing the vehicle 1 to automatically travel. That is, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle 1 within a preset allowable range, but the driving of the vehicle 1 is compulsory under certain conditions (for example, lane departure of the vehicle 1). Interventive control is not included.

  On the other hand, the automatic driving mode is a mode for realizing an operating state in which the vehicle 1 is automatically driven along a road on which the vehicle 1 is traveling, for example. The automatic driving mode includes, for example, a driving state in which the vehicle 1 is automatically driven toward a preset destination without driving by the driver. In the automatic driving mode, it is not always necessary to automatically control all of the vehicle 1, and the driving state in which the driving operation of the driver is reflected in the traveling of the vehicle 1 within a preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in the traveling of the vehicle 1 under certain conditions, while reflecting the driving operation of the driver in the traveling of the vehicle 1 within a preset allowable range.

  The vehicle 1 further includes an outside camera 6, a steering sensor 7, an accelerator pedal sensor 8, a brake pedal sensor 9, a GPS receiver 10, a gyro sensor 11, a vehicle speed sensor 12, a navigation device 13, an automatic A driving control device 14, a driver camera 15, an audio output device 16, and a road-vehicle communication device 17 are provided.

  The vehicle exterior camera 6 is installed at an arbitrary position of the vehicle 1 so that the outside of the vehicle 1 can be photographed. Although one vehicle exterior camera 6 is shown in FIG. 1, the vehicle 1 may include a plurality of vehicle exterior cameras that capture different directions. The outside camera 6 continuously captures the road environment in front of the vehicle 1 and in the vicinity of the vehicle 1. The vehicle exterior camera 6 is activated in response to the start of operation of the vehicle 1 and continuously captures the outside of the vehicle 1. The outside camera 6 is an example of a sensor that monitors the outside of the vehicle 1. The vehicle exterior camera 6 outputs the captured image (hereinafter also referred to as vehicle exterior image data) to the driver monitoring device 2 and the automatic operation control device 14.

The steering sensor 7 detects a steering angle. The steering sensor 7 outputs the detection result to the automatic driving control device 14.
The accelerator pedal sensor 8 detects an operation amount of the accelerator pedal. The accelerator pedal sensor 8 outputs the detection result to the automatic operation control device 14.
The brake pedal sensor 9 detects the operation amount of the brake pedal. The brake pedal sensor 9 outputs the detection result to the automatic operation control device 14.
The GPS receiver 10 receives current position information of the vehicle 1. The GPS receiver 10 outputs current position information to the navigation device 13 and the automatic driving control device 14.
The gyro sensor 11 detects the behavior of the vehicle 1. The gyro sensor 11 outputs the detection result to the automatic operation control device 14.
The vehicle speed sensor 12 detects the speed of the vehicle 1. The vehicle speed sensor 12 outputs the detection result to the automatic driving control device 14.

The navigation device 13 is an example of a video display device that includes a display 131 and an input unit 132.
The display 131 displays an image.
The input unit 132 receives input from a driver or the like. The input unit 132 may be a touch panel or a push button.

The navigation device 13 stores map information. The navigation device 13 uses the information about the destination input by the driver or the like at the input unit 132, the map information, and the current position information from the GPS receiver 10 to obtain route information from the current position to the destination. Extract. The navigation device 13 displays route information on the display 131. The navigation device 13 can also display information other than route information on the display 131.
The navigation device 13 outputs route information to the automatic driving control device 14.

The route information described above may include not only information on the route from the current position to the destination, but also information on the road environment from the current position to the destination.
The information on the road environment may include information on the speed limit of the road that passes from the current position to the destination.
The information on the road environment may include information on installations on the road passing from the current position to the destination. The installation object is, for example, a road sign or an electric light display board, but may be one installed on the road in addition to these. The information of the installed object may include, for example, position information of the installed object and content information of the installed object. The content information of the installation object is, for example, information such as letters, numbers, symbols, and figures drawn on the installation object.
Note that the route information may include information other than the above example as information on the road environment.

The configuration of the automatic operation control device 14 will be described.
The automatic operation control device 14 automatically controls traveling of the vehicle 1 when the operation mode is the automatic operation mode.
The automatic driving control device 14 includes an outside image data from the outside camera 6, a detection result from the steering sensor 7, a detection result from the accelerator pedal sensor 8, a detection result from the brake pedal sensor 9, and a GPS receiver 10. Current position information, a detection result from the gyro sensor 11, a detection result from the vehicle speed sensor 12, and route information from the navigation device 13 are acquired. For example, the automatic driving control device 14 automatically controls the traveling of the vehicle 1 based on these information and traffic information acquired by the road-to-vehicle communication device 17 described later.

  Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed). Automatic steering is an operating state in which the steering device 4 is automatically controlled. Automatic steering includes LKAS (Lane Keeping Assist System). For example, the LKAS automatically controls the steering device 4 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation. Note that the steering operation of the driver may be reflected in the steering of the vehicle 1 within a range (allowable range) in which the vehicle 1 does not deviate from the travel lane even during execution of LKAS. Note that automatic steering is not limited to LKAS.

  Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled. Automatic speed adjustment includes ACC (Adaptive Cruise Control). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle. The automatic operation control device 14 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed. Further, the automatic driving control device 14 can operate the driver's accelerator operation (for example, accelerator) up to a preset maximum permissible speed (for example, the maximum speed legally determined on the traveling road) even when ACC is being executed. The vehicle 1 can be accelerated according to the pedal operation. The automatic speed adjustment is not limited to ACC but also includes CC (Cruise Control).

The configuration of the driver camera 15 will be described.
The driver camera 15 is installed, for example, at a position that is in front of the driver as on a dashboard. The driver camera 15 is an example of a sensor that monitors a driver. The driver camera 15 is activated in response to the start of driving of the vehicle 1 and continuously captures a predetermined range including the driver's face. The driver camera 15 outputs the captured image (hereinafter referred to as driver image data) to the driver monitoring device 2.

  The audio output device 16 includes a speaker 161. The audio output device 16 outputs various information as audio.

The road-to-vehicle communication device 17 is installed on the road and communicates with a roadside device located in the vicinity of the vehicle 1 by radio. The road-vehicle communication device 17 transmits, for example, information related to the vehicle 1 to the roadside machine through road-vehicle communication. The information regarding the vehicle 1 is, for example, position information and speed information, but is not limited thereto. On the other hand, the road-to-vehicle communication device 17 receives various information exemplified below from the roadside machine through road-to-vehicle communication. The road-to-vehicle communication device 17 may receive, for example, information about another vehicle that is close to the vehicle 1 from the roadside machine. The information regarding another vehicle is, for example, position information and speed information, but is not limited thereto. The road-to-vehicle communication device 17 may receive traffic information from a roadside machine, for example. The traffic information includes various kinds of information useful for traveling of the vehicle 1. The traffic information may include, for example, lane regulation information and closed information. The traffic information may include information on an installation located in the vicinity of the roadside machine. The road-vehicle communication device 17 may receive information other than the information described above from the roadside device.
The road-to-vehicle communication device 17 outputs information obtained by road-to-vehicle communication as described above to the automatic driving control device 14.

The configuration of the above driver monitoring device 2 will be described.
The driver monitoring device 2 monitors the degree of recognition of the driver with respect to the road environment. Specifically, the degree of recognition of the road environment is an indicator of whether the driver has recognized an object that is a part of the road environment.
Note that the driver monitoring device 2 may detect the state of the driver from the driver image data and estimate the driver's concentration of driving based on the state of the driver. The state of the driver is not particularly limited, but includes, for example, at least one of the indicators such as the driver's drowsiness and looking aside. The driving concentration degree is a degree suitable for the driver to drive the vehicle 1. The driver monitoring device 2 can determine whether the driver is in a state suitable for driving the vehicle 1 based on the driving concentration degree.

FIG. 2 is a block diagram illustrating a configuration of the driver monitoring apparatus 2 as an example.
The driver monitoring device 2 includes an input / output interface unit 21, a storage unit 22, and a control unit 23.

  The input / output interface unit 21 connects, for example, the outside camera 6, the navigation device 13, the automatic driving control device 14, and the audio output device 16 to the control unit 23.

The configuration of the storage unit 22 will be described.
The storage unit 22 is a non-volatile memory that can be written and read at any time, such as a solid state drive (SSD) and a hard disk drive (HDD). The storage unit 22 includes an outside image data storage unit 221.
The vehicle outside image data storage unit 221 stores vehicle outside image data that the control unit 23 acquires from the vehicle outside camera 6.

The configuration of the control unit 23 will be described.
The control unit 23 includes a processor 231 and a memory 232.
The processor 231 is, for example, a CPU (Central Processing Unit) that constitutes a computer. The configuration of each unit included in the processor 231 will be described later. Note that although one processor 231 is shown in FIG. 2, the control unit 23 may include one or more processors.
The memory 232 is a volatile memory such as a RAM (Random-Access Memory). The memory 232 includes a program that causes the processor 231 to function as each unit included in the processor 231. The program can also be referred to as an instruction for operating the processor 231. The program is stored in the storage unit 22 and is read from the storage unit 22 to the memory 232. The program in the memory 232 is read by the processor 231. The first embodiment may be realized by a program.

The configuration of each unit included in the processor 231 will be described.
The processor 231 includes a driving mode determination unit 2311, a vehicle exterior image data acquisition unit 2312, a target detection unit 2313, a request signal generation unit 2314, a signal output unit 2315, a response signal acquisition unit 2316, and a recognition level detection unit 2317. With. Each unit may be distributed among one or more processors.

The driving mode determination unit 2311 determines whether or not the driving mode of the vehicle 1 is an automatic driving mode. The operation mode determination unit 2311 may determine whether or not the operation mode is an automatic operation mode, for example, as follows. The driver monitoring device 2 receives information regarding the operation mode from the automatic operation control device 14 via the input / output interface unit 21 when the operation mode is switched. The operation mode determination unit 2311 determines whether the operation mode is the automatic operation mode based on information from the automatic operation control device 14.
The operation mode determination unit 2311 outputs information indicating that the operation mode is the automatic operation mode to the target detection unit 2313.

  The vehicle exterior image data acquisition unit 2312 acquires vehicle exterior image data from the vehicle exterior camera 6 via the input / output interface unit 21. The vehicle exterior image data acquisition unit 2312 stores the vehicle exterior image data in the vehicle exterior image data storage unit 221.

  When the driving mode is the automatic driving mode, the target detection unit 2313 detects a target (hereinafter also referred to as a visual target) existing within a predetermined range of the vehicle 1 from the road environment in which the vehicle 1 travels. The predetermined range is set to a range visible to the driver, and the target detection unit 2313 detects a target as an object visible to the driver. For example, the target detection unit 2313 detects a visual target located within a predetermined distance in front of the vehicle 1. Even if the driving mode is the automatic driving mode, the visual recognition target is preferably viewed by the driver in the road environment in which the vehicle 1 travels from the viewpoint of safety. The visual recognition target is, for example, the above-described installation object, but may be an object other than this, such as a pedestrian. For example, the target detection unit 2313 may detect the visual recognition target as follows. The object detection unit 2313 recognizes that the operation mode is the automatic operation mode based on the information from the operation mode determination unit 2311. The target detection unit 2313 acquires the outside image data from the outside image data storage unit 221. The target detection unit 2313 detects the visual recognition target from the image data outside the vehicle.

  For example, the target detection unit 2313 may detect the visual recognition target from the image data outside the vehicle as follows. The target detection unit 2313 compares a list of visual targets registered in advance with image data outside the vehicle. The list of visual recognition targets may be stored in the storage unit 22. The target detection unit 2313 detects one or more visual targets included in the visual target list from the image data outside the vehicle.

  Further, the target detection unit 2313 detects the content of the visual recognition target. If the visual recognition target is an installation or road marking, the visual recognition content is information such as characters, numbers, symbols, and figures displayed on the installation or road marking, for example. When the display content does not change like a road sign or a road marking, the target detection unit 2313 may detect the content of the visual target with reference to the above-described list of visual targets, for example. In the case where the display content changes like an electric light display board, the target detection unit 2313 may detect the content of the visual target by image processing, for example.

  The target detection unit 2313 outputs information indicating that the visual recognition target has been detected and information on the content of the visual recognition target to the request signal generation unit 2314 and the recognition degree detection unit 2317.

The request signal generation unit 2314 generates a request signal for requesting the driver to output a question regarding the visual recognition target. The request signal generation unit 2314 generates a request signal based on information from the target detection unit 2313. For example, the request signal generation unit 2314 generates a request signal for asking the driver about the contents of the visual recognition target. The request signal requests processing from the question output device. When receiving the request signal, the question output device outputs a question to the driver by at least one of display and sound based on the request signal. The question output device is, for example, the navigation device 13 or the voice output device 16, but is not limited thereto.
Request signal generation unit 2314 outputs the request signal to request signal output unit 23151.

The signal output unit 2315 outputs various signals via the input / output interface unit 21.
The signal output unit 2315 includes a request signal output unit 23151, an instruction signal output unit 23152, and a speed control signal output unit 23153.

  The request signal output unit 23151 outputs a request signal. The request signal output unit 23151 outputs a request signal to at least one of the navigation device 13 and the audio output device 16 via the input / output interface unit 21. The timing at which the request signal output unit 23151 outputs the request signal may be before the vehicle 1 passes through the position of the visual recognition target or after it passes through the position of the visual recognition target. Based on the request signal, the navigation device 13 displays a question regarding the visual target for the driver on the display 131 as an image or video. Based on the request signal, the audio output device 16 outputs a question regarding the visual target for the driver from the speaker 161 by voice. For example, if the visual recognition target is a road sign or road marking of a speed limit, the question may relate to the speed limit. For example, if the visual recognition target is an electric display board, the question may relate to the contents of the electric display board. The navigation device 13 receives the driver's answer to the question at the input unit 132 as an answer input unit.

  The instruction signal output unit 23152 outputs an instruction signal that instructs the driver to provide assistance when the degree of recognition of the road environment by the driver does not satisfy the standard. Whether or not the degree of recognition of the road environment satisfies the standard is determined by the recognition degree detection unit 2317 as will be described later. The instruction signal output unit 23152 outputs, for example, an instruction signal to the support providing apparatus. When receiving the instruction signal from instruction signal output unit 23152, the assistance providing apparatus performs assistance for the driver. The support providing device is, for example, the navigation device 13 or the audio output device 16.

  The instruction signal output unit 23152 outputs an instruction signal to at least one of the navigation device 13 and the audio output device 16 via the input / output interface unit 21. Based on the instruction signal, the navigation device 13 displays a warning on the display 131 as an image or video that gives a warning to the driver. The voice output device 16 outputs a warning for giving a warning to the driver from the speaker 161 based on the instruction signal. The warning may be, for example, content that prompts the driver to recognize the road environment, and is not limited to a specific output mode. The driver can recognize from the warning that the driver is not in a state suitable for driving the vehicle 1 in the automatic driving mode. Therefore, the driver can pay attention to the road environment in which the vehicle 1 travels, and can increase the degree of concentration. The instruction signal output unit 23152 may output the instruction signal to a support providing device other than the navigation device 13 and the voice output device 16. For example, the instruction signal output unit 23152 may output an instruction signal to a support providing apparatus that gives an external stimulus such as vibration to the driver. The support for the driver may be any output content that acts on the driver based on the degree of recognition of the road environment. In addition to warnings, alerts, information provision, various support for improving the cognitive state of the road environment Is also included.

  The speed control signal output unit 23153 outputs a speed control signal for stopping the vehicle 1 or reducing the speed of the vehicle 1 when the degree of recognition of the road environment by the driver does not satisfy the standard. The speed control signal output unit 23153 outputs a speed control signal to the automatic operation control device 14 via the input / output interface unit 21. The automatic operation control device 14 stops the vehicle 1 or reduces the speed of the vehicle 1 based on the speed control signal. Thereby, even if the driver cannot pay attention to the road environment, the driver monitoring device 2 can control the vehicle 1 safely.

The response signal acquisition unit 2316 acquires, from the navigation device 13, a response signal including information on the answer input by the driver through the input unit 132 via the input / output interface unit 21. The response signal acquisition unit 2316 outputs the response signal to the recognition level detection unit 2317.
The input unit 132 is an example of an answer input unit that receives an input of a driver's answer to a question. The answer input unit may be equipped with a voice recognition device. In that case, the response signal acquisition unit 2316 may acquire a response signal including information on the speech-recognized answer from the answer input unit.

  The degree-of-recognition detection unit 2317 detects the degree of recognition of the road environment based on whether the driver answers the question. The degree-of-recognition detection unit 2317 may detect the degree of recognition of the road environment as follows, for example. The degree-of-recognition detection unit 2317 compares the answer included in the response signal with the content of the visual target detected by the target detection unit 2313, and determines whether the answer is correct. The degree-of-recognition detection unit 2317 calculates the correct answer rate based on the correctness / incorrectness of the answer to each of the one or more questions. The degree-of-recognition detection unit 2317 compares the correct answer rate with a reference. The standard is for determining whether or not the driver is aware of the road environment in the automatic driving mode. The reference may be arbitrarily changed. When the correct answer rate is equal to or higher than the standard, the recognition level detection unit 2317 determines that the degree of recognition of the road environment satisfies the standard. The degree-of-recognition detection unit 2317 determines that the driver's recognition of the road environment is appropriate.

  On the other hand, when the correct answer rate is lower than the reference, the recognition degree detection unit 2317 determines that the degree of recognition with respect to the road environment does not satisfy the reference. The degree-of-recognition detection unit 2317 determines that the driver's recognition of the road environment is not appropriate. In this case, the recognition degree detection unit 2317 generates at least one of an instruction signal and a speed control signal. The degree-of-recognition detection unit 2317 outputs an instruction signal to the instruction signal output unit 23152. The degree-of-recognition detection unit 2317 outputs a speed control signal to the speed control signal output unit 23153.

  Note that the recognition level detection unit 2317 may omit the process of determining whether the answer is correct in the following cases, for example. The request signal generation unit 2314 generates a request signal including information on the target content, and the request signal output unit 23151 outputs this request signal. The navigation device 13 compares the answer input by the driver with the input unit 132 with the information on the content of the target included in the request signal, and determines whether the answer is correct or incorrect. The navigation device 13 outputs a response signal including correct / incorrect information of the answer. The degree-of-recognition detection unit 2317 detects the degree of recognition of the road environment based on the correctness / incorrectness information of the answer included in the response signal.

(Operation)
Next, operation | movement of the driver | operator monitoring apparatus 2 comprised as mentioned above is demonstrated. FIG. 3 is a flowchart showing a procedure as an example of driver monitoring by the driver monitoring device 2.

  The driving mode determination unit 2311 determines whether or not the driving mode of the vehicle 1 is the automatic driving mode (step S101). The interval at which the operation mode determination unit 2311 determines the operation mode in step S101 may be an arbitrary interval. When the operation mode is not the automatic operation mode (No at Step S101), the operation mode determination unit 2311 continues monitoring until the operation mode becomes the automatic operation mode.

  When the driving mode is the automatic driving mode (step S101, Yes), the target detection unit 2313 detects a visual target existing within a predetermined range of the vehicle 1 from the road environment in which the vehicle 1 travels (step S102). In step S102, the target detection unit 2313 detects a visual recognition target from, for example, image data outside the vehicle.

The request signal generation unit 2314 generates a request signal for requesting the driver to output a question regarding the visual recognition target (step S103).
The request signal output unit 23151 outputs a request signal (step S104).
The degree-of-recognition detection unit 2317 detects the degree of recognition of the road environment based on the correctness of the driver's answer to the question (step S105).

  The degree-of-recognition detection unit 2317 compares the degree of recognition of the road environment with a reference. (Step S106). In step S106, the degree-of-recognition detection unit 2317 compares the correct answer rate based on the driver's answer with a reference, for example.

  When the degree of recognition of the road environment does not satisfy the standard (No at Step S106), the instruction signal output unit 23152 outputs an instruction signal (Step S107). When the degree of recognition with respect to the road environment satisfies the criterion (step S106, Yes), the processing of the driver monitoring device 2 may transition from step S106 to step S101.

  In step S102, the target detection unit 2313 may detect a road sign, a road marking, or an electric display board as a visual recognition target. According to this example, the driver monitoring apparatus 2 can use a road sign, a road sign, or an electric display board that is preferably visually recognized by the driver as a visual recognition target. Accordingly, the driver can pay attention to a visual recognition object that is preferably visually recognized in the automatic operation mode.

  In step S102, the target detection unit 2313 may use traffic information acquired from the road-to-vehicle communication device 17 instead of the image data outside the vehicle. For example, the target detection unit 2313 may detect the visual recognition target using traffic information as follows. The object detection unit 2313 acquires traffic information including information on an installation located near the roadside machine from the roadside machine located near the vehicle 1. The object detection unit 2313 detects an installation object to be visually recognized. The target detection unit 2313 refers to the information of the installation object, and acquires the content of the installation object detected as the visual recognition target.

  In step S102, the target detection unit 2313 may use route information acquired from the navigation device 13 instead of the outside-vehicle image data. For example, the target detection unit 2313 may detect the visual recognition target using the route information as follows. The target detection unit 2313 compares the information on the installation included in the route information with the current position information acquired from the GPS receiver 10, and extracts information on the installation near the current position of the vehicle 1. The target detection unit 2313 detects an installation located near the current position of the vehicle 1 as a visual recognition target. The target detection unit 2313 refers to the information of the installation object, and acquires the content of the installation object detected as the visual recognition target.

  In step S103, the request signal generation unit 2314 may generate a request signal including a request to output a choice of answer to the question in addition to a request to output a question regarding the visual recognition target. On the basis of the request signal, the navigation device 13 displays the answer options together with the question regarding the visual target on the display 131 as an image or video. Similarly, based on the request signal, the audio output device 16 outputs answer options together with the question regarding the visual target from the speaker 161. The navigation device 13 accepts selection of any option by the driver as an answer to the question at the input unit 132. Thus, the driver monitoring device 2 can reduce the burden of answers by the driver. Thereby, the driver can answer the question more easily.

  In step S103, the request signal generation unit 2314 may generate the request signal periodically at a predetermined time interval or a predetermined travel distance interval. In this case, in step S104, the request signal output unit 23151 can periodically output the request signal at a predetermined time interval or a predetermined travel distance interval.

  In step S103, the request signal generation unit 2314 may generate a request signal triggered by a specific event, such as when the target detection unit 2313 detects a predetermined type of road sign.

  In step S104, the request signal output unit 23151 receives the request signal at a point where a change in the road environment is relatively large and the load on the driver's recognition increases, such as when the vehicle 1 is driving near an intersection or turning right or left. May not be generated. That is, the driver monitoring device 2 does not request an output of a question at such a point. Thereby, the load of the answer by the driver is reduced.

In step S104, the request signal output unit 23151 may output the request signal before the vehicle 1 passes through the position to be visually recognized. For example, the driver monitoring device 2 can determine or estimate whether the vehicle 1 is before or after the vehicle 1 passes the position of the visual recognition target as follows.
First, a case where the driver monitoring device 2 detects a visual recognition target using image data outside the vehicle will be described. The driver monitoring device 2 monitors whether or not a visual recognition target can be detected for each of a plurality of continuous image data outside the vehicle. When the driver monitoring device 2 can detect the visual recognition target, the driver monitoring device 2 can determine or estimate that the vehicle 1 is before passing the position of the visual recognition target. On the other hand, if the driver monitoring device 2 detects a transition from the detection state of the visual target to the non-detection state, the driver monitoring device 2 may determine or estimate that the vehicle 1 has passed the visual recognition position. it can.

  Next, a case where the driver monitoring device 2 detects a visual recognition object using traffic information acquired from the road-vehicle communication device 17 will be described. The driver monitoring device 2 determines or estimates whether the vehicle 1 is before or after passing through the vicinity of the roadside device according to information that the vehicle 1 has passed through the vicinity of the roadside device from the roadside device. The driver monitoring device 2 determines or estimates whether the vehicle 1 passes before or after the vehicle 1 passes through the position of the visual target near the roadside device, depending on whether the vehicle 1 passes before or after the roadside device. Can do.

  Next, a case where the driver monitoring device 2 detects a visual recognition target using route information acquired from the navigation device 13 will be described. The driver monitoring device 2 compares the position of the visual target detected from the route information with the current position of the vehicle 1. The driver monitoring device 2 can determine or estimate whether the vehicle 1 is before or after the vehicle 1 passes through the position of the visual target, depending on the relationship between the position of the visual target and the current position of the vehicle 1.

  In this example, before the vehicle 1 passes the position of the visual recognition target, the navigation device 13 displays a question regarding the visual recognition target for the driver on the display 131 with an image or a video. Similarly, before the vehicle 1 passes through the position of the visual recognition target, the voice output device 16 outputs a question regarding the visual recognition target to the driver from the speaker 161 by voice. According to this example, the driver can confirm the visual recognition target after recognizing the question. As a result, the driver can pay attention to the road environment in which the vehicle 1 travels. Further, the driver can appropriately answer the question, so that an unnecessary warning is not received.

  In step S104, the request signal output unit 23151 may output the request signal after passing through the position of the visual recognition target. As described above, the driver monitoring device 2 can determine or estimate whether the vehicle 1 is before or after passing the position of the visual recognition target. In this case, after the vehicle 1 passes through the position of the visual recognition target, the navigation device 13 displays a question regarding the visual recognition target for the driver on the display 131 with an image or a video. Similarly, after the vehicle 1 passes the position of the visual recognition target, the voice output device 16 outputs a question regarding the visual recognition target to the driver from the speaker 161 by voice. According to this example, since the driver cannot grasp when the question is made, the driver always pays attention to the road environment in which the vehicle 1 travels.

  In step S107, the driver monitoring device 2 may output a speed control signal from the speed control signal output unit 23153 instead of outputting the instruction signal from the instruction signal output unit 23152. The driver monitoring device 2 may output an instruction signal from the instruction signal output unit 23152 and may output a speed control signal from the speed control signal output unit 23153. That is, when the degree of recognition with respect to the road environment does not satisfy the standard, the speed control signal output unit 23153 outputs a speed control signal.

  Note that the driver monitoring device 2 may omit the processes in steps S105 to S107. That is, the driver monitoring device 2 may omit the process of answering the question.

(effect)
As described above in detail, in the first embodiment of the present invention, when the driving mode is the automatic driving mode, the driver monitoring device 2 outputs a question regarding the visual target detected from the road environment in which the vehicle 1 travels. Can be requested. Thereby, the driver | operator can recognize the question regarding a visual recognition object. Even if the driving mode is the automatic driving mode, the driver can pay attention to the road environment where the vehicle 1 travels, such as in front of the vehicle 1, and can maintain a state where the vehicle 1 can be safely driven. .

  Therefore, according to the first embodiment, the driver monitoring device 2 can maintain or promote recognition of the road environment in which the vehicle 1 travels with respect to the driver in the automatic driving mode.

  Furthermore, the driver monitoring device 2 can ask the driver questions regarding the visual recognition target. Thereby, the driver strengthens the consciousness of inputting an answer to the question. For this reason, the driver is more conscious of paying attention to the road environment in which the vehicle 1 travels.

  Furthermore, the driver monitoring device 2 can appropriately detect the degree of recognition of the driver with respect to the road environment by referring to the answer having the intention of the driver.

[Second Embodiment]
In the second embodiment, the request signal generated by the request signal generator 2314 is different from that of the first embodiment. In the second embodiment, the request signal generated by the request signal generation unit 2314 requests that the question regarding the visual target be output to the driver, and before the question is output, It also includes requesting the driver to view the vehicle.

Request signal generation unit 2314 outputs the request signal to request signal output unit 23151.
The request signal output unit 23151 outputs a request signal to at least one of the navigation device 13 and the audio output device 16 via the input / output interface unit 21. The timing at which the request signal output unit 23151 outputs the request signal is before the vehicle 1 passes through the position to be viewed.

  Based on the request signal, the navigation device 13 displays an image or video that prompts the driver to visually recognize the visual target on the display 131 before the vehicle 1 passes through the position of the visual target. Note that the timing at which the navigation device 13 displays the question on the display 131 based on the request signal may be before the vehicle 1 passes through the position to be visually recognized, as described in the first embodiment. It may be after the vehicle 1 passes the position of the visual recognition target. Similarly, based on the request signal, the audio output device 16 outputs a sound that prompts the driver to visually recognize the visual target from the speaker 161 before the vehicle 1 passes the position of the visual target. The timing at which the voice output device 16 outputs a question from the speaker 161 based on the request signal may be before the vehicle 1 passes through the position of the visual target, or the vehicle 1 passes through the position of the visual target. It may be later.

  According to the second embodiment, the driver monitoring device 2 can request that the driver be prompted to visually recognize the visual target before outputting the question. As a result, the driver can reinforce awareness of paying attention to the road environment in which the vehicle 1 travels.

[Other Embodiments]
In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine the component covering different embodiment suitably.

  Further, the above embodiment may be realized by a storage medium such as a ROM (Read Only Memory) that stores a program that causes the processor 231 to function as each unit included in the processor 231.

A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited to the following.
(Appendix 1)
Determine whether the driving mode of the vehicle is the automatic driving mode,
When the driving mode is the automatic driving mode, an object existing within a predetermined range of the vehicle from a road environment where the vehicle travels is detected,
Generating a request signal requesting the driver to output a question regarding the subject;
A processor configured to output the request signal;
A memory for storing instructions for operating the processor;
A driver monitoring device comprising:
(Appendix 2)
An operation mode determination process for determining whether or not the vehicle operation mode is an automatic operation mode using at least one processor;
Using the at least one processor, when the driving mode is the automatic driving mode, an object detection process for detecting an object existing within a predetermined range of the vehicle from a road environment in which the vehicle travels;
Using the at least one processor to generate a request signal for requesting the driver to output a question about the subject;
A request signal output process for outputting the request signal using the at least one processor;
A driver monitoring method comprising:

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Driver monitoring device, 3 ... Power unit, 4 ... Steering device, 5 ... Steering wheel, 6 ... Outside vehicle, 7 ... Steering sensor, 8 ... Accelerator pedal sensor, 9 ... Brake pedal sensor, 10 ... GPS Receiver 11, gyro sensor 12, vehicle speed sensor 13, navigation device 14, automatic driving control device 15 driver camera 16 sound output device 17 road-to-vehicle communication device 21 input / output interface unit 22 ... Storage unit, 23 ... Control unit, 131 ... Display, 132 ... Input unit, 161 ... Speaker, 221 ... Outside vehicle image data storage unit, 222 ... Driver image data storage unit, 231 ... Processor, 232 ... Memory, 2311 ... Driving mode determination unit, 2312 ... Outside-vehicle image data acquisition unit, 2313 ... Object detection unit 2314 ... request signal generating unit, 2315 ... signal outputting section, 2316 ... response signal acquiring unit, 2317 ... awareness detector, 23151 ... request signal output unit, 23152 ... instruction signal output unit, 23,153 ... speed control signal output unit.

Claims (8)

An operation mode determination unit for determining whether the operation mode of the vehicle is an automatic operation mode;
When the driving mode is the automatic driving mode, a target detection unit that detects a target existing within a predetermined range of the vehicle from a road environment in which the vehicle travels;
A request signal generator for generating a request signal for requesting the driver to output a question regarding the object; and
A request signal output unit for outputting the request signal;
A driver monitoring device comprising:   The driver monitoring apparatus according to claim 1, further comprising: a recognition degree detection unit that detects a degree of recognition of the driver with respect to the road environment based on whether or not the driver answers the question.   The driver monitoring apparatus according to claim 1, wherein the request signal requests the driver to visually recognize the target before outputting the question. If the degree of recognition does not meet the criteria, an instruction signal output unit that outputs an instruction signal instructing the driver to provide assistance;
The driver monitoring apparatus according to claim 2, comprising:   The driver monitoring apparatus according to claim 2, further comprising: a speed control signal output unit that outputs a speed control signal for stopping the vehicle or reducing the speed of the vehicle when the degree of recognition does not satisfy a standard.   The driver monitoring apparatus according to any one of claims 1 to 5, wherein the object detection unit detects a road sign, a road sign, or an electric light display board as the object. A driving mode determination process for determining whether the driving mode of the vehicle is an automatic driving mode;
When the driving mode is the automatic driving mode, an object detection process for detecting an object existing within a predetermined range of the vehicle from a road environment where the vehicle travels;
A request signal generating process for generating a request signal for requesting the driver to output a question regarding the object;
A request signal output process for outputting the request signal;
A driver monitoring method comprising:   The program for driver | operator monitoring which makes a computer function as each part with which the driver | operator monitoring apparatus of any one of Claim 1 to 6 is provided.