WO2018163492A1 - Driving mode switching control device, method, and program - Google Patents

Driving mode switching control device, method, and program Download PDF

Info

Publication number
WO2018163492A1
WO2018163492A1 PCT/JP2017/036705 JP2017036705W WO2018163492A1 WO 2018163492 A1 WO2018163492 A1 WO 2018163492A1 JP 2017036705 W JP2017036705 W JP 2017036705W WO 2018163492 A1 WO2018163492 A1 WO 2018163492A1
Authority
WO
WIPO (PCT)
Prior art keywords
concentration
monitoring video
shielding
driver
monitoring
Prior art date
Application number
PCT/JP2017/036705
Other languages
French (fr)
Japanese (ja)
Other versions
WO2018163492A9 (en
Inventor
相澤　知禎
初美 青位
岡地　一喜
啓 菅原
充恵 鵜野
光司 滝沢
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Publication of WO2018163492A1 publication Critical patent/WO2018163492A1/en
Publication of WO2018163492A9 publication Critical patent/WO2018163492A9/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to an operation mode switching control device, method and program for switching a vehicle operation mode between a manual operation mode and an automatic operation mode.
  • the automatic driving mode includes, for example, information on a navigation system using Global Positioning System (GPS), traffic information acquired by road-to-vehicle communication, and information on a surrounding monitoring system that monitors the positions and movements of surrounding people and vehicles.
  • GPS Global Positioning System
  • the vehicle can be automatically driven by controlling the power unit, the steering device, the brake, and the like (see, for example, Japanese Patent Laid-Open No. 2015-141053).
  • the automatic driving mode can be expected to reduce the burden on the driving operation of the driver and alleviate the traffic congestion, it is practically difficult to automate all the driving operations from the start of travel to the end of travel. . For this reason, it is required to appropriately switch from the automatic operation mode to the manual operation mode.
  • the operation mode is switched from the automatic operation mode to the manual operation mode, it is necessary to confirm that the driver is in a state where manual operation can be performed prior to the switching.
  • a monitoring camera that monitors the driver's condition may be placed near the meter panel on the back of the steering wheel as viewed from the driver.
  • the shooting range of the camera is blocked by the support member of the steering wheel, and the driver's state monitoring cannot be continued. If the driver's condition cannot be confirmed, the driving operation after switching to the manual operation mode cannot be guaranteed, and the driver's concentration is high.
  • the automatic operation mode can be switched to the manual operation mode. Even in the state, the automatic operation mode is continued. In view of the above, there has been a demand for appropriate measures when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
  • This invention is intended to provide an operation mode switching control device, method and program capable of determining the state of the driver even when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
  • a first aspect according to the first embodiment of the present invention is a vehicle driving mode used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel as viewed from a driver.
  • An operation mode switching control apparatus that performs switching control includes a control unit that determines a concentration degree of the driver based on a monitoring image acquired by the monitoring camera. The control unit determines whether or not the shooting range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel. Is determined, and the concentration degree determined immediately before entering the shielding state is defined as the concentration degree in the shielding state.
  • the control unit determines whether the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Determine whether. Then, when the imaging range of the monitoring camera is shielded by the support member of the steering wheel, the control unit determines the concentration determined at the timing immediately before the shielding as the concentration level when the imaging range of the monitoring camera is shielded. The degree is adopted. For this reason, even when it is desired to use the degree of concentration at the timing when the photographing range of the surveillance camera is shielded by the support member of the steering wheel, for example, whether to switch from automatic driving to manual driving, etc. Can be carried out.
  • the control unit acquires monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera, and includes the acquired monitoring video data
  • a monitoring image acquisition control unit that determines monitoring image data acquired in the shielding state based on a steering angle detected by the steering sensor, and a concentration level in the shielding state is determined immediately before the shielding state is entered.
  • a concentration degree determination unit for determining based on the monitoring video data acquired during the shielding period.
  • the monitoring video acquisition control unit is configured to shield the photographing range of the monitoring camera by the steering wheel support member based on the steering angle detected by the steering sensor.
  • the monitoring video acquired at the time is determined.
  • the concentration level determination unit determines the concentration degree determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the monitoring video acquisition control unit can efficiently sort the monitoring video acquired when the imaging range of the monitoring camera is blocked.
  • the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the monitoring Among the monitoring video data acquired by the video acquisition control unit, the monitoring video data acquired in the shielding state is determined based on the steering angle detected by the steering sensor, and the degree of concentration in the shielding state is determined by A concentration degree determination unit for determining based on the monitoring video data acquired in the non-shielding period immediately before the shielding state.
  • the concentration degree determination unit uses the monitoring video data acquired when the monitoring camera is shielded by the steering wheel based on the steering angle detected by the steering sensor. Determine.
  • the concentration degree determination unit indicates the degree of concentration determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the concentration level determination unit can select the monitoring video data acquired when the imaging range of the monitoring camera is shielded by the steering wheel, and the processing of the monitoring video acquisition control unit can be reduced.
  • the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the acquisition A degree-of-concentration determination unit that determines the degree of concentration of the driver based on the monitored video data, and determines the shielding state based on a steering angle detected by the steering sensor, and immediately before entering the shielding state, An operation mode switching control unit that determines whether or not the operation mode can be switched is used as the concentration level determined by the concentration level determination unit as the concentration level in the shielding state.
  • the monitoring video acquisition control unit acquires monitoring video data from the monitoring video acquired by the monitoring camera at a predetermined cycle.
  • the concentration level determination unit determines the driver's concentration level based on the acquired monitoring video data.
  • the operation mode switching control unit determines the timing at which the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Then, the operation mode switching control unit adopts the concentration determined immediately before entering the shielding state as the concentration when determining whether the operation mode can be switched in the shielding state. For this reason, it is possible to reduce the processing in the monitoring video acquisition control unit and the concentration degree determination unit.
  • a first aspect according to the second embodiment of the present invention is an operation mode switching control for switching a vehicle operation mode, which is used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel when viewed from the driver.
  • the apparatus includes a control unit that determines the degree of concentration of the driver based on a monitoring image acquired by the monitoring camera.
  • the control unit predicts a shielding period in which the imaging range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel.
  • the degree of concentration in the predicted shielding period is determined based on the monitoring video acquired in the non-shielding period before the shielding period.
  • the control unit determines the period during which the imaging range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Predict. Then, the control unit determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, even if it is desired to use the degree of concentration at the timing when the shooting range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, for example, whether or not switching from automatic driving to manual driving is possible. It is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
  • the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from the start point of the shielding period.
  • a shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring image acquisition control unit that acquires monitoring image data from a monitoring image acquired by the monitoring camera at a timing according to the operation signal, and the operation
  • a concentration level determination unit that determines the concentration level of the driver based on the acquired monitoring video data at a timing according to a signal and sets the concentration level as the concentration level in the predicted shielding period. It is a thing.
  • the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and is preset from the predicted start time of the shielding period.
  • An operation signal is output when the previous time is reached.
  • the monitoring video acquisition control unit stores monitoring video data from the monitoring video acquired by the monitoring camera at the timing of receiving the operation signal.
  • the concentration level determination unit determines the concentration level based on the monitoring video data stored by the monitoring video acquisition control unit at the timing when the operation signal is received, and the concentration level is determined as the concentration level in the predicted shielding period. Like to do.
  • the operation mode can be switched at a timing at which the imaging range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, it is set in advance immediately before it is predicted to enter the shielding state It is possible to use the degree of concentration acquired at a timing different from the sampling rate.
  • the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from a start time of the shielding period.
  • a shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring video acquisition control unit that acquires monitoring video data in a predetermined cycle from the monitoring video acquired by the monitoring camera, and the acquired monitoring video data
  • the concentration level determined immediately before receiving the operation signal is set as the concentration level in the predicted shielding period.
  • a determination unit is used to determine the concentration level in the predicted shielding period.
  • the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and presets it from the start time of the predicted shielding period.
  • An operation signal is output at a time point before the period.
  • the monitoring video acquisition control unit samples monitoring video data from the monitoring video acquired by the monitoring camera according to a preset sampling rate.
  • the concentration level determination unit determines the concentration level of the driver at a preset cycle based on the monitoring video data sampled by the monitoring video acquisition control unit. Then, when receiving the operation signal, the concentration determination unit substitutes the concentration determined in the predicted shielding period with the concentration determined immediately before receiving the operation signal. For this reason, it is possible to quickly calculate the degree of concentration at the timing when the operation signal is received.
  • the control unit when the control unit receives the operation signal, the control unit determines whether or not the operation mode can be switched based on the concentration level determined by the concentration level determination unit.
  • the operation mode switching control unit is further provided with the determination result as the determination result in the shielding period.
  • the operation mode switching control unit when the operation mode switching control unit receives the operation signal output from the shielding prediction unit, the operation mode switching control unit represents the concentration level determined by the concentration level determination unit. Get information. The operation mode switching control unit determines whether or not the operation mode can be switched based on the acquired information. The operation mode switching control unit sets the determination result as the determination result in the predicted shielding period. For this reason, when the operation mode switching instruction is given when the surveillance camera is shielded by the steering wheel, the operation mode switching control unit can use the determination result determined immediately before entering the shielding state. Become. That is, it is possible to prepare for an operation mode switching instruction given when the surveillance camera is shielded by the steering wheel.
  • an operation mode switching control device, method, and program capable of determining the state of the driver even when the photographing range of the monitoring camera is blocked by the support member of the steering wheel. it can.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic operation control system including an operation mode switching control device according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to the first embodiment of the present invention.
  • FIG. 3 is a diagram showing the installation position of the driver camera shown in FIG.
  • FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 2 stores the monitoring video data.
  • FIG. 5 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 2 determines the concentration level of the driver.
  • FIG. 6 is a block diagram showing a functional configuration of an operation mode switching control apparatus according to the second embodiment of the present invention.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic operation control system including an operation mode switching control device according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to
  • FIG. 7 is a flowchart illustrating a procedure when the shielding prediction unit illustrated in FIG. 6 outputs an operation signal.
  • FIG. 8 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 6 stores the monitoring video data.
  • FIG. 9 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 6 determines the concentration level of the driver.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic driving control system provided with an operating mode switching control device according to a first embodiment of the present invention.
  • the automatic driving control system is mounted on a vehicle 1 such as a passenger car.
  • the vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. Yes.
  • An engine and / or a motor is used as the power source.
  • the manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example.
  • the manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.
  • the driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example.
  • the driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included.
  • a driver's accelerator operation for example, operation of an accelerator pedal
  • brake operation for example, operation of a brake pedal
  • manual steering manual operation of steering
  • speed manual speed adjustment
  • Adjustment manual operation is also included.
  • manual steering the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4.
  • the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.
  • the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle.
  • the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.
  • the automatic driving mode is, for example, a mode that realizes an operating state in which the vehicle automatically travels along the road on which the vehicle travels.
  • the automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver.
  • the automatic driving mode it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.
  • the automatic operation control system shown in FIG. 1 has an automatic operation control device 5 for executing operation control in the automatic operation mode.
  • the automatic driving control device 5 acquires sensing data from the steering sensor 8, the accelerator pedal sensor 9, the brake pedal sensor 10, the GPS receiver 11, the gyro sensor 12, and the vehicle speed sensor 13, respectively. And these sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and information obtained by a peripheral monitoring system that monitors the positions and movements of surrounding people and vehicles Based on the above, the traveling of the vehicle 1 is automatically controlled.
  • Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed).
  • Automatic steering is an operating state in which the steering device 3 is automatically controlled.
  • Automatic steering includes Lane Keeping Assist (LKA).
  • LKA Lane Keeping Assist
  • the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation.
  • the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range).
  • automatic steering is not limited to LKA.
  • Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled.
  • Automatic speed adjustment includes Adaptive Cruise Control (ACC). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle.
  • the automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed.
  • the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC.
  • the vehicle can be accelerated according to the pedal operation.
  • the automatic speed adjustment is not limited to ACC, but also includes Cruise Control (CC) that performs only constant speed control.
  • CC Cruise Control
  • the automatic operation control system shown in FIG. 1 includes an operation mode switching control device 6, a driver camera 7, and a steering sensor 8 as devices for switching the setting between the manual operation mode and the automatic operation mode. I have.
  • the driver camera 7 is disposed, for example, near the meter panel on the back side of the steering wheel as viewed from the driver.
  • the driver camera 7 captures the driver and obtains a monitoring video about the driver.
  • the driver camera 7 outputs the acquired monitoring video as a video signal to the operation mode switching control device 6.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
  • FIG. 2 is a block diagram illustrating an example of a functional configuration of the operation mode switching control device 6 according to the first embodiment.
  • the operation mode switching control device 6 shown in FIG. 2 is a device that comprehensively controls the automatic operation control device 5 to switch between the manual operation mode and the automatic operation mode.
  • the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.
  • the input / output interface unit 62 receives the video signal output from the driver camera 7 and the steering angle detection signal output from the steering sensor 8.
  • the input / output interface unit converts the received video signal and steering angle detection signal into digital data. Further, the input / output interface unit 62 outputs an operation mode switching control signal output from the control unit 61 to the automatic operation control device 5.
  • the storage unit 63 is a non-volatile memory that can be written and read as needed, such as Solid State Drive (SSD) and Hard Disk Drive (HDD), and volatile like Random Access Memory (RAM). This is realized by a memory or the like.
  • SSD Solid State Drive
  • HDD Hard Disk Drive
  • RAM Random Access Memory
  • the control unit 61 includes a central processing unit (CPU) that constitutes a computer, and a program memory.
  • the control unit 61 causes the CPU to execute a program stored in the program memory so that a monitoring video acquisition control unit 611 and a concentration degree determination unit 612 are control functions necessary for implementing the first embodiment.
  • the operation mode switching control part 613 is implement
  • the monitoring video acquisition control unit 611 takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62.
  • the monitoring video acquisition control unit 611 stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
  • the monitoring image acquisition control unit 611 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62.
  • the monitoring image acquisition control unit 611 determines whether or not the driver camera 7 is in the shielding state based on whether or not the steering angle represented by the captured steering angle data is included in the shielding angle width.
  • FIG. 3 is a schematic diagram showing an example of an installation position of the driver camera 7 shown in FIG.
  • the driver camera 7 images the driver from the back side of the steering wheel 4 as viewed from the driver. For this reason, depending on the steering angle of the steering wheel 4, the photographing range is blocked by the support members 41-1 to 41-3 of the steering wheel 4.
  • the shielding angle width represents the width of the steering angle at which the photographing range of the driver camera 7 is shielded by the support members 41-1 to 41-3.
  • the shielding angle width is set in advance based on, for example, the shape of the steering wheel 4 mounted on the vehicle 1.
  • the monitoring image acquisition control unit 611 determines that the steering angle is included in the shielding angle width
  • the monitoring image acquisition control unit 611 attaches a shielding flag to the monitoring image data sampled while the steering angle is included in the shielding angle width. And stored in the surveillance video storage area 631.
  • the shielding flag is a flag representing monitoring video data sampled while the steering angle is included within the shielding angle width.
  • the concentration degree determination unit 612 reads the monitoring video data stored during this time interval from the monitoring video storage area 631 at a preset time interval.
  • the concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data.
  • the degree-of-concentration determination unit 612 collates the recognized driver movement and / or driver's facial expression with a predetermined criterion for determining the driver's movement and facial expression, and determines the driver's degree of concentration with respect to driving. judge.
  • the concentration level determination unit 612 outputs the determination result of the concentration level to the operation mode switching control unit 613. Further, the concentration degree determination unit 612 stores the determination result of the concentration degree in the concentration degree storage area 632 with a time stamp.
  • the degree of concentration is an index indicating how suitable the driver is for driving.
  • the driver's movements are detected as forward gaze and gaze in a predetermined direction for viewing the rearview mirror or side mirror.
  • a state in which the detected driver's motion is almost occupied by forward gaze and gaze in a predetermined direction is considered that the driver is sufficiently concentrated on driving.
  • the driver is driving, he / she should not be looking closely ahead, talking, sleepy, carrying food in his mouth, or should not be mobile
  • the driver is not concentrated on driving.
  • the driver's facial expression for example, the eye open state of the driver's eyes, the frequency of blinking, or eye movements are detected.
  • the degree of concentration is defined as “high”.
  • 20% or more of the driver's movement is forward gaze or gaze in a predetermined direction, and the driver's movement includes operation of a mobile phone or smartphone and a state of a call.
  • the degree of concentration is defined as “low”.
  • the concentration degree determination unit 612 determines the movement of the driver recognized from the monitoring video data stored in the monitoring video storage area 631 with reference to the determination criterion.
  • the determination criterion is an example, and the designer can appropriately set it appropriately.
  • the determination criterion may be set such that a score is determined in advance for each representative operation, and the total is multiplied by the ratio of each operation to the total time. At this time, the degree of concentration is digitized based on the driver's time-series movement.
  • the concentration level is not limited to two stages.
  • the degree of concentration may be evaluated in three stages by defining the degree of concentration “medium” between the degree of concentration “high” and the degree of concentration “low”.
  • the concentration degree determination unit 612 reads the concentration degree obtained in the immediately preceding determination from the concentration degree storage area 632 when the number of monitoring video data necessary for the concentration degree is not acquired due to the shielding of the driver camera 7 by the steering wheel 4. .
  • the concentration level determination unit 612 outputs the read concentration level to the operation mode switching control unit 613 as the current concentration level.
  • the concentration degree determination unit 612 stores the read concentration degree in the concentration degree storage area 632 with a time stamp.
  • the degree of concentration is described as an example of an index representing the driver's state, but the present invention is not limited to this.
  • the operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612.
  • the operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 with a time stamp.
  • FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit 611 of the control unit 61 provided in the operation mode switching control device 6 shown in FIG. 2 stores the monitoring video data.
  • the driver camera 7 and the steering sensor 8 are activated.
  • the driver camera 7 continuously captures a predetermined range including the driver's face.
  • the driver camera 7 outputs the captured monitoring video to the operation mode switching control device 6 as a video signal.
  • the driver camera 7 may intermittently image the driver at a cycle shorter than the time interval for determining the driver state.
  • the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. By doing so, it is possible to reduce the information amount of the monitoring video data and save the storage capacity of the monitoring video storage area 631.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
  • the monitoring video acquisition control unit 611 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62 (step S41). Further, the monitoring video acquisition control unit 611 receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S42).
  • the monitoring image acquisition control unit 611 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate (step S43). When the steering angle is not included in the shielding angle width (No in step S43), the monitoring video acquisition control unit 611 samples the received monitoring video data at the sampling rate and stores the sampled monitoring video data in the monitoring video storage. It memorize
  • the monitoring video acquisition control unit 611 When the monitoring video data is stored in the monitoring video storage area 631 in step S44 or the monitoring video data to which the shielding flag is attached is stored in the monitoring video storage area 631 in step S45, the monitoring video acquisition control unit 611 operates. It is determined whether or not has been completed (step S46). When the operation is finished (Yes in step S46), the monitoring video acquisition control unit 611 ends the process. On the other hand, when the driving does not end (No in step S46), the monitoring video acquisition control unit 611 shifts the process to step S41, and repeats the operations in steps S41 to S45 until the driving of the vehicle 1 ends.
  • FIG. 5 is a flowchart illustrating an example of a procedure when the concentration degree determination unit 612 of the control unit 61 provided in the operation mode switching control device 6 illustrated in FIG. 2 determines the concentration degree of the driver.
  • the concentration degree determination unit 612 starts timing.
  • the concentration degree determination unit 612 determines whether or not a certain time has elapsed since the start of time measurement (step S51).
  • the concentration degree determination unit 612 stores the preset number of monitoring video data stored in the monitoring video storage area 631 during the predetermined time in the monitoring video storage area 631.
  • Step S52 The concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data.
  • the degree-of-concentration determination unit 612 reads a preset number of monitoring video data each time the monitoring video acquisition control unit 611 newly samples the monitoring video data, and moves the driver based on the read monitoring video data. Etc. may be recognized.
  • the degree-of-concentration determination unit 612 determines whether or not there is data in which the movement of the driver and / or the driver's facial expression cannot be recognized among the read monitoring video data, that is, data with a shielding flag (see FIG. Step S53). When there is data to which the shielding flag is attached (Yes in step S53), the concentration determination unit 612 determines whether there is the number of samples necessary for determining the concentration of the data in which the driver's movement or the like can be recognized. It is determined whether or not (step S54). When the number of samples necessary to determine the concentration level exists (Yes in step S54), the concentration level determination unit 612 determines the concentration level based on data that can recognize the movement of the driver and the like (step S55). ).
  • the concentration level determination unit 612 outputs information indicating the determined concentration level to the operation mode switching control unit 613.
  • the concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S56).
  • the concentration level determination unit 612 refers to the attached time stamp with the concentration level determined immediately before the current time. To read from the concentration storage area 632.
  • the concentration level determination unit 612 employs the read concentration level as the current concentration level (step S57).
  • the concentration degree determination unit 612 outputs information representing the adopted concentration degree to the operation mode switching control unit 613.
  • the concentration degree determination unit 612 stores the information indicating the adopted concentration degree in the concentration degree storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S58).
  • step S54 it is not essential to determine whether or not the number of samples necessary for determining the degree of concentration exists in step S54. For example, when it is determined in step S53 that there is data with a shielding flag, the concentration degree determination unit 612 may shift the process to step S57.
  • step S56 or step S58 when the information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612 determines whether or not the driving is finished (step S59). When the driving is finished (Yes in step S59), the concentration degree determination unit 612 ends the process. On the other hand, when the driving does not end (No in step S59), the concentration degree determination unit 612 shifts the process to step S51, and repeats the operations in steps S51 to S58 until the driving of the vehicle 1 ends.
  • the operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612. For example, when switching from the “automatic operation mode” to the “manual operation mode”, the operation mode switching control unit 613 informs the driver that the operation mode is to be switched and the period until the switching using an alarm generator (not shown). Notice.
  • the driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level during the switching period. The driving mode switching control unit 613 determines that the driver is ready to switch to the “manual driving mode” when the driver's concentration exceeds the preset concentration during the switching period. If it is determined that the driver is ready during the switching period, the driving mode switching control unit 613 switches the “automatic driving mode” to the “manual driving mode” after the switching period has elapsed. Is output to the automatic operation control device 5.
  • the operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 of the storage unit 63 in association with information indicating the determination timing, for example, time stamp information.
  • the determination using the degree of concentration is not limited to the above.
  • the countdown may be started based on the degree of concentration during the switching period for switching the operation mode.
  • the driving mode switching control unit 613 notifies the driver that the driving mode is to be switched, and then whether or not the driver is ready based on the concentration level output from the concentration level determination unit 612. Determine whether.
  • the operation mode switching control unit 613 determines that the driver is ready and starts counting down the period until the operation mode is switched.
  • the control unit 61 is configured such that the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. It is determined whether or not.
  • the control unit 61 sets the concentration level at the time when the shooting range of the driver camera 7 is shielded at the timing immediately before the shielding. The determined degree of concentration is adopted. For this reason, even when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving, etc. It becomes possible to carry out the determination without any problem.
  • the monitoring image acquisition control unit 611 is acquired when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Discriminate surveillance video.
  • the concentration level determination unit 612 determines the concentration level determined at the timing immediately before the shooting range of the driver camera 7 is blocked. 7 is adopted as the degree of concentration while the shooting range is shielded. Therefore, the monitoring video acquisition control unit 611 can efficiently sort the monitoring video acquired when the shooting range of the driver camera 7 is blocked.
  • the case where the monitoring video acquired by the monitoring video acquisition control unit 611 when the imaging range of the driver camera 7 is shielded by the support member of the steering wheel 4 has been described as an example.
  • the steering angle acquired by the steering sensor 8 may be used.
  • the concentration degree determination unit 612 is based on the steering angle acquired by the steering sensor 8, and the monitoring video is in a shielding state where the shooting range of the driver camera 7 is blocked by the steering wheel 4. You may determine with what was acquired.
  • the concentration degree determination unit 612 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62.
  • the concentration degree determination unit 612 determines whether or not the steering angle represented by the received steering angle data is included in a preset shielding angle width at a predetermined sampling rate.
  • the concentration degree determination unit 612 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
  • the concentration degree determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 from the monitoring video storage area 631 for a predetermined time.
  • the degree-of-concentration determination unit 612 determines the monitoring video data acquired in the shielding state from the read monitoring video data by matching the stored timing information with the read monitoring video data.
  • the concentration level determination unit 612 determines immediately before the current time. The degree of concentration is read from the concentration degree storage area 632. Then, the concentration level determination unit 612 employs the read concentration level as the current concentration level.
  • the concentration level determination unit 612 adopts the concentration level determined immediately before the current time as the current concentration level. May be.
  • the concentration degree determination unit 612 can select the monitoring video data acquired when the shooting range of the driver camera 7 is shielded by the steering wheel 4, and the processing of the monitoring video acquisition control unit 611 can be reduced. It becomes.
  • the operation mode switching control unit 613 determines that the shooting range of the driver camera 7 is blocked by the steering wheel 4 based on the steering angle acquired by the steering sensor 8. Good. At this time, the operation mode switching control unit 613 operates as follows, for example. In the following operation, the concentration level determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 every time a predetermined time elapses, and determines the concentration level based on the read monitoring video data. The concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632.
  • the operation mode switching control unit 613 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62. The operation mode switching control unit 613 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate. When the steering angle is included in the shielding angle width, the operation mode switching control unit 613 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
  • the operation mode switching control unit 613 determines whether or not the operation mode can be switched, the operation mode switching control unit 613 reads, from the concentration level storage area 632, information indicating the concentration level acquired by the concentration level determination unit 612 immediately before the determination time point. Based on the timing information stored in the storage unit 63, the operation mode switching control unit 613 determines whether or not the read concentration level is the concentration level acquired in the shielding state. When the read concentration degree is the concentration degree acquired in the shielding state, the operation mode switching control unit 613 reads information indicating the concentration degree acquired immediately before entering the shielding state from the concentration degree storage area 632. The operation mode switching control unit 613 determines whether or not the operation mode can be switched based on the degree of concentration represented by the newly read information.
  • the operation mode switching control unit 613 switches the operation mode using the degree of concentration acquired immediately before entering the shielding state. It becomes possible. For this reason, it is possible to reduce processing in the monitoring video acquisition control unit 611 and the concentration degree determination unit 612.
  • the concentration determined in the past when the photographing range of the driver camera 7 is shielded by the support member of the steering wheel 4 is used.
  • the case where it is adopted has been described as an example.
  • the second embodiment a case will be described in which the timing at which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 is predicted and handled.
  • FIG. 6 is a block diagram showing an example of a functional configuration of the operation mode switching control device 6B according to the second embodiment of the present invention.
  • the operation mode switching control device 6B includes a control unit 61B, an input / output interface unit 62, and a storage unit 63.
  • the control unit 61 by causing the CPU to execute a program stored in the program memory, a monitoring video acquisition control unit 611B and a concentration degree determination unit 612B, which are control functions necessary for implementing the second embodiment, are provided. Then, the operation mode switching control unit 613 and the shielding prediction unit 614 are realized.
  • the shielding prediction unit 614 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62.
  • the shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the acquired steering angle data.
  • the shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity.
  • the shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B when a time that is a predetermined period before the predicted start time of the shielding period comes.
  • the monitoring video acquisition control unit 611B takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62.
  • the monitoring video acquisition control unit 611B stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
  • the monitoring image acquisition control unit 611B synchronizes with the timing at which the operation signal is received, and stores the captured monitoring image data in the monitoring image storage area 631 of the storage unit 63.
  • the concentration determination unit 612B reads the monitoring video data stored during the time interval from the monitoring video storage area 631 at a preset time interval.
  • the concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data.
  • the concentration degree determination unit 612B determines the degree of concentration of the driver with respect to driving based on the recognized movement of the driver.
  • the concentration determination unit 612B receives the operation signal transmitted from the shielding prediction unit 614, the monitoring video data stored during a predetermined period from the time including the received timing is received from the monitoring video storage area 631. Read. The concentration level determination unit 612B determines the concentration level based on the read monitoring video data. The concentration level determination unit 612B outputs the determination result of the concentration level to the operation mode switching control unit 613. In addition, the concentration level determination unit 612B stores the determination result of the concentration level in the concentration level storage area 632 with a time stamp.
  • FIG. 7 is a flowchart illustrating an example of a procedure when the shielding prediction unit 614 of the control unit 61B provided in the operation mode switching control device 6B illustrated in FIG. 6 outputs an operation signal.
  • the steering sensor 8 when driving is started, the steering sensor 8 is activated.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle as a steering angle detection signal to the operation mode switching control device 6B.
  • the shielding prediction unit 614 receives the steering angle data for a predetermined period output from the steering sensor 8 and converted by the input / output interface unit 62 (step S71).
  • the shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the received steering angle data (step S72).
  • the shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity ( Step S73).
  • the shielding prediction unit 614 determines whether or not the current time is a time that is a time period (for example, s seconds) prior to the predicted start time of the shielding period (step S74). When it is a time before a predetermined period from the start time of the shielding period (Yes in step S74), the shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B ( Step S75).
  • a time period for example, s seconds
  • the shielding prediction unit 614 has finished driving the vehicle 1 It is determined whether or not (step S76).
  • the shielding prediction unit 614 ends the process when the driving is finished (Yes in Step S76).
  • the shielding prediction unit 614 shifts the process to step S71, and repeats the operations in steps S71 to S45 until the driving of the vehicle 1 is finished.
  • FIG. 8 is a flowchart showing an example of a procedure when the monitoring video acquisition control unit 611B of the control unit 61B shown in FIG. 6 stores the monitoring video data.
  • the driver camera 7 is activated.
  • the driver camera 7 continuously captures a predetermined range including the driver's face.
  • the driver camera 7 outputs the captured monitoring video as a video signal to the operation mode switching control device 6B.
  • the monitoring video acquisition control unit 611B receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S81).
  • the monitoring video acquisition control unit 611B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S82).
  • the monitoring video acquisition control unit 611B samples the received monitoring video data at a timing at which the operation signal is received, that is, at a timing different from a preset sampling rate.
  • the sampled monitoring video data is stored in the monitoring video storage area 631 (step S83).
  • the monitoring video acquisition control unit 611B samples the received monitoring video data at a preset sampling rate, and the sampled monitoring video data is stored in the monitoring video storage area. 631 (step S84).
  • the monitoring video acquisition control unit 611B determines whether or not the operation is finished (step S85). When the driving is finished (Yes in step S85), the monitoring video acquisition control unit 611B ends the process. On the other hand, when the driving does not end (No in step S85), the monitoring video acquisition control unit 611B shifts the process to step S81, and repeats the operations in steps S81 to S84 until the driving of the vehicle 1 ends.
  • FIG. 9 is a flowchart showing an example of a procedure when the concentration determination unit 612B of the control unit 61B shown in FIG. 6 determines the concentration of the driver.
  • the concentration degree determination unit 612B starts measuring time. After starting the time measurement, the concentration degree determination unit 612B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S91). When the operation signal is received (Yes in step S91), the concentration determination unit 612B receives the monitoring video data stored in the monitoring video storage area 631 before a predetermined period from the time including the timing at which the operation signal is received. Reading from the monitoring video storage area 631 (step S92).
  • the concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data.
  • the concentration level determination unit 612B determines the concentration level based on the movement of the driver recognized in the monitoring video data (step S93).
  • Concentration determination unit 612B outputs information indicating the determined concentration to operation mode switching control unit 613.
  • the concentration level determination unit 612B stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating determination timing, for example, time stamp information (step S94).
  • step S91 when the operation signal is not received, the concentration degree determination unit 612B determines whether or not a predetermined time has elapsed since the start of time measurement (step S95).
  • the concentration determination unit 612B reads the monitoring video data stored in the monitoring video storage area 631 during the predetermined time from the monitoring video storage area 631 (step S96). The process proceeds to step S93.
  • the concentration determination unit 612B may read a preset number of monitoring video data each time the monitoring video acquisition control unit 611B newly samples the monitoring video data, and may shift the process to step S93.
  • step S93 when information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612B determines whether or not the driving is finished (step S97). When the driving is finished (Yes in step S97), the concentration degree determination unit 612B ends the process. On the other hand, when the driving does not end (No in step S97), the concentration degree determination unit 612B shifts the process to step S91, and repeats the operations in steps S91 to S96 until the driving of the vehicle 1 ends.
  • the control unit 61B has the shooting range of the driver camera 7 shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Predict the period. Then, the control unit 61B determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, at the timing when the shooting range of the driver camera 7 is predicted to be shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving. However, it is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
  • the shielding prediction unit 614 outputs an operation signal when a time point that is a predetermined time period before the predicted start time of the shielding period is reached.
  • the monitoring video acquisition control unit 611B stores the monitoring video data captured at the timing when the operation signal is received.
  • the concentration level determination unit 612B determines the concentration level based on the monitoring video data acquired by the monitoring video acquisition control unit 611B at the timing at which the operation signal is received, and this concentration level is predicted as a shielding. The degree of concentration in the period is set.
  • the concentration determination unit 612B determines the concentration based on the monitoring video data acquired by the monitoring image acquisition control unit 611B at this timing when the operation signal is received.
  • the case where this concentration level is set as the concentration level in the predicted shielding period has been described as an example.
  • the concentration degree determination unit 612B may set the concentration degree determined immediately before receiving the operation signal in accordance with a preset period as the concentration degree in the predicted shielding period. As a result, the degree of concentration at the timing when the operation signal is received can be quickly calculated.
  • the driving mode switching control part 613 acquires the information showing a concentration degree when switching a driving mode similarly to 1st Embodiment, and a driver
  • the driving mode switching control unit 613 acquires information indicating the concentration level immediately before the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, and the driver's concentration level is set in advance based on the acquired information. You may make it determine whether it exceeds the concentration degree to be performed.
  • the operation mode switching control unit 613 when the operation mode switching control unit 613 receives an operation signal from the shielding prediction unit 614, the operation mode switching control unit 613 acquires information indicating the degree of concentration at the timing when the operation signal is received.
  • the information acquired here may be information indicating the degree of concentration determined at a preset timing, or information indicating the degree of concentration determined at the timing when the operation signal is received in the concentration degree determination unit 612. It may be.
  • the driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level based on the acquired information.
  • the operation mode switching control unit 613 stores the determination result in the storage unit 63.
  • the operation mode switching control unit 613 uses the determination result stored in the storage unit 63 to switch the operation mode. Judgment is made. As a result, the determination result can be stored in preparation for an operation mode switching instruction given when the driver camera 7 is shielded by the steering wheel 4.
  • the vehicle type, the function of the automatic operation control device, the control function and control procedure of the operation mode switching control device, and the control contents can be variously modified and implemented without departing from the gist of the present invention.
  • the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.
  • various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.
  • a driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
  • a memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
  • the at least one hardware processor comprises: Based on the monitoring video data stored in the memory, determine the concentration of the driver, Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, it is determined whether or not the imaging range of the monitoring camera is in a shielding state that is shielded by a support member of the steering wheel;
  • An operation mode switching control device configured to set the concentration degree determined immediately before entering the shielding state as the concentration degree in the shielding state in the shielding state.
  • a driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
  • a memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
  • At least one hardware processor connected to the memory, the at least one hardware processor comprising: Based on the monitoring video data stored in the memory, determine the concentration of the driver, Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel;
  • An operation mode switching control device configured to determine the degree of concentration in the predicted shielding period based on monitoring video data acquired before the shielding period.
  • the process is A determination process for determining whether or not the imaging range of the monitoring camera is in a shielded state that is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
  • An operation mode switching control method comprising: a determination process in which the concentration degree determined immediately before entering the shielding state is the concentration degree in the shielding state in the shielding state.
  • a driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver, Using at least one hardware processor to perform a step of determining the concentration of the driver based on a monitoring image captured by the monitoring camera;
  • the process is A prediction process for predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
  • An operation mode switching control method comprising: a determination process for determining the degree of concentration in the predicted shielding period based on a monitoring video acquired before the shielding period is reached.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

The present invention presents the following countermeasure, for the purpose of allowing a condition of a driver to be determined even if the capture range of a monitoring camera is blocked by a support member of a steering wheel. Provided is a control part that is in a driving mode switching control device, said driving mode switching control device being used in a vehicle in which a monitoring camera is positioned on a back side of a steering wheel, as viewed by a driver, and providing switching control of a driving mode of the vehicle, said control part determining the level of concentration of the driver on the basis of a monitoring video that is captured by the monitoring camera. On the basis of a steering angle that is detected by a steering sensor, said steering sensor detecting the steering angle of a steering wheel, the control part determines whether there is a blocked condition in which the capture range of the monitoring camera is blocked by a support member of the steering wheel, and if it determined that there is a blocked condition, treats the level of concentration that was determined immediately before the blocked condition as the level of concentration during the blocked condition.

Description

運転モード切替制御装置、方法、及びプログラムOperation mode switching control device, method, and program
　この発明は、車両の運転モードを手動運転モードと自動運転モードとの間で切り替えるための運転モード切替制御装置、方法およびプログラムに関する。 The present invention relates to an operation mode switching control device, method and program for switching a vehicle operation mode between a manual operation mode and an automatic operation mode.
　近年、車両の運転モードとして、運転者の運転操作に基づいて車両を走行させる手動運転モード以外に、運転者の運転操作によらず予め設定された経路に沿って車両を走行させる自動運転モードの開発が進められている。自動運転モードは、例えば、Global Positioning System（ＧＰＳ）を利用したナビゲーションシステムの情報、路車間通信により取得される交通情報、及び周辺の人や車両の位置と動きとを監視する周辺モニタリングシステムの情報をもとに、パワーユニット、操舵装置、及びブレーキ等を制御することで、車両の自動運転を可能にするものである（例えば、特開２０１５－１４１０５３号公報を参照）。 In recent years, as a driving mode of a vehicle, in addition to a manual driving mode in which the vehicle is driven based on a driving operation of the driver, an automatic driving mode in which the vehicle is driven along a predetermined route without depending on the driving operation of the driver. Development is underway. The automatic driving mode includes, for example, information on a navigation system using Global Positioning System (GPS), traffic information acquired by road-to-vehicle communication, and information on a surrounding monitoring system that monitors the positions and movements of surrounding people and vehicles. Based on the above, the vehicle can be automatically driven by controlling the power unit, the steering device, the brake, and the like (see, for example, Japanese Patent Laid-Open No. 2015-141053).
　ところで、自動運転モードは運転者の運転操作の負担軽減、及び交通渋滞の緩和等の効果が期待できるものの、走行開始から走行終了までのすべての運転操作を自動化することは現実的に困難である。このため、自動運転モードから手動運転モードへの切り替えを適切に行うことが求められている。運転モードを自動運転モードから手動運転モードに切り替える場合には、切り替えに先立ち運転者が手動運転操作可能な状態にあることを確認する必要がある。 By the way, although the automatic driving mode can be expected to reduce the burden on the driving operation of the driver and alleviate the traffic congestion, it is practically difficult to automate all the driving operations from the start of travel to the end of travel. . For this reason, it is required to appropriately switch from the automatic operation mode to the manual operation mode. When the operation mode is switched from the automatic operation mode to the manual operation mode, it is necessary to confirm that the driver is in a state where manual operation can be performed prior to the switching.
　車両の形状によっては、運転者の状態を監視する監視カメラが、運転者から見てステアリングホイールの裏側の、メータパネル付近に配置されていることがある。この場合、ステアリングホイールの角度によっては、ステアリングホイールの支持部材によりカメラの撮影範囲が遮られることになり、運転者の状態監視を継続することができない。運転者の状態が確認できないと、手動運転モードへ切り替えた後の運転操作を保証することができず、運転者の集中度が高い、本来なら自動運転モードから手動運転モードへの切り替えが可能な状態であっても、自動運転モードが継続されてしまう。以上のことから、ステアリングホイールの支持部材により監視カメラの撮影範囲が遮られる場合の適切な対策が求められていた。 Depending on the shape of the vehicle, a monitoring camera that monitors the driver's condition may be placed near the meter panel on the back of the steering wheel as viewed from the driver. In this case, depending on the angle of the steering wheel, the shooting range of the camera is blocked by the support member of the steering wheel, and the driver's state monitoring cannot be continued. If the driver's condition cannot be confirmed, the driving operation after switching to the manual operation mode cannot be guaranteed, and the driver's concentration is high. Originally, the automatic operation mode can be switched to the manual operation mode. Even in the state, the automatic operation mode is continued. In view of the above, there has been a demand for appropriate measures when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
　この発明は、ステアリングホイールの支持部材により監視カメラの撮影範囲が遮られる場合でも、運転者の状態を判定可能な運転モード切替制御装置、方法およびプログラムを提供しようとするものである。 This invention is intended to provide an operation mode switching control device, method and program capable of determining the state of the driver even when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
　上記課題を解決するためにこの発明の第１の実施形態に係る第１の態様は、監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置にあって、前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する制御部を具備する。そして、前記制御部は、前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽されている遮蔽状態か否かを判定し、前記遮蔽状態であると判定した場合、前記遮蔽状態となる直前に判定された集中度を、前記遮蔽状態における集中度としたものである。 In order to solve the above problems, a first aspect according to the first embodiment of the present invention is a vehicle driving mode used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel as viewed from a driver. An operation mode switching control apparatus that performs switching control includes a control unit that determines a concentration degree of the driver based on a monitoring image acquired by the monitoring camera. The control unit determines whether or not the shooting range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel. Is determined, and the concentration degree determined immediately before entering the shielding state is defined as the concentration degree in the shielding state.
　この発明の第１の実施形態に係る第１の態様によれば、制御部は、ステアリングセンサで検出される操舵角に基づき、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されているか否かを判定する。そして、制御部は、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されている場合には、監視カメラの撮影範囲が遮蔽された時点の集中度として、遮蔽直前のタイミングに判定された集中度を採用するようにしている。このため、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されたタイミングで、例えば、自動運転から手動運転への切替可否の判定等で集中度を利用したい場合であっても、滞りなく判定を実施することが可能となる。 According to the first aspect of the first embodiment of the present invention, the control unit determines whether the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Determine whether. Then, when the imaging range of the monitoring camera is shielded by the support member of the steering wheel, the control unit determines the concentration determined at the timing immediately before the shielding as the concentration level when the imaging range of the monitoring camera is shielded. The degree is adopted. For this reason, even when it is desired to use the degree of concentration at the timing when the photographing range of the surveillance camera is shielded by the support member of the steering wheel, for example, whether to switch from automatic driving to manual driving, etc. Can be carried out.
　この発明の第１の実施形態に係る第２の態様は、前記制御部が、前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得し、前記取得した監視映像データのうち、前記遮蔽状態において取得された監視映像データを、前記ステアリングセンサで検出される操舵角に基づいて判別する監視映像取得制御部と、前記遮蔽状態における集中度を、前記遮蔽状態となる直前の非遮蔽期間に取得された監視映像データに基づいて判定する集中度判定部とを備えるようにしたものである。 According to a second aspect of the first embodiment of the present invention, the control unit acquires monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera, and includes the acquired monitoring video data A monitoring image acquisition control unit that determines monitoring image data acquired in the shielding state based on a steering angle detected by the steering sensor, and a concentration level in the shielding state is determined immediately before the shielding state is entered. And a concentration degree determination unit for determining based on the monitoring video data acquired during the shielding period.
　この発明の第１の実施形態に係る第２の態様によれば、監視映像取得制御部は、ステアリングセンサで検出される操舵角に基づき、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽された際に取得された監視映像を判別する。集中度判定部は、監視カメラの撮影範囲が遮蔽された際に取得された監視映像がある場合、監視カメラの撮影範囲が遮蔽される直前のタイミングで判定した集中度を、監視カメラの撮影範囲が遮蔽されている間の集中度として採用するようにしている。このため、監視映像取得制御部で、監視カメラの撮影範囲が遮蔽された際に取得された監視映像を効率的に選別できるようになる。 According to the second aspect of the first embodiment of the present invention, the monitoring video acquisition control unit is configured to shield the photographing range of the monitoring camera by the steering wheel support member based on the steering angle detected by the steering sensor. The monitoring video acquired at the time is determined. When there is a monitoring image acquired when the shooting range of the monitoring camera is shielded, the concentration level determination unit determines the concentration degree determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the monitoring video acquisition control unit can efficiently sort the monitoring video acquired when the imaging range of the monitoring camera is blocked.
　この発明の第１の実施形態に係る第３の態様は、前記制御部が、前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、前記監視映像取得制御部により取得された監視映像データのうち、前記遮蔽状態において取得された監視映像データを、前記ステアリングセンサで検出される操舵角に基づいて判別し、前記遮蔽状態における集中度を、前記遮蔽状態となる直前の非遮蔽期間に取得された監視映像データに基づいて判定する集中度判定部とを備えるようにしたものである。 According to a third aspect of the first embodiment of the present invention, the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the monitoring Among the monitoring video data acquired by the video acquisition control unit, the monitoring video data acquired in the shielding state is determined based on the steering angle detected by the steering sensor, and the degree of concentration in the shielding state is determined by A concentration degree determination unit for determining based on the monitoring video data acquired in the non-shielding period immediately before the shielding state.
　この発明の第１の実施形態に係る第３の態様によれば、集中度判定部は、ステアリングセンサで検出される操舵角に基づき、ステアリングホイールによる監視カメラの遮蔽時に取得された監視映像データを判別する。集中度判定部は、ステアリングホイールによる監視カメラの遮蔽時に取得された監視映像がある場合、監視カメラの撮影範囲が遮蔽された時点の直前のタイミングに判定された集中度を、監視カメラの撮影範囲が遮蔽されている間の集中度として採用するようにしている。このため、集中度判定部で、監視カメラの撮影範囲がステアリングホイールにより遮蔽された際に取得された監視映像データを選別でき、監視映像取得制御部の処理を低減させることが可能となる。 According to the third aspect of the first embodiment of the present invention, the concentration degree determination unit uses the monitoring video data acquired when the monitoring camera is shielded by the steering wheel based on the steering angle detected by the steering sensor. Determine. When there is a monitoring image acquired when the surveillance camera is blocked by the steering wheel, the concentration degree determination unit indicates the degree of concentration determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the concentration level determination unit can select the monitoring video data acquired when the imaging range of the monitoring camera is shielded by the steering wheel, and the processing of the monitoring video acquisition control unit can be reduced.
　この発明の第１の実施形態に係る第４の態様は、前記制御部が、前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、前記取得された監視映像データに基づいて前記運転者の集中度を判定する集中度判定部と、前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽状態を判別し、前記遮蔽状態となる直前に前記集中度判定部により判定された集中度を、前記遮蔽状態における集中度として運転モードの切り替え可否を判定する運転モード切替制御部とを備えるようにしたものである。 According to a fourth aspect of the first embodiment of the present invention, the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the acquisition A degree-of-concentration determination unit that determines the degree of concentration of the driver based on the monitored video data, and determines the shielding state based on a steering angle detected by the steering sensor, and immediately before entering the shielding state, An operation mode switching control unit that determines whether or not the operation mode can be switched is used as the concentration level determined by the concentration level determination unit as the concentration level in the shielding state.
　この発明の第１の実施形態に係る第４の態様によれば、監視映像取得制御部は、監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する。集中度判定部は、取得された監視映像データに基づき、運転者の集中度を判定する。運転モード切替制御部は、ステアリングセンサで検出される操舵角に基づいて監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されたタイミングを判別する。そして、運転モード切替制御部は、遮蔽状態となる直前に判定された集中度を、遮蔽状態において運転モードの切替可否を判定する際の集中度として採用するようにしている。このため、監視映像取得制御部、及び集中度判定部での処理を低減させることが可能となる。 According to the fourth aspect of the first embodiment of the present invention, the monitoring video acquisition control unit acquires monitoring video data from the monitoring video acquired by the monitoring camera at a predetermined cycle. The concentration level determination unit determines the driver's concentration level based on the acquired monitoring video data. The operation mode switching control unit determines the timing at which the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Then, the operation mode switching control unit adopts the concentration determined immediately before entering the shielding state as the concentration when determining whether the operation mode can be switched in the shielding state. For this reason, it is possible to reduce the processing in the monitoring video acquisition control unit and the concentration degree determination unit.
　この発明の第２の実施形態に係る第１の態様は、監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置にあって、前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する制御部を具備する。そして、前記制御部は、前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽される遮蔽期間を予測し、前記予測した遮蔽期間における集中度を、前記遮蔽期間となる前の非遮蔽期間に取得された監視映像に基づいて判定するようにしたものである。 A first aspect according to the second embodiment of the present invention is an operation mode switching control for switching a vehicle operation mode, which is used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel when viewed from the driver. The apparatus includes a control unit that determines the degree of concentration of the driver based on a monitoring image acquired by the monitoring camera. The control unit predicts a shielding period in which the imaging range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel. The degree of concentration in the predicted shielding period is determined based on the monitoring video acquired in the non-shielding period before the shielding period.
　この発明の第２の実施形態に係る第１の態様によれば、制御部は、ステアリングセンサで検出される操舵角に基づき、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽される期間を予測する。そして、制御部は、予測した期間における運転者の集中度を、遮蔽期間となる前に取得した監視映像データに基づいて判定するようにしている。このため、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されると予測されるタイミングで、例えば、自動運転から手動運転への切替可否の判定等で集中度を利用したい場合であっても、遮蔽状態となると予測される直前に取得された集中度を利用して切替可否の判定が可能となる。 According to the first aspect of the second embodiment of the present invention, the control unit determines the period during which the imaging range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Predict. Then, the control unit determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, even if it is desired to use the degree of concentration at the timing when the shooting range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, for example, whether or not switching from automatic driving to manual driving is possible. It is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
　この発明の第２の実施形態に係る第２の態様は、前記制御部が、前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽期間を予測し、前記遮蔽期間の開始時点から予め設定した期間だけ前の時点において動作信号を出力する遮蔽予測部と、前記動作信号に応じたタイミングで、前記監視カメラにより取得される監視映像から監視映像データを取得する監視映像取得制御部と、前記動作信号に応じたタイミングで、前記取得された監視映像データに基づいて前記運転者の集中度を判定し、当該集中度を前記予測した遮蔽期間における集中度とする集中度判定部とを備えるようにしたものである。 According to a second aspect of the second embodiment of the present invention, the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from the start point of the shielding period. A shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring image acquisition control unit that acquires monitoring image data from a monitoring image acquired by the monitoring camera at a timing according to the operation signal, and the operation A concentration level determination unit that determines the concentration level of the driver based on the acquired monitoring video data at a timing according to a signal and sets the concentration level as the concentration level in the predicted shielding period. It is a thing.
　この発明の第２の実施形態に係る第２の態様によれば、遮蔽予測部は、ステアリングセンサで検出される操舵角に基づいて遮蔽期間を予測し、予測した遮蔽期間の開始時点から予め設定した期間だけ前の時点になると動作信号を出力する。監視映像取得制御部は、動作信号を受信したタイミングで、監視カメラにより取得される監視映像から監視映像データを記憶する。そして、集中度判定部は、動作信号を受信したタイミングで、監視映像取得制御部により記憶された監視映像データに基づいて集中度を判定し、この集中度を、予測した遮蔽期間における集中度とするようにしている。このため、監視カメラの撮影範囲がステアリングホイールの支持部材により遮蔽されると予測されるタイミングで運転モードの切替可否を判定したい場合であっても、遮蔽状態となると予測される直前の、予め設定したサンプリングレートとは異なるタイミングで取得された集中度を利用することが可能となる。 According to the second aspect of the second embodiment of the present invention, the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and is preset from the predicted start time of the shielding period. An operation signal is output when the previous time is reached. The monitoring video acquisition control unit stores monitoring video data from the monitoring video acquired by the monitoring camera at the timing of receiving the operation signal. Then, the concentration level determination unit determines the concentration level based on the monitoring video data stored by the monitoring video acquisition control unit at the timing when the operation signal is received, and the concentration level is determined as the concentration level in the predicted shielding period. Like to do. For this reason, even when it is desired to determine whether or not the operation mode can be switched at a timing at which the imaging range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, it is set in advance immediately before it is predicted to enter the shielding state It is possible to use the degree of concentration acquired at a timing different from the sampling rate.
　この発明の第２の実施形態に係る第３の態様は、前記制御部が、前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽期間を予測し、前記遮蔽期間の開始時点から予め設定した期間だけ前の時点において動作信号を出力する遮蔽予測部と、前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、前記取得された監視映像データに基づいて所定の周期で前記運転者の集中度を判定し、前記動作信号を受信すると、前記動作信号を受信する直前に判定した集中度を、前記予測した遮蔽期間における集中度とする集中度判定部とを備えるようにしたものである。 According to a third aspect of the second embodiment of the present invention, the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from a start time of the shielding period. A shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring video acquisition control unit that acquires monitoring video data in a predetermined cycle from the monitoring video acquired by the monitoring camera, and the acquired monitoring video data When the driver's concentration level is determined at a predetermined period based on the control signal and the operation signal is received, the concentration level determined immediately before receiving the operation signal is set as the concentration level in the predicted shielding period. And a determination unit.
　この発明の第２の実施形態に係る第３の態様によれば、遮蔽予測部は、ステアリングセンサで検出される操舵角に基づいて遮蔽期間を予測し、予測した遮蔽期間の開始時点から予め設定した期間だけ前の時点において動作信号を出力する。監視映像取得制御部は、監視カメラにより取得される監視映像から予め設定されたサンプリングレートに従って監視映像データをサンプリングする。集中度判定部は、監視映像取得制御部でサンプリングされた監視映像データに基づき、予め設定された周期で、運転者の集中度を判定する。そして、集中度判定部は、動作信号を受信すると、予測された遮蔽期間における集中度を、動作信号を受信した直前に判定した集中度で代用するようにしている。このため、動作信号を受信したタイミングでの集中度を、迅速に算出することが可能となる。 According to the third aspect of the second embodiment of the present invention, the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and presets it from the start time of the predicted shielding period. An operation signal is output at a time point before the period. The monitoring video acquisition control unit samples monitoring video data from the monitoring video acquired by the monitoring camera according to a preset sampling rate. The concentration level determination unit determines the concentration level of the driver at a preset cycle based on the monitoring video data sampled by the monitoring video acquisition control unit. Then, when receiving the operation signal, the concentration determination unit substitutes the concentration determined in the predicted shielding period with the concentration determined immediately before receiving the operation signal. For this reason, it is possible to quickly calculate the degree of concentration at the timing when the operation signal is received.
　この発明の第２の実施形態に係る第４の態様は、前記制御部が、前記動作信号を受信すると、前記集中度判定部で判定された集中度に基づいて運転モードの切り替え可否を判定し、当該判定結果を、前記遮蔽期間における判定結果とする運転モード切替制御部をさらに備えるようにしたものである。 According to a fourth aspect of the second embodiment of the present invention, when the control unit receives the operation signal, the control unit determines whether or not the operation mode can be switched based on the concentration level determined by the concentration level determination unit. The operation mode switching control unit is further provided with the determination result as the determination result in the shielding period.
　この発明の第２の実施形態に係る第４の態様によれば、運転モード切替制御部は、遮蔽予測部から出力される動作信号を受信すると、集中度判定部で判定された集中度を表す情報を取得する。運転モード切替制御部は、取得した情報に基づき、運転モードの切替可否を判定する。そして、運転モード切替制御部は、この判定結果を、予測された遮蔽期間における判定結果とするようにしている。このため、運転モード切替制御部は、監視カメラがステアリングホイールによって遮蔽されている際に運転モードの切替指示が与えられた場合、遮蔽状態になる直前に判定した判定結果を利用することが可能となる。すなわち、監視カメラがステアリングホイールによって遮蔽されている際に与えられる、運転モードの切替指示に備えることが可能となる。 According to the fourth aspect of the second embodiment of the present invention, when the operation mode switching control unit receives the operation signal output from the shielding prediction unit, the operation mode switching control unit represents the concentration level determined by the concentration level determination unit. Get information. The operation mode switching control unit determines whether or not the operation mode can be switched based on the acquired information. The operation mode switching control unit sets the determination result as the determination result in the predicted shielding period. For this reason, when the operation mode switching instruction is given when the surveillance camera is shielded by the steering wheel, the operation mode switching control unit can use the determination result determined immediately before entering the shielding state. Become. That is, it is possible to prepare for an operation mode switching instruction given when the surveillance camera is shielded by the steering wheel.
　すなわち、この発明の各態様によれば、ステアリングホイールの支持部材により監視カメラの撮影範囲が遮られる場合でも、運転者の状態を判定可能な運転モード切替制御装置、方法およびプログラムを提供することができる。 That is, according to each aspect of the present invention, it is possible to provide an operation mode switching control device, method, and program capable of determining the state of the driver even when the photographing range of the monitoring camera is blocked by the support member of the steering wheel. it can.
図１は、この発明の第１の実施形態に係る運転モード切替制御装置を備えた自動運転制御システムの全体構成の例を示す図である。FIG. 1 is a diagram showing an example of the overall configuration of an automatic operation control system including an operation mode switching control device according to the first embodiment of the present invention. 図２は、この発明の第１の実施形態に係る運転モード切替制御装置の機能構成を示すブロック図である。FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to the first embodiment of the present invention. 図３は、図１に示されるドライバカメラの設置位置を表す図である。FIG. 3 is a diagram showing the installation position of the driver camera shown in FIG. 図４は、図２に示される監視映像取得制御部が監視映像データを記憶する際の手順を示すフローチャートである。FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 2 stores the monitoring video data. 図５は、図２に示される集中度判定部がドライバの集中度を判定する際の手順を示すフローチャートである。FIG. 5 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 2 determines the concentration level of the driver. 図６は、この発明の第２の実施形態に係る運転モード切替制御装置の機能構成を示すブロック図である。FIG. 6 is a block diagram showing a functional configuration of an operation mode switching control apparatus according to the second embodiment of the present invention. 図７は、図６に示される遮蔽予測部が動作信号を出力する際の手順を示すフローチャートである。FIG. 7 is a flowchart illustrating a procedure when the shielding prediction unit illustrated in FIG. 6 outputs an operation signal. 図８は、図６に示される監視映像取得制御部が監視映像データを記憶する際の手順を示すフローチャートである。FIG. 8 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 6 stores the monitoring video data. 図９は、図６に示される集中度判定部がドライバの集中度を判定する際の手順を示すフローチャートである。FIG. 9 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 6 determines the concentration level of the driver.
実施形態Embodiment
　以下、図面を参照してこの発明に係わる実施形態を説明する。　
　［第１の実施形態］
　図１は、この発明の第１の実施形態に係る運転モード切替制御装置を備えた自動運転制御システムの全体構成の例を示す図で、この自動運転制御システムは乗用車等の車両１に搭載される。 Embodiments according to the present invention will be described below with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram showing an example of the overall configuration of an automatic driving control system provided with an operating mode switching control device according to a first embodiment of the present invention. The automatic driving control system is mounted on a vehicle 1 such as a passenger car. The
　車両１は、基本設備として、動力源、及び変速装置を含むパワーユニット２と、ステアリングホイール４が装備された操舵装置３とを備え、さらに運転モードとしては手動運転モードと自動運転モードとを備えている。動力源としては、エンジンまたはモータ、あるいはその両方が用いられる。 The vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. Yes. An engine and / or a motor is used as the power source.
　手動運転モードは、例えば、運転者の手動による運転操作を主体として車両１を走行させるモードである。手動運転モードには、例えば、運転者の運転操作のみに基づいて車両を走行させる動作モードと、運転者の運転操作を主体としながら運転者の運転操作を支援する運転操作支援制御を行う動作モードが含まれる。 The manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example. The manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.
　運転操作支援制御は、例えば、車両１のカーブ走行時にカーブの曲率に基づいて運転者の操舵が適切な操舵量となるように操舵トルクをアシストする。また運転操作支援制御には、運転者のアクセル操作（例えばアクセルペダルの操作）またはブレーキ操作（例えばブレーキペダルの操作）を支援する制御と、手動操舵（操舵の手動運転）および手動速度調整（速度調整の手動運転）も含まれる。手動操舵は、運転者のステアリングホイール４の操作を主体として車両１の操舵を行う。手動速度調整は、運転者のアクセル操作又はブレーキ操作を主体として車両の速度調整を行う。 The driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example. The driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included. In manual steering, the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4. In the manual speed adjustment, the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.
　なお、運転操作支援制御には、運転者の運転操作に強制的に介入して、車両を自動走行させる制御は含まれない。すなわち、手動運転モードには、予め設定された許容範囲において運転者の運転操作を車両の走行に反映させるが、一定条件（例えば車両の車線逸脱等）の下で車両の走行に強制的に介入する制御は含まれない。 Note that the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle. In other words, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.
　自動運転モードは、例えば、車両の走行する道路に沿って自動で車両を走行させる運転状態を実現するモードである。自動運転モードには、例えば、運転者が運転操作をすることなく、予め設定された目的地に向かって自動的に車両を走行させる運転状態が含まれる。自動運転モードは、必ずしも車両の全ての制御を自動で行う必要はなく、予め設定された許容範囲において運転者の運転操作を車両の走行に反映する運転状態も自動運転モードに含まれる。すなわち、自動運転モードには、予め設定された許容範囲において運転者の運転操作を車両の走行に反映させるが、一定条件の下で車両の走行に強制的に介入する制御が含まれる。 The automatic driving mode is, for example, a mode that realizes an operating state in which the vehicle automatically travels along the road on which the vehicle travels. The automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver. In the automatic driving mode, it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.
　図１に示される自動運転制御システムは、自動運転モードによる運転制御を実行するための自動運転制御装置５を有している。自動運転制御装置５は、ステアリングセンサ８、アクセルペダルセンサ９、ブレーキペダルセンサ１０、ＧＰＳ受信機１１、ジャイロセンサ１２、及び車速センサ１３からそれぞれセンシングデータを取得する。そして、これらのセンシングデータと、図示しないナビゲーションシステムで生成される経路情報、路車間通信により取得される交通情報、及び周辺の人や車両の位置と動きとを監視する周辺モニタリングシステムにより得られる情報をもとに、車両１の走行を自動制御する。 The automatic operation control system shown in FIG. 1 has an automatic operation control device 5 for executing operation control in the automatic operation mode. The automatic driving control device 5 acquires sensing data from the steering sensor 8, the accelerator pedal sensor 9, the brake pedal sensor 10, the GPS receiver 11, the gyro sensor 12, and the vehicle speed sensor 13, respectively. And these sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and information obtained by a peripheral monitoring system that monitors the positions and movements of surrounding people and vehicles Based on the above, the traveling of the vehicle 1 is automatically controlled.
　自動制御には、例えば、自動操舵（操舵の自動運転）と自動速度調整（速度の自動運転）とがある。自動操舵は、操舵装置３を自動で制御する運転状態である。自動操舵にはLane Keeping Assist（ＬＫＡ）が含まれる。ＬＫＡは、例えば、運転者がステアリング操作をしない場合であっても、車両１が走行車線から逸脱しないように自動で操舵装置３を制御する。なお、ＬＫＡの実行中であっても、車両１が走行車線を逸脱しない範囲（許容範囲）において運転者のステアリング操作を車両の操舵に反映してもよい。なお、自動操舵はＬＫＡに限らない。 Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed). Automatic steering is an operating state in which the steering device 3 is automatically controlled. Automatic steering includes Lane Keeping Assist (LKA). For example, the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation. Even when LKA is being executed, the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range). Note that automatic steering is not limited to LKA.
　自動速度調整は、車両１の速度を自動で制御する運転状態である。自動速度調整にはAdaptive Cruise Control（ＡＣＣ）が含まれる。ＡＣＣとは、例えば、車両１の前方に先行車が存在しない場合は予め設定された設定速度で車両１を定速走行させる定速制御を行い、車両１の前方に先行車が存在する場合には先行車との車間距離に応じて車両１の車速を調整する追従制御を行うものである。自動運転制御装置５は、ＡＣＣを実行中であっても、運転者のブレーキ操作（例えばブレーキペダルの操作）に応じて車両１を減速させる。また自動運転制御装置５は、ＡＣＣを実行中であっても、予め設定された最大許容速度（例えば走行中の道路において法的に定められた最高速度）まで、運転者のアクセル操作（例えばアクセルペダルの操作）に応じて車両を加速させることもできる。なお、自動速度調整は、ＡＣＣに限らず、定速制御のみ行うCruise Control（ＣＣ）等も含まれる。 Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled. Automatic speed adjustment includes Adaptive Cruise Control (ACC). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle. The automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed. In addition, the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC. The vehicle can be accelerated according to the pedal operation. The automatic speed adjustment is not limited to ACC, but also includes Cruise Control (CC) that performs only constant speed control.
　また、図１に示される自動運転制御システムは、手動運転モードと自動運転モードとの間で設定を切り替えるための装置として、運転モード切替制御装置６と、ドライバカメラ７と、ステアリングセンサ８とを備えている。 The automatic operation control system shown in FIG. 1 includes an operation mode switching control device 6, a driver camera 7, and a steering sensor 8 as devices for switching the setting between the manual operation mode and the automatic operation mode. I have.
　ドライバカメラ７は、例えば、運転者から見てステアリングホイールの裏側の、メータパネル付近に配置されている。ドライバカメラ７は、運転者を撮像して運転者についての監視映像を取得する。ドライバカメラ７は、取得した監視映像を映像信号として運転モード切替制御装置６へ出力する。ステアリングセンサ８は、ステアリングホイール４の操舵角を検出する。ステアリングセンサ８は、検出した操舵角を操舵角検出信号として運転モード切替制御装置６へ出力する。 The driver camera 7 is disposed, for example, near the meter panel on the back side of the steering wheel as viewed from the driver. The driver camera 7 captures the driver and obtains a monitoring video about the driver. The driver camera 7 outputs the acquired monitoring video as a video signal to the operation mode switching control device 6. The steering sensor 8 detects the steering angle of the steering wheel 4. The steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
　図２は、第１の実施形態に係る運転モード切替制御装置６の機能構成の例を示すブロック図である。図２に示される運転モード切替制御装置６は、自動運転制御装置５に対し、手動運転モードと自動運転モードとの間の切り替えを統括的に制御する装置である。 FIG. 2 is a block diagram illustrating an example of a functional configuration of the operation mode switching control device 6 according to the first embodiment. The operation mode switching control device 6 shown in FIG. 2 is a device that comprehensively controls the automatic operation control device 5 to switch between the manual operation mode and the automatic operation mode.
　図２によれば、運転モード切替制御装置６は、制御ユニット６１と、入出力インタフェースユニット６２と、記憶ユニット６３とを備えている。 2, the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.
　入出力インタフェースユニット６２は、ドライバカメラ７から出力される映像信号、及びステアリングセンサ８から出力される操舵角検出信号を受信する。入出力インタフェースユニットは、受信した映像信号、及び操舵角検出信号をディジタルデータに変換する。また、入出力インタフェースユニット６２は、制御ユニット６１から出力される運転モード切替制御信号を自動運転制御装置５へ出力する。 The input / output interface unit 62 receives the video signal output from the driver camera 7 and the steering angle detection signal output from the steering sensor 8. The input / output interface unit converts the received video signal and steering angle detection signal into digital data. Further, the input / output interface unit 62 outputs an operation mode switching control signal output from the control unit 61 to the automatic operation control device 5.
　記憶ユニット６３は、記憶媒体として、例えばSolid State Drive（ＳＳＤ）やHard Disk Drive（ＨＤＤ）等の随時書き込み、及び読み出しが可能な不揮発性メモリ、並びに、Random Access Memory（ＲＡＭ）のような揮発性メモリ等により実現される。本実施形態を実施するために使用する記憶領域として、監視映像記憶領域６３１と、集中度記憶領域６３２と、履歴記憶領域６３３とを備えている。 The storage unit 63 is a non-volatile memory that can be written and read as needed, such as Solid State Drive (SSD) and Hard Disk Drive (HDD), and volatile like Random Access Memory (RAM). This is realized by a memory or the like. As storage areas used for carrying out the present embodiment, a surveillance video storage area 631, a concentration degree storage area 632, and a history storage area 633 are provided.
　制御ユニット６１は、コンピュータを構成するCentral Processing Unit（ＣＰＵ）、及びプログラムメモリを有する。制御ユニット６１では、プログラムメモリに格納されたプログラムをＣＰＵに実行させることにより、第１の実施形態を実施するために必要な制御機能である、監視映像取得制御部６１１と、集中度判定部６１２と、運転モード切替制御部６１３とが実現される。 The control unit 61 includes a central processing unit (CPU) that constitutes a computer, and a program memory. The control unit 61 causes the CPU to execute a program stored in the program memory so that a monitoring video acquisition control unit 611 and a concentration degree determination unit 612 are control functions necessary for implementing the first embodiment. And the operation mode switching control part 613 is implement | achieved.
　監視映像取得制御部６１１は、ドライバカメラ７から出力されたドライバの映像信号についてのディジタルデータ（監視映像データ）を入出力インタフェースユニット６２から取り込む。監視映像取得制御部６１１は、取り込んだ監視映像データを、所定のサンプリングレートで記憶ユニット６３の監視映像記憶領域６３１に記憶させる。 The monitoring video acquisition control unit 611 takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62. The monitoring video acquisition control unit 611 stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
　また、監視映像取得制御部６１１は、ステアリングセンサ８から出力された操舵角検出信号についてのディジタルデータ（操舵角データ）を入出力インタフェースユニット６２から取り込む。監視映像取得制御部６１１は、取り込んだ操舵角データにより表される操舵角が遮蔽角度幅内に含まれるか否かに基づき、ドライバカメラ７が遮蔽状態にあるか否かを判定する。 Also, the monitoring image acquisition control unit 611 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62. The monitoring image acquisition control unit 611 determines whether or not the driver camera 7 is in the shielding state based on whether or not the steering angle represented by the captured steering angle data is included in the shielding angle width.
　図３は、図１に示されるドライバカメラ７の設置位置の例を表す模式図である。ドライバカメラ７は、運転者から見てステアリングホイール４の裏側から運転者を撮像する。そのため、ステアリングホイール４の操舵角によっては、ステアリングホイール４の支持部材４１－１～４１－３によって撮影範囲が遮られることになる。本実施形態において、遮蔽角度幅とは、ドライバカメラ７の撮影範囲が支持部材４１－１～４１－３によって遮られる操舵角の幅を表す。遮蔽角度幅は、例えば、車両１に搭載されるステアリングホイール４の形状に基づいて予め設定されている。 FIG. 3 is a schematic diagram showing an example of an installation position of the driver camera 7 shown in FIG. The driver camera 7 images the driver from the back side of the steering wheel 4 as viewed from the driver. For this reason, depending on the steering angle of the steering wheel 4, the photographing range is blocked by the support members 41-1 to 41-3 of the steering wheel 4. In the present embodiment, the shielding angle width represents the width of the steering angle at which the photographing range of the driver camera 7 is shielded by the support members 41-1 to 41-3. The shielding angle width is set in advance based on, for example, the shape of the steering wheel 4 mounted on the vehicle 1.
　監視映像取得制御部６１１は、操舵角が遮蔽角度幅内に含まれていると判定した場合、操舵角が遮蔽角度幅内に含まれている間にサンプリングした監視映像データを、遮蔽フラグを付して監視映像記憶領域６３１に記憶させる。本実施形態において、遮蔽フラグは、操舵角が遮蔽角度幅内に含まれている間にサンプリングされた監視映像データであることを表すフラグである。 When the monitoring image acquisition control unit 611 determines that the steering angle is included in the shielding angle width, the monitoring image acquisition control unit 611 attaches a shielding flag to the monitoring image data sampled while the steering angle is included in the shielding angle width. And stored in the surveillance video storage area 631. In the present embodiment, the shielding flag is a flag representing monitoring video data sampled while the steering angle is included within the shielding angle width.
　集中度判定部６１２は、予め設定された時間間隔で、この時間間隔の間に記憶された監視映像データを、監視映像記憶領域６３１から読み込む。集中度判定部６１２は、読み込んだ監視映像データに基づき、運転者の動き、及び／又は運転者の表情を認識する。集中度判定部６１２は、認識した運転者の動き、及び／又は運転者の表情を、予め定義されている運転者の動き、及び表情に関する判定基準に照合させ、運転者の運転に対する集中度を判定する。集中度判定部６１２は、集中度の判定結果を運転モード切替制御部６１３へ出力する。また、集中度判定部６１２は、集中度の判定結果を、タイムスタンプを付して集中度記憶領域６３２に記憶させる。 The concentration degree determination unit 612 reads the monitoring video data stored during this time interval from the monitoring video storage area 631 at a preset time interval. The concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data. The degree-of-concentration determination unit 612 collates the recognized driver movement and / or driver's facial expression with a predetermined criterion for determining the driver's movement and facial expression, and determines the driver's degree of concentration with respect to driving. judge. The concentration level determination unit 612 outputs the determination result of the concentration level to the operation mode switching control unit 613. Further, the concentration degree determination unit 612 stores the determination result of the concentration degree in the concentration degree storage area 632 with a time stamp.
　本実施形態において、集中度とは、運転者が運転にどれだけ適しているかを示す指標である。例えば、車両１を普通に運転している状態であれば、運転者の動きとしては、前方注視と、バックミラー、又はサイドミラーを見るための所定方向への注視等とが検出される。例えば、検出される運転者の動作が、前方注視と、所定方向への注視とでほぼ占められているような状態は、運転者が運転に十分に集中しているとみなされる。その逆に、例えば運転者が運転中、あまり前方を注視していなかったり、話をしていたり、眠そうであったり、口に食べ物を運んでいたり、あるいはあってはならないことであるが携帯電話、又はスマートフォンを操作していたり、通話しているような場合には、運転者は運転にあまり集中していないとみなされる。また、運転者の表情としては、例えば、運転者の眼の開眼の状態、まばたきの頻度、又は眼球運動等が検出される。 In the present embodiment, the degree of concentration is an index indicating how suitable the driver is for driving. For example, if the vehicle 1 is in a normal driving state, the driver's movements are detected as forward gaze and gaze in a predetermined direction for viewing the rearview mirror or side mirror. For example, a state in which the detected driver's motion is almost occupied by forward gaze and gaze in a predetermined direction is considered that the driver is sufficiently concentrated on driving. On the other hand, for example, while the driver is driving, he / she should not be looking closely ahead, talking, sleepy, carrying food in his mouth, or should not be mobile When operating a telephone or a smartphone or making a phone call, it is considered that the driver is not concentrated on driving. Further, as the driver's facial expression, for example, the eye open state of the driver's eyes, the frequency of blinking, or eye movements are detected.
　判定基準では、例えば、運転者の動きの例えば８０％以上が前方注視、又は所定方向への注視である場合、集中度が「高」と定義されている。また、判定基準では、例えば、運転者の動きの例えば２０％以上が前方注視、又は所定方向への注視であったり、運転者の動きに携帯電話、又はスマートフォンの操作、並びに通話の様子が含まれている場合、集中度が「低」と定義されている。集中度判定部６１２は、監視映像記憶領域６３１に記憶された監視映像データから認識した運転者の動きを、判定基準を参照して判定する。 In the determination criteria, for example, when 80% or more of the driver's movement is forward gaze or gaze in a predetermined direction, the degree of concentration is defined as “high”. In addition, in the determination criteria, for example, 20% or more of the driver's movement is forward gaze or gaze in a predetermined direction, and the driver's movement includes operation of a mobile phone or smartphone and a state of a call. The degree of concentration is defined as “low”. The concentration degree determination unit 612 determines the movement of the driver recognized from the monitoring video data stored in the monitoring video storage area 631 with reference to the determination criterion.
　なお、このような判定基準は、一例であって、設計者が適宜適切に設定することが可能である。例えば、判定基準を、代表的な動作毎に予め点数を決めておき、全体時間に対する各動作の割合を掛けて合計する、と設定してもよい。このとき、集中度は、運転者の時系列的な動きに基づいて数値化されることになる。 Note that such a determination criterion is an example, and the designer can appropriately set it appropriately. For example, the determination criterion may be set such that a score is determined in advance for each representative operation, and the total is multiplied by the ratio of each operation to the total time. At this time, the degree of concentration is digitized based on the driver's time-series movement.
　また、集中度は、２段階に限定されるものではない。例えば、集中度「高」と集中度「低」との間に集中度「中」を定義することで、集中度を３段階で評価してもよい。また、集中度を、４段階以上で評価してもよい。 Also, the concentration level is not limited to two stages. For example, the degree of concentration may be evaluated in three stages by defining the degree of concentration “medium” between the degree of concentration “high” and the degree of concentration “low”. Moreover, you may evaluate a concentration degree in four steps or more.
　集中度判定部６１２は、集中度に必要な数の監視映像データがステアリングホイール４によるドライバカメラ７の遮蔽により取得されない場合、直前の判定で得られた集中度を、集中度記憶領域６３２から読み出す。集中度判定部６１２は、読み出した集中度を、現時点における集中度として運転モード切替制御部６１３へ出力する。また、集中度判定部６１２は、読み出した集中度を、タイムスタンプを付して集中度記憶領域６３２に記憶させる。なお、本実施形態では、運転者の状態を表す指標として集中度を例に説明しているが、これに限定されない。 The concentration degree determination unit 612 reads the concentration degree obtained in the immediately preceding determination from the concentration degree storage area 632 when the number of monitoring video data necessary for the concentration degree is not acquired due to the shielding of the driver camera 7 by the steering wheel 4. . The concentration level determination unit 612 outputs the read concentration level to the operation mode switching control unit 613 as the current concentration level. In addition, the concentration degree determination unit 612 stores the read concentration degree in the concentration degree storage area 632 with a time stamp. In this embodiment, the degree of concentration is described as an example of an index representing the driver's state, but the present invention is not limited to this.
　運転モード切替制御部６１３は、集中度判定部６１２で判定された集中度に基づき、手動運転モードと、自動運転モードとの間で設定を切り替えるように自動運転制御装置５を制御する。運転モード切替制御部６１３は、切替制御を表す情報を、タイムスタンプを付して履歴記憶領域６３３に記憶させる。 The operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612. The operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 with a time stamp.
　次に、以上のように構成された運転モード切替制御装置６の動作を説明する。　
　図４は、図２に示される運転モード切替制御装置６に設けられる制御ユニット６１の監視映像取得制御部６１１が監視映像データを記憶する際の手順を示すフローチャートである。 Next, the operation of the operation mode switching control device 6 configured as described above will be described.
FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit 611 of the control unit 61 provided in the operation mode switching control device 6 shown in FIG. 2 stores the monitoring video data.
　まず、運転が開始されるとドライバカメラ７、及びステアリングセンサ８が起動する。ドライバカメラ７は、運転者の顔を含む所定の範囲を連続的に撮像する。ドライバカメラ７は、撮像した監視映像を映像信号として運転モード切替制御装置６へ出力する。なお、ドライバカメラ７は、ドライバの状態を判定する時間間隔より短い周期で間欠的にドライバを撮像してもよい。また、ドライバカメラ７、又は入出力インタフェースユニット６２は、映像信号を所定の符号化方式に応じて符号化するようにしても構わない。このようにすることで、監視映像データの情報量を減らして監視映像記憶領域６３１の記憶容量を節約することが可能となる。 First, when driving is started, the driver camera 7 and the steering sensor 8 are activated. The driver camera 7 continuously captures a predetermined range including the driver's face. The driver camera 7 outputs the captured monitoring video to the operation mode switching control device 6 as a video signal. The driver camera 7 may intermittently image the driver at a cycle shorter than the time interval for determining the driver state. The driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. By doing so, it is possible to reduce the information amount of the monitoring video data and save the storage capacity of the monitoring video storage area 631.
　ステアリングセンサ８は、ステアリングホイール４の操舵角を検出する。ステアリングセンサ８は、検出した操舵角を操舵角検出信号として運転モード切替制御装置６へ出力する。 The steering sensor 8 detects the steering angle of the steering wheel 4. The steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
　監視映像取得制御部６１１は、ステアリングセンサ８から出力され、入出力インタフェースユニット６２で変換された操舵角データを受信する（ステップＳ４１）。また、監視映像取得制御部６１１は、ドライバカメラ７から出力され、入出力インタフェースユニット６２で変換された監視映像データを受信する（ステップＳ４２）。 The monitoring video acquisition control unit 611 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62 (step S41). Further, the monitoring video acquisition control unit 611 receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S42).
　監視映像取得制御部６１１は、所定のサンプリングレートで、受信した操舵角データにより表される操舵角が、予め設定される遮蔽角度幅に含まれているか否かを判定する（ステップＳ４３）。操舵角が遮蔽角度幅に含まれていない場合（ステップＳ４３のＮｏ）、監視映像取得制御部６１１は、上記サンプリングレートで、受信した監視映像データをサンプリングし、サンプリングした監視映像データを監視映像記憶領域６３１に記憶させる（ステップＳ４４）。操舵角が遮蔽角度幅に含まれている場合（ステップＳ４３のＹｅｓ）、監視映像取得制御部６１１は、上記サンプリングレートで、受信した監視映像データをサンプリングし、サンプリングした監視映像データを、遮蔽フラグを付して監視映像記憶領域６３１に記憶させる（ステップＳ４５）。 The monitoring image acquisition control unit 611 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate (step S43). When the steering angle is not included in the shielding angle width (No in step S43), the monitoring video acquisition control unit 611 samples the received monitoring video data at the sampling rate and stores the sampled monitoring video data in the monitoring video storage. It memorize | stores in the area | region 631 (step S44). When the steering angle is included in the shielding angle width (Yes in step S43), the monitoring video acquisition control unit 611 samples the received monitoring video data at the sampling rate, and uses the sampled monitoring video data as the shielding flag. And stored in the monitoring video storage area 631 (step S45).
　ステップＳ４４において監視映像データを監視映像記憶領域６３１に記憶させ、若しくは、ステップＳ４５において遮蔽フラグが付された監視映像データを監視映像記憶領域６３１に記憶させると、監視映像取得制御部６１１は、運転が終了したか否かを判定する（ステップＳ４６）。監視映像取得制御部６１１は、運転が終了した場合（ステップＳ４６のＹｅｓ）、処理を終了させる。一方、運転が終了しない場合（ステップＳ４６のＮｏ）、監視映像取得制御部６１１は、処理をステップＳ４１へ移行させ、ステップＳ４１～ステップＳ４５の動作を車両１の運転が終了するまで繰り返す。 When the monitoring video data is stored in the monitoring video storage area 631 in step S44 or the monitoring video data to which the shielding flag is attached is stored in the monitoring video storage area 631 in step S45, the monitoring video acquisition control unit 611 operates. It is determined whether or not has been completed (step S46). When the operation is finished (Yes in step S46), the monitoring video acquisition control unit 611 ends the process. On the other hand, when the driving does not end (No in step S46), the monitoring video acquisition control unit 611 shifts the process to step S41, and repeats the operations in steps S41 to S45 until the driving of the vehicle 1 ends.
　図５は、図２に示される運転モード切替制御装置６に設けられる制御ユニット６１の集中度判定部６１２がドライバの集中度を判定する際の手順の例を示すフローチャートである。 FIG. 5 is a flowchart illustrating an example of a procedure when the concentration degree determination unit 612 of the control unit 61 provided in the operation mode switching control device 6 illustrated in FIG. 2 determines the concentration degree of the driver.
　図４に示される監視映像データの記憶が開始されると集中度判定部６１２は、計時を開始する。集中度判定部６１２は、計時を開始してから一定時間が経過したか否かを判定する（ステップＳ５１）。一定時間が経過すると（ステップＳ５１のＹｅｓ）、集中度判定部６１２は、一定時間の間に監視映像記憶領域６３１に記憶された、予め設定された数の監視映像データを、監視映像記憶領域６３１から読み込む（ステップＳ５２）。集中度判定部６１２は、読み込んだ監視映像データに基づき、運転者の動き、及び／又は運転者の表情を認識する。なお、集中度判定部６１２は、監視映像取得制御部６１１が監視映像データを新たにサンプリングする毎に、予め設定した数の監視映像データを読み込み、読み込んだ監視映像データに基づき、運転者の動き等を認識するようにしてもよい。 When the storage of the monitoring video data shown in FIG. 4 is started, the concentration degree determination unit 612 starts timing. The concentration degree determination unit 612 determines whether or not a certain time has elapsed since the start of time measurement (step S51). When the predetermined time has elapsed (Yes in step S51), the concentration degree determination unit 612 stores the preset number of monitoring video data stored in the monitoring video storage area 631 during the predetermined time in the monitoring video storage area 631. (Step S52). The concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data. The degree-of-concentration determination unit 612 reads a preset number of monitoring video data each time the monitoring video acquisition control unit 611 newly samples the monitoring video data, and moves the driver based on the read monitoring video data. Etc. may be recognized.
　集中度判定部６１２は、読み込んだ監視映像データのうち、運転者の動き、及び／又は運転者の表情が認識できないデータ、すなわち、遮蔽フラグが付されたデータがあるか否かを判定する（ステップＳ５３）。遮蔽フラグが付されたデータがある場合（ステップＳ５３のＹｅｓ）、集中度判定部６１２は、運転者の動き等が認識できたデータが、集中度を判定するのに必要なサンプル数存在するか否かを判定する（ステップＳ５４）。集中度を判定するのに必要なサンプル数が存在する場合（ステップＳ５４のＹｅｓ）、集中度判定部６１２は、運転者の動き等を認識可能なデータに基づいて集中度を判定する（ステップＳ５５）。集中度判定部６１２は、判定した集中度を表す情報を、運転モード切替制御部６１３へ出力する。また、集中度判定部６１２は、判定した集中度を表す情報を、判定タイミングを表す情報、例えばタイムスタンプ情報と関連付けて集中度記憶領域６３２に記憶させる（ステップＳ５６）。 The degree-of-concentration determination unit 612 determines whether or not there is data in which the movement of the driver and / or the driver's facial expression cannot be recognized among the read monitoring video data, that is, data with a shielding flag (see FIG. Step S53). When there is data to which the shielding flag is attached (Yes in step S53), the concentration determination unit 612 determines whether there is the number of samples necessary for determining the concentration of the data in which the driver's movement or the like can be recognized. It is determined whether or not (step S54). When the number of samples necessary to determine the concentration level exists (Yes in step S54), the concentration level determination unit 612 determines the concentration level based on data that can recognize the movement of the driver and the like (step S55). ). The concentration level determination unit 612 outputs information indicating the determined concentration level to the operation mode switching control unit 613. In addition, the concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S56).
　集中度を判定するのに必要なサンプル数が存在しない場合（ステップＳ５４のＮｏ）、集中度判定部６１２は、現時刻の直前に判定された集中度を、付されているタイムスタンプを参照して集中度記憶領域６３２から読み出す。集中度判定部６１２は、読み出した集中度を、現時点における集中度として採用する（ステップＳ５７）。集中度判定部６１２は、採用した集中度を表す情報を、運転モード切替制御部６１３へ出力する。また、集中度判定部６１２は、採用した集中度を表す情報を、判定タイミングを表す情報、例えばタイムスタンプ情報と関連付けて集中度記憶領域６３２に記憶させる（ステップＳ５８）。 When the number of samples necessary to determine the concentration level does not exist (No in step S54), the concentration level determination unit 612 refers to the attached time stamp with the concentration level determined immediately before the current time. To read from the concentration storage area 632. The concentration level determination unit 612 employs the read concentration level as the current concentration level (step S57). The concentration degree determination unit 612 outputs information representing the adopted concentration degree to the operation mode switching control unit 613. In addition, the concentration degree determination unit 612 stores the information indicating the adopted concentration degree in the concentration degree storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S58).
　なお、ステップＳ５４における、集中度を判定するのに必要なサンプル数存在するか否かの判定は、必須ではない。例えば、ステップＳ５３において、遮蔽フラグが付されたデータがあると判定した場合、集中度判定部６１２は、処理をステップＳ５７へ移行させるようにしてもよい。 Note that it is not essential to determine whether or not the number of samples necessary for determining the degree of concentration exists in step S54. For example, when it is determined in step S53 that there is data with a shielding flag, the concentration degree determination unit 612 may shift the process to step S57.
　ステップＳ５６、又はステップＳ５８において、集中度を表す情報を監視映像記憶領域６３１に記憶させると、集中度判定部６１２は、運転が終了したか否かを判定する（ステップＳ５９）。集中度判定部６１２は、運転が終了した場合（ステップＳ５９のＹｅｓ）、処理を終了させる。一方、運転が終了しない場合（ステップＳ５９のＮｏ）、集中度判定部６１２は、処理をステップＳ５１へ移行させ、ステップＳ５１～ステップＳ５８の動作を車両１の運転が終了するまで繰り返す。 In step S56 or step S58, when the information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612 determines whether or not the driving is finished (step S59). When the driving is finished (Yes in step S59), the concentration degree determination unit 612 ends the process. On the other hand, when the driving does not end (No in step S59), the concentration degree determination unit 612 shifts the process to step S51, and repeats the operations in steps S51 to S58 until the driving of the vehicle 1 ends.
　運転モード切替制御部６１３は、集中度判定部６１２で判定された集中度に基づき、手動運転モードと、自動運転モードとの間で設定を切り替えるように自動運転制御装置５を制御する。例えば、運転モード切替制御部６１３は、「自動運転モード」から、「手動運転モード」へ切り替える際、運転モードを切り替える旨、及び切り替えまでの期間を、図示しないアラーム発生器を用いて運転者へ通知する。 The operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612. For example, when switching from the “automatic operation mode” to the “manual operation mode”, the operation mode switching control unit 613 informs the driver that the operation mode is to be switched and the period until the switching using an alarm generator (not shown). Notice.
　運転モード切替制御部６１３は、この切替期間中において、運転者の集中度が予め設定される集中度を超えたか否かを判定する。運転モード切替制御部６１３は、切替期間中に、運転者の集中度が予め設定される集中度を超えた場合、「手動運転モード」へ切り替える準備が運転者の中で整ったと判定する。切替期間中に運転者の準備が整ったと判定すると、運転モード切替制御部６１３は、切替期間が経過した後に、「自動運転モード」から、「手動運転モード」へ切り替える旨の運転モード切替制御信号を自動運転制御装置５へ出力する。運転モード切替制御部６１３は、切替制御を表す情報を、判定タイミングを表す情報、例えばタイムスタンプ情報と関連付けて記憶ユニット６３の履歴記憶領域６３３に記憶させる。 The driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level during the switching period. The driving mode switching control unit 613 determines that the driver is ready to switch to the “manual driving mode” when the driver's concentration exceeds the preset concentration during the switching period. If it is determined that the driver is ready during the switching period, the driving mode switching control unit 613 switches the “automatic driving mode” to the “manual driving mode” after the switching period has elapsed. Is output to the automatic operation control device 5. The operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 of the storage unit 63 in association with information indicating the determination timing, for example, time stamp information.
　なお、集中度を用いた判定は、上記に限定されない。例えば、運転モードを切り替える切替期間は、集中度に基づいてカウントダウンを開始されてもよい。具体的には、例えば、運転モード切替制御部６１３は、運転モードを切り替える旨を運転者へ通知した後、集中度判定部６１２から出力される集中度に基づき、運転者の準備が整ったか否かを判定する。運転モード切替制御部６１３は、受信した集中度が予め設定される集中度を超えた場合、運転者の準備が整ったと判定し、運転モード切替までの期間のカウントダウンを開始する。 Note that the determination using the degree of concentration is not limited to the above. For example, the countdown may be started based on the degree of concentration during the switching period for switching the operation mode. Specifically, for example, the driving mode switching control unit 613 notifies the driver that the driving mode is to be switched, and then whether or not the driver is ready based on the concentration level output from the concentration level determination unit 612. Determine whether. When the received concentration level exceeds the preset concentration level, the operation mode switching control unit 613 determines that the driver is ready and starts counting down the period until the operation mode is switched.
　以上詳述したようにこの発明の第１の実施形態では、制御ユニット６１は、ステアリングセンサ８で検出される操舵角に基づき、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽されているか否かを判定する。そして、制御ユニット６１は、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽されている場合には、ドライバカメラ７の撮影範囲が遮蔽された時点の集中度として、遮蔽直前のタイミングに判定された集中度を採用するようにしている。このため、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽されたタイミングで、例えば、自動運転から手動運転への切替可否の判定等で集中度を利用したい場合であっても、滞りなく判定を実施することが可能となる。 As described above in detail, in the first embodiment of the present invention, the control unit 61 is configured such that the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. It is determined whether or not. When the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, the control unit 61 sets the concentration level at the time when the shooting range of the driver camera 7 is shielded at the timing immediately before the shielding. The determined degree of concentration is adopted. For this reason, even when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving, etc. It becomes possible to carry out the determination without any problem.
　また、第１の実施形態では、監視映像取得制御部６１１は、ステアリングセンサ８で検出される操舵角に基づき、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽された際に取得された監視映像を判別する。集中度判定部６１２は、ドライバカメラ７の撮影範囲が遮蔽された際に取得された監視映像がある場合、ドライバカメラ７の撮影範囲が遮蔽される直前のタイミングで判定した集中度を、ドライバカメラ７の撮影範囲が遮蔽されている間の集中度として採用するようにしている。このため、監視映像取得制御部６１１で、ドライバカメラ７の撮影範囲が遮蔽された際に取得された監視映像を効率的に選別できるようになる。 In the first embodiment, the monitoring image acquisition control unit 611 is acquired when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Discriminate surveillance video. When there is a monitoring image acquired when the shooting range of the driver camera 7 is blocked, the concentration level determination unit 612 determines the concentration level determined at the timing immediately before the shooting range of the driver camera 7 is blocked. 7 is adopted as the degree of concentration while the shooting range is shielded. Therefore, the monitoring video acquisition control unit 611 can efficiently sort the monitoring video acquired when the shooting range of the driver camera 7 is blocked.
　（他の実施例）
　第１の実施形態では、監視映像取得制御部６１１により、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽された際に取得された監視映像を判別する場合を例に説明した。しかしながら、これに限定されない。制御ユニット６１における、監視映像取得制御部６１１以外の機能において、ステアリングセンサ８で取得された操舵角が利用されても構わない。例えば、監視映像取得制御部６１１に代えて集中度判定部６１２が、ステアリングセンサ８で取得された操舵角に基づき、監視映像がステアリングホイール４によりドライバカメラ７の撮影範囲が遮蔽される遮蔽状態において取得されたものであると判定してもよい。 (Other examples)
In the first embodiment, the case where the monitoring video acquired by the monitoring video acquisition control unit 611 when the imaging range of the driver camera 7 is shielded by the support member of the steering wheel 4 has been described as an example. However, it is not limited to this. In functions other than the monitoring image acquisition control unit 611 in the control unit 61, the steering angle acquired by the steering sensor 8 may be used. For example, instead of the monitoring video acquisition control unit 611, the concentration degree determination unit 612 is based on the steering angle acquired by the steering sensor 8, and the monitoring video is in a shielding state where the shooting range of the driver camera 7 is blocked by the steering wheel 4. You may determine with what was acquired.
　具体的には、例えば、集中度判定部６１２は、ステアリングセンサ８から出力され、入出力インタフェースユニット６２で変換された操舵角データを受信する。集中度判定部６１２は、所定のサンプリングレートで、受信した操舵角データにより表される操舵角が、予め設定される遮蔽角度幅に含まれているか否かを判定する。操舵角が遮蔽角度幅に含まれている場合、集中度判定部６１２は、遮蔽状態にあると判定したタイミングを表すタイミング情報を記憶ユニット６３に記憶させる。 Specifically, for example, the concentration degree determination unit 612 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62. The concentration degree determination unit 612 determines whether or not the steering angle represented by the received steering angle data is included in a preset shielding angle width at a predetermined sampling rate. When the steering angle is included in the shielding angle width, the concentration degree determination unit 612 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
　集中度判定部６１２は、一定時間の間に監視映像記憶領域６３１に記憶された監視映像データを、監視映像記憶領域６３１から読み込む。集中度判定部６１２は、記憶しているタイミング情報と、読み込んだ監視映像データとを突合わせることで、読み込んだ監視映像データのうち、遮蔽状態において取得された監視映像データを判別する。 The concentration degree determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 from the monitoring video storage area 631 for a predetermined time. The degree-of-concentration determination unit 612 determines the monitoring video data acquired in the shielding state from the read monitoring video data by matching the stored timing information with the read monitoring video data.
　読み込んだ監視映像データのうち、遮蔽状態でない場合に取得された監視映像データの数が集中度を判定可能な数だけ存在していない場合、集中度判定部６１２は、現時刻の直前に判定された集中度を集中度記憶領域６３２から読み出す。そして、集中度判定部６１２は、読み出した集中度を、現時点における集中度として採用する。なお、読み込んだ監視映像データに、遮蔽状態において取得された監視映像データが含まれている場合、集中度判定部６１２は、現時刻の直前に判定された集中度を現時点における集中度として採用してもよい。 In the read monitoring video data, when the number of monitoring video data acquired when not in the shielding state does not exist so that the concentration level can be determined, the concentration level determination unit 612 is determined immediately before the current time. The degree of concentration is read from the concentration degree storage area 632. Then, the concentration level determination unit 612 employs the read concentration level as the current concentration level. When the read monitoring video data includes monitoring video data acquired in the shielding state, the concentration level determination unit 612 adopts the concentration level determined immediately before the current time as the current concentration level. May be.
　これにより、集中度判定部６１２で、ドライバカメラ７の撮影範囲がステアリングホイール４により遮蔽された際に取得された監視映像データを選別でき、監視映像取得制御部６１１の処理を低減させることが可能となる。 Thereby, the concentration degree determination unit 612 can select the monitoring video data acquired when the shooting range of the driver camera 7 is shielded by the steering wheel 4, and the processing of the monitoring video acquisition control unit 611 can be reduced. It becomes.
　また、監視映像取得制御部６１１に代えて、運転モード切替制御部６１３が、ステアリングセンサ８で取得された操舵角に基づき、ステアリングホイール４によりドライバカメラ７の撮影範囲が遮蔽されたと判定してもよい。このとき、運転モード切替制御部６１３は、例えば、以下のように動作する。なお、以下の動作において、集中度判定部６１２は、一定時間が経過する度に監視映像記憶領域６３１に記憶された監視映像データを読み出し、読み出した監視映像データに基づいて集中度を判定する。集中度判定部６１２は、判定した集中度を表す情報を集中度記憶領域６３２に記憶する。 Further, instead of the monitoring video acquisition control unit 611, the operation mode switching control unit 613 determines that the shooting range of the driver camera 7 is blocked by the steering wheel 4 based on the steering angle acquired by the steering sensor 8. Good. At this time, the operation mode switching control unit 613 operates as follows, for example. In the following operation, the concentration level determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 every time a predetermined time elapses, and determines the concentration level based on the read monitoring video data. The concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632.
　運転モード切替制御部６１３は、ステアリングセンサ８から出力され、入出力インタフェースユニット６２で変換された操舵角データを受信する。運転モード切替制御部６１３は、所定のサンプリングレートで、受信した操舵角データにより表される操舵角が、予め設定される遮蔽角度幅に含まれているか否かを判定する。操舵角が遮蔽角度幅に含まれている場合、運転モード切替制御部６１３は、遮蔽状態にあると判定したタイミングを表すタイミング情報を記憶ユニット６３に記憶させる。 The operation mode switching control unit 613 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62. The operation mode switching control unit 613 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate. When the steering angle is included in the shielding angle width, the operation mode switching control unit 613 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
　運転モード切替制御部６１３は、運転モードの切替可否を判定する際、判定時点の直前に集中度判定部６１２で取得された集中度を表す情報を、集中度記憶領域６３２から読み込む。運転モード切替制御部６１３は、記憶ユニット６３に記憶したタイミング情報に基づき、読み込んだ集中度が、遮蔽状態において取得された集中度であるか否かを判定する。読み込んだ集中度が遮蔽状態において取得された集中度である場合、運転モード切替制御部６１３は、遮蔽状態になる直前に取得された集中度を表す情報を、集中度記憶領域６３２から読み込む。運転モード切替制御部６１３は、新たに読み込んだ情報により表される集中度に基づいて、運転モードの切替可否を判定する。 When the operation mode switching control unit 613 determines whether or not the operation mode can be switched, the operation mode switching control unit 613 reads, from the concentration level storage area 632, information indicating the concentration level acquired by the concentration level determination unit 612 immediately before the determination time point. Based on the timing information stored in the storage unit 63, the operation mode switching control unit 613 determines whether or not the read concentration level is the concentration level acquired in the shielding state. When the read concentration degree is the concentration degree acquired in the shielding state, the operation mode switching control unit 613 reads information indicating the concentration degree acquired immediately before entering the shielding state from the concentration degree storage area 632. The operation mode switching control unit 613 determines whether or not the operation mode can be switched based on the degree of concentration represented by the newly read information.
　これにより、ドライバカメラ７の撮影範囲がステアリングホイール４により遮蔽された場合であっても、運転モード切替制御部６１３で、遮蔽状態になる直前に取得された集中度を利用して運転モードを切り替えることが可能となる。このため、監視映像取得制御部６１１、及び集中度判定部６１２での処理を低減させることが可能となる。 As a result, even when the shooting range of the driver camera 7 is shielded by the steering wheel 4, the operation mode switching control unit 613 switches the operation mode using the degree of concentration acquired immediately before entering the shielding state. It becomes possible. For this reason, it is possible to reduce processing in the monitoring video acquisition control unit 611 and the concentration degree determination unit 612.
　［第２の実施形態］
　第１の実施形態では、運転者の集中度を判定するための第１の手段として、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽された際に過去に判定された集中度を採用する場合を例に説明した。第２の実施形態では、ステアリングホイール４の支持部材によりドライバカメラ７の撮影範囲が遮蔽されるタイミングを予測して対応する場合を説明する。 [Second Embodiment]
In the first embodiment, as a first means for determining the driver's concentration, the concentration determined in the past when the photographing range of the driver camera 7 is shielded by the support member of the steering wheel 4 is used. The case where it is adopted has been described as an example. In the second embodiment, a case will be described in which the timing at which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 is predicted and handled.
　図６は、この発明の第２の実施形態に係る運転モード切替制御装置６Ｂの機能構成の例を示すブロック図である。図６によれば、運転モード切替制御装置６Ｂは、制御ユニット６１Ｂと、入出力インタフェースユニット６２と、記憶ユニット６３とを備えている。制御ユニット６１では、プログラムメモリに格納されたプログラムをＣＰＵに実行させることにより、第２の実施形態を実施するために必要な制御機能である、監視映像取得制御部６１１Ｂと、集中度判定部６１２Ｂと、運転モード切替制御部６１３と、遮蔽予測部６１４とが実現される。 FIG. 6 is a block diagram showing an example of a functional configuration of the operation mode switching control device 6B according to the second embodiment of the present invention. According to FIG. 6, the operation mode switching control device 6B includes a control unit 61B, an input / output interface unit 62, and a storage unit 63. In the control unit 61, by causing the CPU to execute a program stored in the program memory, a monitoring video acquisition control unit 611B and a concentration degree determination unit 612B, which are control functions necessary for implementing the second embodiment, are provided. Then, the operation mode switching control unit 613 and the shielding prediction unit 614 are realized.
　遮蔽予測部６１４は、ステアリングセンサ８から出力された操舵角検出信号についてのディジタルデータ（操舵角データ）を入出力インタフェースユニット６２から取り込む。遮蔽予測部６１４は、取り込んだ操舵角データの経時変化に基づき、操舵角速度を算出する。遮蔽予測部６１４は、ドライバカメラ７と、ステアリングホイール４との相対位置と、算出した操舵角速度とに基づき、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽される期間を予測する。遮蔽予測部６１４は、予測した遮蔽期間の開始時刻から予め設定した期間だけ前の時刻になると動作信号を監視映像取得制御部６１１Ｂ、及び集中度判定部６１２Ｂへ出力する。 The shielding prediction unit 614 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62. The shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the acquired steering angle data. The shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity. The shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B when a time that is a predetermined period before the predicted start time of the shielding period comes.
　監視映像取得制御部６１１Ｂは、ドライバカメラ７から出力されたドライバの映像信号についてのディジタルデータ（監視映像データ）を入出力インタフェースユニット６２から取り込む。監視映像取得制御部６１１Ｂは、取り込んだ監視映像データを、所定のサンプリングレートで記憶ユニット６３の監視映像記憶領域６３１に記憶させる。 The monitoring video acquisition control unit 611B takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62. The monitoring video acquisition control unit 611B stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
　また、監視映像取得制御部６１１Ｂは、遮蔽予測部６１４から送信される動作信号を受信すると、動作信号を受信したタイミングと同期させて、取り込んだ監視映像データを記憶ユニット６３の監視映像記憶領域６３１に記憶させる。 Further, when receiving the operation signal transmitted from the shielding prediction unit 614, the monitoring image acquisition control unit 611B synchronizes with the timing at which the operation signal is received, and stores the captured monitoring image data in the monitoring image storage area 631 of the storage unit 63. Remember me.
　集中度判定部６１２Ｂは、予め設定された時間間隔で、この時間間隔の間に記憶された監視映像データを、監視映像記憶領域６３１から読み込む。集中度判定部６１２Ｂは、読み込んだ監視映像データに基づき、運転者の動き、及び／又は運転者の表情を認識する。集中度判定部６１２Ｂは、認識した運転者の動き等に基づき、運転者の運転に対する集中度を判定する。 The concentration determination unit 612B reads the monitoring video data stored during the time interval from the monitoring video storage area 631 at a preset time interval. The concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data. The concentration degree determination unit 612B determines the degree of concentration of the driver with respect to driving based on the recognized movement of the driver.
　また、集中度判定部６１２Ｂは、遮蔽予測部６１４から送信される動作信号を受信すると、受信したタイミングを含む時点から所定期間前の間に記憶された監視映像データを、監視映像記憶領域６３１から読み込む。集中度判定部６１２Ｂは、読み込んだ監視映像データに基づいて集中度を判定する。集中度判定部６１２Ｂは、集中度の判定結果を運転モード切替制御部６１３へ出力する。また、集中度判定部６１２Ｂは、集中度の判定結果を、タイムスタンプを付して集中度記憶領域６３２に記憶させる。 In addition, when the concentration determination unit 612B receives the operation signal transmitted from the shielding prediction unit 614, the monitoring video data stored during a predetermined period from the time including the received timing is received from the monitoring video storage area 631. Read. The concentration level determination unit 612B determines the concentration level based on the read monitoring video data. The concentration level determination unit 612B outputs the determination result of the concentration level to the operation mode switching control unit 613. In addition, the concentration level determination unit 612B stores the determination result of the concentration level in the concentration level storage area 632 with a time stamp.
　次に、以上のように構成された運転モード切替制御装置６Ｂの動作を説明する。　
　図７は、図６に示される運転モード切替制御装置６Ｂに設けられる制御ユニット６１Ｂの遮蔽予測部６１４が動作信号を出力する際の手順の例を示すフローチャートである。 Next, the operation of the operation mode switching control device 6B configured as described above will be described.
FIG. 7 is a flowchart illustrating an example of a procedure when the shielding prediction unit 614 of the control unit 61B provided in the operation mode switching control device 6B illustrated in FIG. 6 outputs an operation signal.
　まず、運転が開始されるとステアリングセンサ８が起動する。ステアリングセンサ８は、ステアリングホイール４の操舵角を検出する。ステアリングセンサ８は、検出した操舵角を操舵角検出信号として運転モード切替制御装置６Ｂへ出力する。 First, when driving is started, the steering sensor 8 is activated. The steering sensor 8 detects the steering angle of the steering wheel 4. The steering sensor 8 outputs the detected steering angle as a steering angle detection signal to the operation mode switching control device 6B.
　遮蔽予測部６１４は、ステアリングセンサ８から出力され、入出力インタフェースユニット６２で変換された所定期間における操舵角データを受信する（ステップＳ７１）。遮蔽予測部６１４は、受信した操舵角データの経時変化に基づき、操舵角速度を算出する（ステップＳ７２）。遮蔽予測部６１４は、ドライバカメラ７と、ステアリングホイール４との相対位置と、算出した操舵角速度とに基づき、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽される期間を予測する（ステップＳ７３）。 The shielding prediction unit 614 receives the steering angle data for a predetermined period output from the steering sensor 8 and converted by the input / output interface unit 62 (step S71). The shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the received steering angle data (step S72). The shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity ( Step S73).
　遮蔽予測部６１４は、現在の時刻が、予測した遮蔽期間の開始時刻から予め設定した期間（例えばｓ秒）だけ前の時刻になったか否かを判定する（ステップＳ７４）。遮蔽期間の開始時刻から予め設定した期間だけ前の時刻である場合（ステップＳ７４のＹｅｓ）、遮蔽予測部６１４は、動作信号を監視映像取得制御部６１１Ｂ、及び集中度判定部６１２Ｂへ出力する（ステップＳ７５）。 The shielding prediction unit 614 determines whether or not the current time is a time that is a time period (for example, s seconds) prior to the predicted start time of the shielding period (step S74). When it is a time before a predetermined period from the start time of the shielding period (Yes in step S74), the shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B ( Step S75).
　遮蔽期間の開始時刻から予め設定した期間だけ前の時刻ではない場合（ステップＳ７４のＮｏ）、又は、ステップＳ７５で動作信号を出力した後、遮蔽予測部６１４は、車両１の運転が終了したか否かを判定する（ステップＳ７６）。遮蔽予測部６１４は、運転が終了した場合（ステップＳ７６のＹｅｓ）、処理を終了させる。一方、運転が終了しない場合（ステップＳ７６のＮｏ）、遮蔽予測部６１４は、処理をステップＳ７１へ移行させ、ステップＳ７１～ステップＳ４５の動作を車両１の運転が終了するまで繰り返す。 If the time is not the time previously set by the preset period from the start time of the shielding period (No in step S74), or after outputting the operation signal in step S75, the shielding prediction unit 614 has finished driving the vehicle 1 It is determined whether or not (step S76). The shielding prediction unit 614 ends the process when the driving is finished (Yes in Step S76). On the other hand, when the driving is not finished (No in step S76), the shielding prediction unit 614 shifts the process to step S71, and repeats the operations in steps S71 to S45 until the driving of the vehicle 1 is finished.
　図８は、図６に示される制御ユニット６１Ｂの監視映像取得制御部６１１Ｂが監視映像データを記憶する際の手順の例を示すフローチャートである。 FIG. 8 is a flowchart showing an example of a procedure when the monitoring video acquisition control unit 611B of the control unit 61B shown in FIG. 6 stores the monitoring video data.
　まず、運転が開始されるとドライバカメラ７が起動する。ドライバカメラ７は、運転者の顔を含む所定の範囲を連続的に撮像する。ドライバカメラ７は、撮像した監視映像を映像信号として運転モード切替制御装置６Ｂへ出力する。 First, when the operation is started, the driver camera 7 is activated. The driver camera 7 continuously captures a predetermined range including the driver's face. The driver camera 7 outputs the captured monitoring video as a video signal to the operation mode switching control device 6B.
　監視映像取得制御部６１１Ｂは、ドライバカメラ７から出力され、入出力インタフェースユニット６２で変換された監視映像データを受信する（ステップＳ８１）。監視映像取得制御部６１１Ｂは、遮蔽予測部６１４により送信される動作信号を受信したか否かを判定する（ステップＳ８２）。動作信号を受信した場合（ステップＳ８２のＹｅｓ）、監視映像取得制御部６１１Ｂは、動作信号を受信したタイミング、すなわち、予め設定されるサンプリングレートとは異なるタイミングで、受信した監視映像データをサンプリングし、サンプリングした監視映像データを監視映像記憶領域６３１に記憶させる（ステップＳ８３）。 The monitoring video acquisition control unit 611B receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S81). The monitoring video acquisition control unit 611B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S82). When the operation signal is received (Yes in step S82), the monitoring video acquisition control unit 611B samples the received monitoring video data at a timing at which the operation signal is received, that is, at a timing different from a preset sampling rate. The sampled monitoring video data is stored in the monitoring video storage area 631 (step S83).
　動作信号を受信していない場合（ステップＳ８２のＮｏ）、監視映像取得制御部６１１Ｂは、予め設定されるサンプリングレートで、受信した監視映像データをサンプリングし、サンプリングした監視映像データを監視映像記憶領域６３１に記憶させる（ステップＳ８４）。 When the operation signal has not been received (No in step S82), the monitoring video acquisition control unit 611B samples the received monitoring video data at a preset sampling rate, and the sampled monitoring video data is stored in the monitoring video storage area. 631 (step S84).
　ステップＳ４４、及びステップＳ４５において監視映像データを監視映像記憶領域６３１に記憶させると、監視映像取得制御部６１１Ｂは、運転が終了したか否かを判定する（ステップＳ８５）。監視映像取得制御部６１１Ｂは、運転が終了した場合（ステップＳ８５のＹｅｓ）、処理を終了させる。一方、運転が終了しない場合（ステップＳ８５のＮｏ）、監視映像取得制御部６１１Ｂは、処理をステップＳ８１へ移行させ、ステップＳ８１～ステップＳ８４の動作を車両１の運転が終了するまで繰り返す。 When the monitoring video data is stored in the monitoring video storage area 631 in step S44 and step S45, the monitoring video acquisition control unit 611B determines whether or not the operation is finished (step S85). When the driving is finished (Yes in step S85), the monitoring video acquisition control unit 611B ends the process. On the other hand, when the driving does not end (No in step S85), the monitoring video acquisition control unit 611B shifts the process to step S81, and repeats the operations in steps S81 to S84 until the driving of the vehicle 1 ends.
　図９は、図６に示される制御ユニット６１Ｂの集中度判定部６１２Ｂがドライバの集中度を判定する際の手順の例を示すフローチャートである。 FIG. 9 is a flowchart showing an example of a procedure when the concentration determination unit 612B of the control unit 61B shown in FIG. 6 determines the concentration of the driver.
　図９に示される監視映像データの記憶が開始されると集中度判定部６１２Ｂは、計時を開始する。計時を開始した後、集中度判定部６１２Ｂは、遮蔽予測部６１４により送信される動作信号を受信したか否かを判定する（ステップＳ９１）。動作信号を受信した場合（ステップＳ９１のＹｅｓ）、集中度判定部６１２Ｂは、動作信号を受信したタイミングを含む時点から所定期間前の間に監視映像記憶領域６３１に記憶された監視映像データを、監視映像記憶領域６３１から読み込む（ステップＳ９２）。 When the storage of the monitoring video data shown in FIG. 9 is started, the concentration degree determination unit 612B starts measuring time. After starting the time measurement, the concentration degree determination unit 612B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S91). When the operation signal is received (Yes in step S91), the concentration determination unit 612B receives the monitoring video data stored in the monitoring video storage area 631 before a predetermined period from the time including the timing at which the operation signal is received. Reading from the monitoring video storage area 631 (step S92).
　集中度判定部６１２Ｂは、読み込んだ監視映像データに基づき、運転者の動き、及び／又は運転者の表情を認識する。集中度判定部６１２Ｂは、監視映像データにおいて認識した運転者の動き等に基づいて集中度を判定する（ステップＳ９３）。集中度判定部６１２Ｂは、判定した集中度を表す情報を、運転モード切替制御部６１３へ出力する。また、集中度判定部６１２Ｂは、判定した集中度を表す情報を、判定タイミングを表す情報、例えばタイムスタンプ情報と関連付けて集中度記憶領域６３２に記憶させる（ステップＳ９４）。 The concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data. The concentration level determination unit 612B determines the concentration level based on the movement of the driver recognized in the monitoring video data (step S93). Concentration determination unit 612B outputs information indicating the determined concentration to operation mode switching control unit 613. In addition, the concentration level determination unit 612B stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating determination timing, for example, time stamp information (step S94).
　ステップＳ９１において、動作信号を受信していない場合、集中度判定部６１２Ｂは、計時を開始してから一定時間が経過したか否かを判定する（ステップＳ９５）。一定時間が経過すると（ステップＳ９５のＹｅｓ）、集中度判定部６１２Ｂは、一定時間の間に監視映像記憶領域６３１に記憶された監視映像データを、監視映像記憶領域６３１から読み込み（ステップＳ９６）、処理をステップＳ９３に移行させる。なお、集中度判定部６１２Ｂは、監視映像取得制御部６１１Ｂが監視映像データを新たにサンプリングする毎に、予め設定した数の監視映像データを読み込み、処理をステップＳ９３に移行させてもよい。 In step S91, when the operation signal is not received, the concentration degree determination unit 612B determines whether or not a predetermined time has elapsed since the start of time measurement (step S95). When the predetermined time has elapsed (Yes in step S95), the concentration determination unit 612B reads the monitoring video data stored in the monitoring video storage area 631 during the predetermined time from the monitoring video storage area 631 (step S96). The process proceeds to step S93. Note that the concentration determination unit 612B may read a preset number of monitoring video data each time the monitoring video acquisition control unit 611B newly samples the monitoring video data, and may shift the process to step S93.
　ステップＳ９３において、集中度を表す情報を監視映像記憶領域６３１に記憶させると、集中度判定部６１２Ｂは、運転が終了したか否かを判定する（ステップＳ９７）。集中度判定部６１２Ｂは、運転が終了した場合（ステップＳ９７のＹｅｓ）、処理を終了させる。一方、運転が終了しない場合（ステップＳ９７のＮｏ）、集中度判定部６１２Ｂは、処理をステップＳ９１へ移行させ、ステップＳ９１～ステップＳ９６の動作を車両１の運転が終了するまで繰り返す。 In step S93, when information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612B determines whether or not the driving is finished (step S97). When the driving is finished (Yes in step S97), the concentration degree determination unit 612B ends the process. On the other hand, when the driving does not end (No in step S97), the concentration degree determination unit 612B shifts the process to step S91, and repeats the operations in steps S91 to S96 until the driving of the vehicle 1 ends.
　以上詳述したようにこの発明の第２の実施形態では、制御ユニット６１Ｂは、ステアリングセンサ８で検出される操舵角に基づき、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽される期間を予測する。そして、制御ユニット６１Ｂは、予測した期間における運転者の集中度を、遮蔽期間となる前に取得した監視映像データに基づいて判定するようにしている。このため、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽されると予測されるタイミングで、例えば、自動運転から手動運転への切替可否の判定等で集中度を利用したい場合であっても、遮蔽状態となると予測される直前に取得された集中度を利用して切替可否の判定が可能となる。 As described above in detail, in the second embodiment of the present invention, the control unit 61B has the shooting range of the driver camera 7 shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Predict the period. Then, the control unit 61B determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, at the timing when the shooting range of the driver camera 7 is predicted to be shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving. However, it is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
　また、第２の実施形態では、遮蔽予測部６１４は、予測した遮蔽期間の開始時点から予め設定した期間だけ前の時点になると動作信号を出力する。監視映像取得制御部６１１Ｂは、動作信号を受信したタイミングで取り込んだ監視映像データを記憶する。そして、集中度判定部６１２Ｂは、動作信号を受信したタイミングで、このタイミングにおいて監視映像取得制御部６１１Ｂにより取得された監視映像データに基づいて集中度を判定し、この集中度を、予測した遮蔽期間における集中度とするようにしている。このため、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽されると予測されるタイミングで運転モードの切替可否を判定したい場合であっても、遮蔽状態となると予測される直前の、予め設定したサンプリングレートとは異なるタイミングで取得された集中度を利用することが可能となる。 Also, in the second embodiment, the shielding prediction unit 614 outputs an operation signal when a time point that is a predetermined time period before the predicted start time of the shielding period is reached. The monitoring video acquisition control unit 611B stores the monitoring video data captured at the timing when the operation signal is received. Then, the concentration level determination unit 612B determines the concentration level based on the monitoring video data acquired by the monitoring video acquisition control unit 611B at the timing at which the operation signal is received, and this concentration level is predicted as a shielding. The degree of concentration in the period is set. For this reason, even when it is desired to determine whether or not the operation mode can be switched at a timing at which the shooting range of the driver camera 7 is predicted to be shielded by the support member of the steering wheel 4, It is possible to use the degree of concentration acquired at a timing different from the preset sampling rate.
　（他の実施例）
　上記第２の実施形態では、例えば、集中度判定部６１２Ｂは、動作信号を受信したタイミングで、このタイミングにおいて監視映像取得制御部６１１Ｂにより取得された監視映像データに基づいて集中度を判定し、この集中度を、予測された遮蔽期間における集中度とする場合を例に説明した。しかしながら、これに限定されない。集中度判定部６１２Ｂは、予め設定される周期に則って、動作信号を受信する直前に判定した集中度を、予測された遮蔽期間における集中度とするようにしてもよい。これにより、動作信号を受信したタイミングでの集中度を、迅速に算出することが可能となる。 (Other examples)
In the second embodiment, for example, the concentration determination unit 612B determines the concentration based on the monitoring video data acquired by the monitoring image acquisition control unit 611B at this timing when the operation signal is received, The case where this concentration level is set as the concentration level in the predicted shielding period has been described as an example. However, it is not limited to this. The concentration degree determination unit 612B may set the concentration degree determined immediately before receiving the operation signal in accordance with a preset period as the concentration degree in the predicted shielding period. As a result, the degree of concentration at the timing when the operation signal is received can be quickly calculated.
　また、上記第２の実施形態では、運転モード切替制御部６１３は、第１の実施形態と同様に、運転モードを切り替える際に集中度を表す情報を取得し、取得した情報に基づき、運転者の集中度が予め設定される集中度を超えるか否かを判定するようにしている。しかしながら、これに限定されない。運転モード切替制御部６１３は、ドライバカメラ７の撮影範囲がステアリングホイール４の支持部材により遮蔽される直前に集中度を表す情報を取得し、取得した情報に基づき、運転者の集中度が予め設定される集中度を超えるか否かを判定するようにしてもよい。 Moreover, in the said 2nd Embodiment, the driving mode switching control part 613 acquires the information showing a concentration degree when switching a driving mode similarly to 1st Embodiment, and a driver | operator is based on the acquired information. It is determined whether or not the degree of concentration exceeds a preset degree of concentration. However, it is not limited to this. The driving mode switching control unit 613 acquires information indicating the concentration level immediately before the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, and the driver's concentration level is set in advance based on the acquired information. You may make it determine whether it exceeds the concentration degree to be performed.
　具体的には、例えば、運転モード切替制御部６１３は、遮蔽予測部６１４から動作信号を受信すると、動作信号を受信したタイミングで集中度を表す情報を取得する。ここで取得する情報は、予め設定されるタイミングで判定された集中度を表す情報であってもよいし、集中度判定部６１２において、動作信号を受信したタイミングで判定された集中度を表す情報であってもよい。 Specifically, for example, when the operation mode switching control unit 613 receives an operation signal from the shielding prediction unit 614, the operation mode switching control unit 613 acquires information indicating the degree of concentration at the timing when the operation signal is received. The information acquired here may be information indicating the degree of concentration determined at a preset timing, or information indicating the degree of concentration determined at the timing when the operation signal is received in the concentration degree determination unit 612. It may be.
　運転モード切替制御部６１３は、取得した情報に基づき、運転者の集中度が予め設定される集中度を超えるか否かを判定する。運転モード切替制御部６１３は、判定結果を記憶ユニット６３に記憶させる。運転モード切替制御部６１３は、ドライバカメラ７がステアリングホイール４によって遮蔽されている際に、運転モードの切り替えが指示される場合には、記憶ユニット６３に記憶されている判定結果を用いて切り替えの可否を判定する。これにより、ドライバカメラ７がステアリングホイール４によって遮蔽されている際に与えられる運転モードの切替指示に備えて判定結果を記憶することが可能となる。 The driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level based on the acquired information. The operation mode switching control unit 613 stores the determination result in the storage unit 63. When the driver camera 7 is shielded by the steering wheel 4 and the operation mode switching control unit 613 is instructed to switch the operation mode, the operation mode switching control unit 613 uses the determination result stored in the storage unit 63 to switch the operation mode. Judgment is made. As a result, the determination result can be stored in preparation for an operation mode switching instruction given when the driver camera 7 is shielded by the steering wheel 4.
　その他、車両の種類、自動運転制御装置の機能、運転モード切替制御装置の制御機能と制御手順および制御内容等についても、この発明の要旨を逸脱しない範囲で種々変形して実施可能である。 In addition, the vehicle type, the function of the automatic operation control device, the control function and control procedure of the operation mode switching control device, and the control contents can be variously modified and implemented without departing from the gist of the present invention.
　要するにこの発明は、上記実施形態そのままに限定されるものではなく、実施段階ではその要旨を逸脱しない範囲で構成要素を変形して具体化できる。また、上記実施形態に開示されている複数の構成要素の適宜な組み合せにより種々の発明を形成できる。例えば、実施形態に示される全構成要素から幾つかの構成要素を削除してもよい。さらに、異なる実施形態に亘る構成要素を適宜組み合せてもよい。 In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.
　上記の実施形態の一部又は全部は、以下の付記のようにも記載され得るが、以下には限られるものではない。　
　（付記１）
　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置であって、
　前記監視カメラにより撮影される監視映像を表す監視映像データを記憶するメモリと、
　前記メモリと接続する少なくとも１つのハードウェアプロセッサと
を具備し、
　前記少なくとも１つのハードウェアプロセッサは、
　　前記メモリに記憶される監視映像データに基づき、前記運転者の集中度を判定し、
　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽されている遮蔽状態か否かを判定し、
　　前記遮蔽状態である場合、前記遮蔽状態となる直前に判定された集中度を、前記遮蔽状態における集中度とするように構成される運転モード切替制御装置。（付記２）
　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置であって、
　前記監視カメラにより撮影される監視映像を表す監視映像データを記憶するメモリと、
　前記メモリと接続する少なくとも１つのハードウェアプロセッサであって、前記少なくとも１つのハードウェアプロセッサは、
　　前記メモリに記憶される監視映像データに基づき、前記運転者の集中度を判定し、
　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽される遮蔽期間を予測し、
　　前記予測した遮蔽期間における集中度を、前記遮蔽期間となる前に取得した監視映像データに基づいて判定するように構成される運転モード切替制御装置。（付記３）
　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する装置が実行する運転モード切替制御方法であって、
　少なくとも１つのハードウェアプロセッサを用いて、前記監視カメラにより撮影される監視映像に基づき、前記運転者の集中度を判定する過程を実行し、
　前記過程は、
　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽されている遮蔽状態か否かを判定する判定過程と、
　　前記遮蔽状態である場合、前記遮蔽状態となる直前に判定された集中度を、前記遮蔽状態における集中度とする判定過程とを備える運転モード切替制御方法。（付記４）
　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する装置が実行する運転モード切替制御方法であって、
　少なくとも１つのハードウェアプロセッサを用いて、前記監視カメラにより撮影される監視映像に基づき、前記運転者の集中度を判定する過程を実行し、
　前記過程は、
　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽される遮蔽期間を予測する予測過程と、
　　前記予測した遮蔽期間における集中度を、前記遮蔽期間となる前に取得した監視映像に基づいて判定する判定過程とを備える運転モード切替制御方法。 A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
A memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
Comprising at least one hardware processor connected to the memory;
The at least one hardware processor comprises:
Based on the monitoring video data stored in the memory, determine the concentration of the driver,
Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, it is determined whether or not the imaging range of the monitoring camera is in a shielding state that is shielded by a support member of the steering wheel;
An operation mode switching control device configured to set the concentration degree determined immediately before entering the shielding state as the concentration degree in the shielding state in the shielding state. (Appendix 2)
A driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
A memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
At least one hardware processor connected to the memory, the at least one hardware processor comprising:
Based on the monitoring video data stored in the memory, determine the concentration of the driver,
Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel;
An operation mode switching control device configured to determine the degree of concentration in the predicted shielding period based on monitoring video data acquired before the shielding period. (Appendix 3)
A driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
Using at least one hardware processor to perform a step of determining the concentration of the driver based on a monitoring image captured by the monitoring camera;
The process is
A determination process for determining whether or not the imaging range of the monitoring camera is in a shielded state that is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
An operation mode switching control method comprising: a determination process in which the concentration degree determined immediately before entering the shielding state is the concentration degree in the shielding state in the shielding state. (Appendix 4)
A driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
Using at least one hardware processor to perform a step of determining the concentration of the driver based on a monitoring image captured by the monitoring camera;
The process is
A prediction process for predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
An operation mode switching control method comprising: a determination process for determining the degree of concentration in the predicted shielding period based on a monitoring video acquired before the shielding period is reached.

Claims (11)

  1. 　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置であって、
    　前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する制御部を具備し、
    　前記制御部は、前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽されている遮蔽状態か否かを判定し、前記遮蔽状態であると判定した場合に、前記遮蔽状態となる直前に判定された集中度を、前記遮蔽状態における集中度とする運転モード切替制御装置。     A driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
    Based on the monitoring video acquired by the monitoring camera, comprising a control unit for determining the concentration of the driver
    The control unit determines whether or not the photographing range of the monitoring camera is in a shielding state where the photographing range of the monitoring camera is shielded based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel. And when it determines with it being the said shielding state, the operation mode switching control apparatus which makes the concentration degree determined immediately before becoming the said shielding state the concentration degree in the said shielding state.
  2. 　前記制御部は、
    　　前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得し、前記取得した監視映像データのうち、前記遮蔽状態において取得された監視映像データを、前記ステアリングセンサで検出される操舵角に基づいて判別する監視映像取得制御部と、
    　　前記遮蔽状態における集中度を、前記遮蔽状態となる直前の非遮蔽期間に取得された監視映像データに基づいて判定する集中度判定部とを有する請求項１記載の運転モード切替制御装置。     The controller is
    Monitoring video data is acquired at a predetermined cycle from the monitoring video acquired by the monitoring camera, and the monitoring video data acquired in the shielding state among the acquired monitoring video data is detected by the steering sensor. A monitoring video acquisition control unit for determining based on a corner;
    The operation mode switching control device according to claim 1, further comprising: a concentration level determination unit that determines a concentration level in the shielding state based on monitoring video data acquired in a non-shielding period immediately before the shielding state.
  3. 　前記制御部は、
    　　前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、
    　　前記監視映像取得制御部により取得された監視映像データのうち、前記遮蔽状態において取得された監視映像データを、前記ステアリングセンサで検出される操舵角に基づいて判別し、前記遮蔽状態における集中度を、前記遮蔽状態となる直前の非遮蔽期間に取得された監視映像データに基づいて判定する集中度判定部とを有する請求項１記載の運転モード切替制御装置。     The controller is
    A monitoring video acquisition control unit for acquiring monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera;
    Among the monitoring video data acquired by the monitoring video acquisition control unit, the monitoring video data acquired in the shielding state is determined based on the steering angle detected by the steering sensor, and the degree of concentration in the shielding state is determined. The operation mode switching control device according to claim 1, further comprising: a concentration degree determination unit that determines based on monitoring video data acquired during a non-shielding period immediately before the shielding state.
  4. 　前記制御部は、
    　　前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、
    　　前記取得された監視映像データに基づいて前記運転者の集中度を判定する集中度判定部と、
    　　前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽状態を判別し、前記遮蔽状態となる直前に前記集中度判定部により判定された集中度を、前記遮蔽状態における集中度として運転モードの切り替え可否を判定する運転モード切替制御部とを有する請求項１記載の運転モード切替制御装置。     The controller is
    A monitoring video acquisition control unit for acquiring monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera;
    A concentration level determination unit that determines the concentration level of the driver based on the acquired monitoring video data;
    Switching the operation mode by determining the shielding state based on the steering angle detected by the steering sensor, and using the concentration degree determined by the concentration degree determination unit immediately before entering the shielding state as the concentration degree in the shielding state. The operation mode switching control device according to claim 1, further comprising an operation mode switching control unit that determines whether or not it is possible.
  5. 　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する運転モード切替制御装置であって、
    　前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する制御部を具備し、
    　前記制御部は、前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽される遮蔽期間を予測し、前記予測した遮蔽期間における集中度を、前記遮蔽期間となる前の非遮蔽期間に取得された監視映像に基づいて判定する運転モード切替制御装置。     A driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
    Based on the monitoring video acquired by the monitoring camera, comprising a control unit for determining the concentration of the driver
    The control unit predicts a shielding period in which an imaging range of the monitoring camera is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, and the prediction The operation mode switching control device that determines the degree of concentration during the shielding period based on the monitoring video acquired in the non-shielding period before the shielding period.
  6. 　前記制御部は、
    　　前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽期間を予測し、前記遮蔽期間の開始時点から予め設定した期間だけ前の時点において動作信号を出力する遮蔽予測部と、
    　　前記動作信号に応じたタイミングで、前記監視カメラにより取得される監視映像から監視映像データを取得する監視映像取得制御部と、
    　　前記動作信号に応じたタイミングで、前記取得された監視映像データに基づいて前記運転者の集中度を判定し、当該集中度を前記予測した遮蔽期間における集中度とする集中度判定部とを有する請求項５記載の運転モード切替制御装置。     The controller is
    A shielding prediction unit that predicts the shielding period based on a steering angle detected by the steering sensor, and outputs an operation signal at a time point that is a predetermined period before the start point of the shielding period;
    A monitoring video acquisition control unit that acquires monitoring video data from the monitoring video acquired by the monitoring camera at a timing according to the operation signal;
    A concentration level determination unit that determines the concentration level of the driver based on the acquired monitoring video data at a timing according to the operation signal and sets the concentration level as the concentration level in the predicted shielding period; The operation mode switching control device according to claim 5.
  7. 　前記制御部は、
    　　前記ステアリングセンサで検出される操舵角に基づいて前記遮蔽期間を予測し、前記遮蔽期間の開始時点から予め設定した期間だけ前の時点において動作信号を出力する遮蔽予測部と、
    　　前記監視カメラにより取得される監視映像から所定の周期で監視映像データを取得する監視映像取得制御部と、
    　　前記取得された監視映像データに基づいて所定の周期で前記運転者の集中度を判定し、前記動作信号を受信すると、前記動作信号を受信する直前に判定した集中度を、前記予測した遮蔽期間における集中度とする集中度判定部とを有する請求項５記載の運転モード切替制御装置。     The controller is
    A shielding prediction unit that predicts the shielding period based on a steering angle detected by the steering sensor, and outputs an operation signal at a time point that is a predetermined period before the start point of the shielding period;
    A monitoring video acquisition control unit for acquiring monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera;
    Based on the acquired monitoring video data, the concentration level of the driver is determined at a predetermined cycle, and when the operation signal is received, the concentration level determined immediately before receiving the operation signal is determined based on the predicted shielding period. The operation mode switching control device according to claim 5, further comprising: a concentration level determination unit that sets a concentration level at.
  8. 　前記制御部は、
    　　前記動作信号を受信すると、前記集中度判定部で判定された集中度に基づいて運転モードの切り替え可否を判定し、当該判定結果を、前記遮蔽期間における判定結果とする運転モード切替制御部をさらに有する請求項６、又は請求項７に記載の運転モード切替制御装置。     The controller is
    When receiving the operation signal, the operation mode switching control unit further determines whether or not the operation mode can be switched based on the concentration level determined by the concentration level determination unit, and sets the determination result as a determination result in the shielding period. The operation mode switching control device according to claim 6 or claim 7.
  9. 　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する装置が実行する運転モード切替制御方法であって、
    　前記装置が、前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する過程を具備し、
    　前記過程は、
    　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽されている遮蔽状態か否かを判定する判定過程と、
    　　前記遮蔽状態である場合、前記遮蔽状態となる直前に判定された集中度を、前記遮蔽状態における集中度とする判定過程とを備える運転モード切替制御方法。     A driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
    The apparatus comprises a step of determining the concentration of the driver based on a monitoring image acquired by the monitoring camera;
    The process is
    A determination process for determining whether or not the imaging range of the monitoring camera is in a shielded state that is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
    An operation mode switching control method comprising: a determination process in which the concentration degree determined immediately before entering the shielding state is the concentration degree in the shielding state in the shielding state.
  10. 　監視カメラが運転者から見てステアリングホイールの裏側に設置されている車両で用いられる、車両の運転モードを切替制御する装置が実行する運転モード切替制御方法であって、
    　前記装置が、前記監視カメラにより取得される監視映像に基づき、前記運転者の集中度を判定する過程を具備し、
    　前記過程は、
    　　前記ステアリングホイールの操舵角を検出するステアリングセンサにより検出される操舵角に基づき、前記監視カメラの撮影範囲が前記ステアリングホイールの支持部材により遮蔽される遮蔽期間を予測する予測過程と、
    　　前記予測した遮蔽期間における集中度を、前記遮蔽期間となる前の非遮蔽期間に取得された監視映像に基づいて判定する判定過程とを備える運転モード切替制御方法。     A driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
    The apparatus comprises a step of determining the concentration of the driver based on a monitoring image acquired by the monitoring camera;
    The process is
    A prediction process for predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
    An operation mode switching control method comprising: a determination process for determining the degree of concentration in the predicted shielding period based on a monitoring image acquired in a non-shielding period before the shielding period.
  11. 　請求項１乃至請求項８の何れかに記載の運転モード切替制御装置が備える前記各部としてコンピュータを機能させるプログラム。 A program that causes a computer to function as each of the units included in the operation mode switching control device according to any one of claims 1 to 8.
PCT/JP2017/036705 2017-03-09 2017-10-10 Driving mode switching control device, method, and program WO2018163492A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-045212 2017-03-09
JP2017045212A JP2018149821A (en) 2017-03-09 2017-03-09 Operation mode changeover control device, method and program

Publications (2)

Publication Number Publication Date
WO2018163492A1 true WO2018163492A1 (en) 2018-09-13
WO2018163492A9 WO2018163492A9 (en) 2018-11-15

Family

ID=63448514

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/036705 WO2018163492A1 (en) 2017-03-09 2017-10-10 Driving mode switching control device, method, and program

Country Status (2)

Country Link
JP (1) JP2018149821A (en)
WO (1) WO2018163492A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113014913A (en) * 2019-12-19 2021-06-22 合肥君正科技有限公司 Method for detecting shielding of monitoring camera in vehicle in segmented mode
CN114179831A (en) * 2021-12-29 2022-03-15 吉林大学 Man-machine steering switching control method based on driver distraction judgment

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7346969B2 (en) * 2019-07-22 2023-09-20 トヨタ自動車株式会社 Driving support device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002254956A (en) * 2001-03-02 2002-09-11 Toyota Industries Corp Device for detecting falling-off awakened condition
JP2016057839A (en) * 2014-09-09 2016-04-21 株式会社デンソーアイティーラボラトリ Facial direction detection device and warning system for vehicle
WO2016132848A1 (en) * 2015-02-16 2016-08-25 株式会社デンソー Driving assistance device and driving assistance method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002254956A (en) * 2001-03-02 2002-09-11 Toyota Industries Corp Device for detecting falling-off awakened condition
JP2016057839A (en) * 2014-09-09 2016-04-21 株式会社デンソーアイティーラボラトリ Facial direction detection device and warning system for vehicle
WO2016132848A1 (en) * 2015-02-16 2016-08-25 株式会社デンソー Driving assistance device and driving assistance method

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113014913A (en) * 2019-12-19 2021-06-22 合肥君正科技有限公司 Method for detecting shielding of monitoring camera in vehicle in segmented mode
CN113014913B (en) * 2019-12-19 2022-09-06 合肥君正科技有限公司 Method for detecting shielding of in-vehicle monitoring camera in segmented mode
CN114179831A (en) * 2021-12-29 2022-03-15 吉林大学 Man-machine steering switching control method based on driver distraction judgment
CN114179831B (en) * 2021-12-29 2023-02-17 吉林大学 Man-machine steering switching control method based on driver distraction judgment

Also Published As

Publication number Publication date
WO2018163492A9 (en) 2018-11-15
JP2018149821A (en) 2018-09-27

Similar Documents

Publication Publication Date Title
CN108140319B (en) Parking assistance method and parking assistance device
RU2656933C2 (en) Method and device for early warning during meeting at curves
US11027746B2 (en) Drive mode switching control device, method and program
US8666599B2 (en) Driving support system
JP4206928B2 (en) Collision possibility judgment device
CN109844842B (en) Driving mode switching control device, system, method and storage medium
JP5585194B2 (en) Accident situation recording system
US20200017123A1 (en) Drive mode switch controller, method, and program
CN110546699B (en) Recommended driving output device, recommended driving output method, and recommended driving output system
JP2015164027A (en) Electronic equipment, control method and control program of electronic equipment
CN109843690B (en) Driving mode switching control device, system and method
WO2018163492A1 (en) Driving mode switching control device, method, and program
JP6528213B2 (en) Vehicle control device, vehicle control method, and vehicle control program
US20210150903A1 (en) Information provision system, server, and mobile terminal
JP7073972B2 (en) Information processing systems, programs, and control methods
CN111469762B (en) Display system, travel control device, display control method, and storage medium
JP2013109639A (en) Image processing system, server, portable terminal device and image processing method
JP7202195B2 (en) DISPLAY SYSTEM, RUNNING CONTROL DEVICE, DISPLAY CONTROL METHOD AND PROGRAM
JP2018149822A (en) Operation mode changeover control device, method and program
JP4830457B2 (en) Navigation device, navigation method, navigation system, and traffic jam information guide program
US11023750B2 (en) Video output system
WO2018163454A1 (en) Drive mode switching control device, method and program
JP2019028481A (en) On-vehicle device and driving support apparatus
WO2018163493A1 (en) Monitoring device, method, and program
WO2024048517A1 (en) Information processing method and information processing device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17899803

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17899803

Country of ref document: EP

Kind code of ref document: A1