WO2022018781A1 - Vehicle control device, program, and vehicle control method - Google Patents

Abstract

The present invention comprises: a driver awareness level specifying unit (160) that specifies a driver awareness level, which is the level of awareness of the driver of a vehicle; a passenger behavior specifying unit (110) that specifies passenger behavior, which is the behavior of a passenger in the vehicle other than the driver; a driver state specifying unit (170) that specifies a driver state, which is the state of the driver, based on a combination of the driver awareness level and the passenger behavior; and a vehicle control unit (180) that controls the vehicle in accordance with the driver state.

Description

Vehicle control devices, programs and vehicle control methods

This disclosure relates to a vehicle control device, a program, and a vehicle control method.

Patent Document 1 describes a driving support device that determines whether or not the passenger is nervous when the direction of the line of sight of the passenger of the vehicle is outside the vehicle or in front of the vehicle. This driving support device supports the driver's driving operation based on the degree of tension of the passenger.

Japanese Unexamined Patent Publication No. 2014-75008

In the driving support device described in Patent Document 1, the direction of the line of sight of the passenger is outside the vehicle or in front of the vehicle, and the driver is not observed. The status information cannot be reflected.

Therefore, the position or a plurality of aspects of the present disclosure is intended to enable more accurate determination of the driver's condition by using the behavior of the passenger when determining the driver's condition.

The vehicle control device according to one aspect of the present disclosure includes a driver consciousness level specifying unit that specifies a driver consciousness level, which is a level of consciousness of the driver of a vehicle, and an action of a passenger other than the driver in the vehicle. A passenger behavior specifying unit that specifies a certain passenger behavior, a driver state specifying unit that specifies the driver state that is the driver's state from the combination of the driver awareness level and the passenger behavior, and the driving. It is characterized by including a vehicle control unit that controls the vehicle according to a person's state.

The vehicle control device according to one aspect of the present disclosure is a combination of a driver consciousness level, which is the level of consciousness of the driver of the vehicle, and a passenger behavior, which is the behavior of a passenger other than the driver in the vehicle, and the driving. From the driver state storage unit that stores driver state-related information associated with the driver state, which is the state of the driver, the driver consciousness level specifying unit that specifies the driver consciousness level from the driver, and the passenger. When the amount of data stored in the passenger behavior specifying unit for specifying the passenger behavior and the driver state storage unit is less than a predetermined threshold value, the driver consciousness level specifying unit is specified. The driver state is specified from the combination of the driver consciousness level and the passenger behavior specified by the passenger behavior specifying unit, and the driver consciousness level and the driver consciousness level specified by the driver consciousness level specifying unit are specified. The combination of the passenger behavior specified by the passenger behavior specifying unit and the specified passenger behavior are associated with each other and stored in the driver state storage unit as the driver state-related information. When the amount of data stored in the driver state storage unit is equal to or greater than a predetermined threshold value, the driver awareness level and the driver awareness level specified by the driver awareness level specifying unit from the driver state-related information The driver state specifying unit that specifies the driver state associated with the combination of the passenger behaviors specified by the passenger behavior specifying unit and the driver state specified by the driver state specifying unit. Accordingly, it is characterized by including a vehicle control unit that controls the vehicle.

The program according to one aspect of the present disclosure uses a computer as a driver consciousness level specifying unit that specifies a driver consciousness level, which is a level of consciousness of the driver of a vehicle, and an action of a passenger other than the driver in the vehicle. A passenger behavior specifying unit that specifies a certain passenger behavior, a driver state specifying unit that specifies the driver state that is the driver's state from the combination of the driver consciousness level and the passenger behavior, and the driving. It is characterized in that it functions as a vehicle control unit that controls the vehicle according to a person's state.

The program according to one aspect of the present disclosure uses a computer as a combination of a driver consciousness level, which is the level of consciousness of the driver of the vehicle, and a passenger behavior, which is the behavior of a passenger other than the driver in the vehicle. A driver state storage unit that stores driver state-related information associated with the driver state, which is the driver's state, a driver consciousness level specifying unit that specifies the driver consciousness level from the driver, and the passenger said The driving specified by the driver consciousness level specifying unit when the amount of data stored in the passenger behavior specifying unit for specifying the passenger behavior and the driver state storage unit is less than a predetermined threshold value. The driver state is specified from the combination of the person consciousness level and the passenger behavior specified by the passenger behavior specifying unit, and the driver consciousness level and the riding passenger specified by the driver consciousness level specifying unit. The combination of the passenger behavior specified by the person behavior specifying unit and the specified passenger behavior are associated with each other and stored in the driver state storage unit as the driver state-related information, and the driving is performed. When the amount of data stored in the person state storage unit is equal to or greater than a predetermined threshold value, the driver consciousness level and the rider specified by the driver consciousness level specifying unit from the driver state related information. According to the driver state specifying unit that specifies the driver state associated with the combination of passenger behaviors specified by the person behavior specifying unit, and the driver state specified by the driver state specifying unit. It is characterized in that it functions as a vehicle control unit that controls the vehicle.

The vehicle control method according to one aspect of the present disclosure specifies the driver consciousness level, which is the level of consciousness of the driver of the vehicle, and specifies the passenger behavior, which is the behavior of passengers other than the driver in the vehicle. It is characterized in that the driver state, which is the state of the driver, is specified from the combination of the driver consciousness level and the passenger behavior, and the vehicle is controlled according to the driver state.

The vehicle control method according to one aspect of the present disclosure specifies the driver consciousness level, which is the level of consciousness of the driver of the vehicle, and specifies the passenger behavior, which is the behavior of passengers other than the driver in the vehicle. , The specified driver consciousness level and the specified passenger behavior are combined to identify the driver state, which is the driver's state, and the specified driver consciousness level and the specified passenger. It is a vehicle control method that stores driver state-related information that associates a combination of actions with the specified driver state, and the amount of data of the stored driver state-related information is equal to or higher than a predetermined threshold value. When this happens, the method of identifying the driver state is associated with the combination of the specified driver awareness level and the specified passenger behavior from the driver state related information. It is characterized by switching to a method of specifying.

According to one or more aspects of the present disclosure, the behavior of the passenger can also be used when determining the state of the driver, so that the state of the driver can be determined more accurately.

It is a block diagram schematically showing the structure of the vehicle control device which concerns on Embodiment 1. FIG. It is a block diagram which shows schematic structure of the passenger behavior identification part in Embodiment 1. FIG. It is a schematic diagram which shows an example of a passenger behavior result table for specifying a passenger behavior. It is a schematic diagram which shows an example of the driver consciousness level result table for specifying the driver consciousness level. It is a schematic diagram which shows an example of the driver state result table for specifying a driver state. It is a block diagram which shows schematic structure of the vehicle control part in Embodiment 1. FIG. (A) and (B) are block diagrams showing a hardware configuration example. It is a state chart diagram which shows an example of the process which controls a vehicle from the specific result in the driver consciousness level specifying part and the passenger behavior specifying part in Embodiment 1. FIG. It is a state chart diagram which shows an example of the processing in a passenger behavior identification part. It is a state chart diagram which shows an example of the processing of the vehicle control part in Embodiment 1. FIG. It is a schematic diagram which shows an example of the inside of the cockpit of a vehicle equipped with a vehicle control device. FIG. 3 is a block diagram schematically showing a configuration of a vehicle control device according to a second embodiment. It is a block diagram which shows schematic structure of the vehicle control part in Embodiment 2. FIG. It is a state chart diagram which shows an example of the process which controls a vehicle from the specific result in the driver consciousness level specifying part and the passenger behavior specifying part in Embodiment 2. FIG. It is a state chart diagram which shows an example of the processing of the vehicle control part in Embodiment 2.

Embodiment 1.
FIG. 1 is a block diagram schematically showing the configuration of the vehicle control device 100 according to the first embodiment.
The vehicle control device 100 includes a passenger behavior specifying unit 110, a driver consciousness level specifying unit 160, a driver state specifying unit 170, and a vehicle control unit 180.

The passenger behavior specifying unit 110 identifies the passenger behavior, which is the behavior of a passenger other than the driver in the vehicle.
For example, the passenger behavior specifying unit 110 identifies the passenger behavior from information about the passenger such as line-of-sight information, voice information, or biological information of the passenger, and gives the specified passenger behavior to the driver state specifying unit 170. ..
Here, it is desirable that the passenger behavior is an behavior performed by the passenger by observing the driver.

FIG. 2 is a block diagram schematically showing the configuration of the passenger behavior specifying unit 110 in the first embodiment.
The passenger behavior specifying unit 110 includes a passenger monitoring unit 120, a load analysis unit 140, a line-of-sight analysis unit 141, a voice analysis unit 142, and a specific unit 146.

The passenger monitoring unit 120 detects the state of the passenger.
The passenger monitoring unit 120 includes a load detection unit 121, a load determination unit 122, a face image acquisition unit 123, a line-of-sight detection unit 124, a line-of-sight determination unit 125, a heart rate detection unit 126, and a heart rate determination unit 127. A sweating amount detection unit 128, a sweating amount determination unit 129, a voice detection unit 130, and a voice determination unit 131 are provided.

The load detection unit 121 detects the value of the load on the passenger seat from the signal detected by the load sensor (not shown) installed in the passenger seat of the vehicle. Then, the load detection unit 121 generates load detection data indicating the value of the load, and gives the load detection data to the load determination unit 122.

The load determination unit 122 gives the load detection data given by the load detection unit 121 to the load analysis unit 140.
Further, the load determination unit 122 gives the load information generated by the load analysis unit 140 to the specific unit 146, as will be described later.

The load analysis unit 140 detects the presence or absence of a passenger by analyzing the load detection data given by the load detection unit 121. Further, the load analysis unit 140 analyzes the load detection data given by the load detection unit 121 when there is a passenger, so that the position of the passenger in the vehicle, the direction in which the load is applied by the passenger, and the load are applied by the passenger. It identifies the magnitude of the load and generates load information indicating the identified position, direction and magnitude. The load information is given to the load determination unit 122, and is given to the specific unit 146 from the load determination unit 122.

For example, when a passenger gets on board, the load sensor installed in the passenger's seat reacts, so that the load analysis unit 140 can detect that the passenger has boarded. Further, even when the passenger shifts the center of gravity to the driver's seat side or the window side, the load analysis unit 140 detects in which direction the passenger shifts the center of gravity from the load status detected by the load sensor. Can be done.

When the passenger gets on the vehicle, the face image acquisition unit 123 identifies the passenger's face from the image data showing the image of the inside of the vehicle, and the face image data which is the data of the face image showing the specified face. To get. The face image data is given to the line-of-sight detection unit 124.

The line-of-sight detection unit 124 detects the line-of-sight of the passenger from the face image indicated by the face image data given by the face image acquisition unit 123. The line-of-sight detection unit 124 generates line-of-sight data indicating the detected line-of-sight, and gives the line-of-sight data to the line-of-sight determination unit 125.

The line-of-sight determination unit 125 gives the line-of-sight data given by the line-of-sight detection unit 124 to the line-of-sight analysis unit 141.
Then, the line-of-sight determination unit 125 gives the line-of-sight information generated by the line-of-sight analysis unit 141 to the specific unit 146, as will be described later.

The line-of-sight analysis unit 141 identifies the direction the passenger is looking at by analyzing the line-of-sight data given by the line-of-sight detection unit 124. Then, the line-of-sight analysis unit 141 generates line-of-sight information indicating the direction the passenger is looking at, and gives the line-of-sight information to the line-of-sight determination unit 125.

The heart rate detection unit 126 detects the heart rate of the passenger and generates heart rate information indicating the detected heart rate. The heart rate information is given to the heart rate determination unit 127.
The heart rate determination unit 127 gives heart rate information to the specific unit 146.

The sweating amount detection unit 128 detects the sweating amount of the passenger and generates sweating amount information indicating the detected sweating amount. The sweating amount information is given to the sweating amount determination unit 129.
The sweating amount determination unit 129 gives the sweating amount information to the specific unit 146.

The voice detection unit 130 detects the voice emitted by the passenger to generate voice data indicating the detected voice. The voice data is given to the voice determination unit 131.
The voice determination unit 131 gives voice data to the voice analysis unit 142.
Further, the voice determination unit 131 gives the voice information generated by the voice analysis unit 142 to the specific unit 146, as will be described later.

The voice analysis unit 142 identifies the content of the voice emitted by the passenger by analyzing the voice data.
The voice analysis unit 142 includes a language analysis unit 143, a dictionary storage unit 144, and a dictionary reference unit 145.

The language analysis unit 143 divides the voice indicated by the voice data into morphemes by performing morphological analysis. Then, the language analysis unit 143 gives the divided morphemes to the dictionary reference unit 145.

The dictionary storage unit 144 stores dictionary data indicating the meaning of each morpheme.
The dictionary reference unit 145 identifies the meaning of each morpheme given by the language analysis unit 143 by referring to the dictionary data stored in the dictionary storage unit 144, and gives the meaning to the language analysis unit 143.

The language analysis unit 143 identifies the content of the voice emitted by the passenger from the meaning of each morpheme, and generates voice information indicating the specified content. The voice information is given to the voice determination unit 131, and the voice determination unit 131 gives the voice information to the specific unit 146. Here, the voice emitted by the passenger includes non-verbal voice such as screaming or sobbing.

As described above, the load detection unit 121, the load determination unit 122, and the load analysis unit 140 constitute a load identification unit 111 that specifies the direction and magnitude of the load in the passenger's seat.
Further, the face image acquisition unit 123, the line-of-sight detection unit 124, the line-of-sight determination unit 125, and the line-of-sight analysis unit 141 constitute a line-of-sight identification unit 112 that specifies the direction of the passenger's line of sight.

The heart rate detection unit 126 and the heart rate determination unit 127 configure a heart rate identification unit 113 for specifying the heart rate of the passenger.
The sweating amount detecting unit 128 and the sweating amount determining unit 129 constitute a sweating amount specifying unit 114 that specifies the sweating amount of the passenger.
It is assumed that the heart rate specifying unit 113 and the sweating amount specifying unit 114 constitute a physical quantity specifying unit that specifies the physical quantity of the passenger's living body. In the first embodiment, the heart rate and the sweating amount are specified as physical quantities related to the living body of the passenger, but the first embodiment is not limited to such an example. For example, blood pressure, respiratory rate, etc. may be specified. The physical quantity specifying unit provides the specific unit 146 with biological information indicating the physical quantity of the passenger's living body.

Further, the voice detection unit 130, the voice determination unit 131, and the voice analysis unit 142 constitute a voice identification unit 115 for specifying the content of the voice emitted by the passenger.
The passenger behavior specifying unit 110 does not have to include all of the load specifying unit 111, the line-of-sight specifying unit 112, the heart rate specifying unit 113, the sweating amount specifying unit 114, and the voice specifying unit 115, and at least any of these. You only need to have one.

The identification unit 146 identifies the passenger behavior, which is the behavior of the passenger, based on the information given from the passenger monitoring unit 120. For example, the identification unit 146 identifies the passenger behavior as shown in FIG. 3 based on the information given from the passenger monitoring unit 120.

FIG. 3 is a schematic diagram showing an example of a passenger behavior result table for specifying passenger behavior.
The passenger action result table 150 shown in FIG. 3 includes a corresponding column 150a, a PID (Passenger Identity) column 150b, and a passenger action column 150c.
The passenger behavior row 150c stores the passenger behavior.
The PID column 150b stores the PID as the passenger behavior identification information which is the identification information for identifying the passenger behavior.
The corresponding column 150a stores whether or not it corresponds to the passenger behavior according to the information given from the passenger monitoring unit 120.

For example, the specific unit 146 has "the center of gravity of the passenger tilted toward the driver", "the center of gravity of the passenger tilts toward the window", and "the passenger stands up from the seat" based on the load information. "Or" the passenger is shaking the driver ". Specifically, the specific unit 146 does not correspond to "the center of gravity of the passenger is tilted toward the driver", "the passenger is standing up from the seat", or "the passenger is shaking the driver". In that case, it may be determined that the center of gravity of the passenger is tilted toward the window.

The identification unit 146 specifies that "the line of sight of the passenger is facing forward" or "the line of sight of the passenger is facing the driver" based on the line-of-sight information. Specifically, the specific unit 146 may determine that "the passenger's line of sight is facing forward" when it does not correspond to "the passenger's line of sight is facing the driver".

The specifying unit 146 specifies that "the passenger's heart rate is high", "the passenger's heart rate is normal", or "the passenger's heart rate is low" based on the heart rate information. For example, the specific unit 146 may make this determination using a predetermined threshold value.

The specific unit 146 specifies that "the passenger's sweating amount is large", "the passenger's sweating amount is normal", or "the passenger's sweating amount is small" based on the sweating amount information. For example, the specific unit 146 may make this determination using a predetermined threshold value.

Based on the voice information, the specific unit 146 "passenger is uttering an arbitrary word", "passenger is uttering a specific word", "passenger is screaming", or "passenger is screaming". The person is not speaking. " Specifically, the specific unit 146 does not correspond to "the passenger is uttering a specific word", "the passenger is screaming", or "the passenger is not speaking". , "The passenger is uttering any word."

The specific unit 146 provides the driver state specific unit 170 with passenger behavior information indicating the passenger behavior specified as described above. Here, when determining the behavior of the passenger, various data such as load, line of sight, heart rate, sweating amount, and voice are adopted, but other data related to the passenger, such as pupil diameter or number of blinks, are used. It may be used.

Here, the specific unit 146 acquires information from all of the load specific unit 111, the line of sight specific unit 112, the heart rate specific unit 113, the sweating amount specific unit 114, and the voice specific unit 115, and specifies the passenger behavior. However, Embodiment 1 is not limited to such an example. If the passenger behavior can be specified from the result specified by at least one of the load specifying unit 111, the line-of-sight specifying unit 112, the heart rate specifying unit 113, the sweating amount specifying unit 114, and the voice specifying unit 115. good.

Returning to FIG. 1, the driver consciousness level specifying unit 160 specifies the driver consciousness level, which is the level of the driver's consciousness. For example, the driver consciousness level specifying unit 160 is shown in FIG. 4 based on at least one of the driver's image, voice, heart rate, and sweating amount, similarly to the passenger behavior specifying unit 110. Identify the driver awareness level shown in the table. The identified driver awareness level is given to the driver state identification unit 170.

FIG. 4 is a schematic diagram showing an example of a driver consciousness level result table for specifying a driver consciousness level.
The driver awareness level result table 151 shown in FIG. 4 includes a corresponding column 151a, a DID (Driver IDentification) column 151b, and a driver awareness level column 151c.

The driver awareness level column 151c stores the driver awareness level.
The DID column 151b stores the DID as the driver consciousness level identification information for identifying the driver consciousness level.
The corresponding column 151a stores whether or not it corresponds to the driver consciousness level determined by the driver consciousness level specifying unit 160.

The driver state specifying unit 170 is a driver's state based on a combination of the passenger behavior given by the passenger behavior specifying unit 110 and the driver consciousness level given by the driver consciousness level specifying unit 160. To identify.
When the passenger is not in the vehicle, the driver state specifying unit 170 may specify the driver state from the driver consciousness level.
For example, the driver state specifying unit 170 identifies the driver state shown in the table shown in FIG. The identified driver state is given to the vehicle control unit 180.

FIG. 5 is a schematic diagram showing an example of a driver state result table for specifying a driver state.
The driver status result table 152 shown in FIG. 5 includes a corresponding row 152a, a JCS (Japan COMA Scale) row 152b, and a driver status row 152c.
The JCS column 152b stores the consciousness level number specified by JCS indicating the driver status.
The driver status column 152c stores the driver status result corresponding to the awareness level number.
The corresponding column 152a stores whether or not the driver state corresponds to the driver state determined by the driver state specifying unit 170.

Here, one or more passenger behaviors specified in the passenger behavior result table shown in FIG. 3 and one or more driving specified in the driver awareness level result table shown in FIG. It is assumed that any of the driver states specified in the driver state table shown in FIG. 5 is associated with the combination with the person consciousness level.
For example, in the passenger behavior result table shown in FIG. 3, the passenger behavior corresponds to two or more items among P01, P03, P04, P05, P07, P09, P11, P12 and P15. Moreover, in the driver consciousness level result table shown in FIG. 4, the driver consciousness level is two of D02, D03, D05, D06, D08, D09, D10, D11, D13, D14, D15 and D16. When it corresponds to one or more items, it corresponds to "300" of JCS shown in FIG. In this case, the driver state specifying unit 170 determines that the driver is unconscious.

In the present embodiment, JCS is adopted as the driver state, but the present embodiment is not limited to such an example. For example, GCS (Glasgow Coma Scale) and other consciousness level indicators may be adopted.
That is, the driver state specifying unit 170 outputs any driver state as shown in FIG. 5 from the combination of the passenger behavior and the driver consciousness level.

The vehicle control unit 180 controls the vehicle according to the driver state given by the driver state specifying unit 170. Here, the vehicle control unit 180 shall control the vehicle safely.
Specifically, the vehicle control unit 180 controls the vehicle when the driver state specified by the driver state specifying unit 170 indicates that the driver cannot normally drive the vehicle. Here, the vehicle control unit 180 stops the vehicle in a safe place.

FIG. 6 is a block diagram schematically showing the configuration of the vehicle control unit 180 according to the first embodiment.
The vehicle control unit 180 includes a road condition acquisition unit 181, a stop position determination unit 182, and a stop unit 183.

The road condition acquisition unit 181 acquires the road condition indicating the road condition. For example, although not shown, the road condition acquisition unit 181 has a road condition around the vehicle from a front camera, a side camera, a sonar, a millimeter wave radar, a LIDAR (Light Detection and Ranking), etc., which are installed in the vehicle. And give the road condition to the stop position determination unit 182.

The stop position determination unit 182 determines a stop position at which the vehicle can be safely stopped according to the road condition given by the road condition acquisition unit 181. Then, the stop position determination unit 182 gives the stop position information indicating the determined stop position to the stop unit 183.

The stop unit 183 stops the vehicle at the stop position indicated by the stop position information given by the stop position determination unit 182.

A part or all of the vehicle control device 100 described above includes, for example, a memory 10 and a CPU (Central Processing) that executes a program stored in the memory 10, as shown in FIG. 7A. It can be configured by a processor 11 such as Unit). Such a program may be provided through a network, or may be recorded and provided on a recording medium. That is, such a program may be provided, for example, as a program product. In such a case, the vehicle control device 100 can be realized by, for example, a computer.

Further, a part or all of the vehicle control device 100 may be, for example, as shown in FIG. 7B, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific ASIC). It can also be configured by a processing circuit 12 such as an Integrated Circuit) or an FPGA (Field Programmable Gate Array).
As described above, the vehicle control device 100 can be realized by the processing network.

FIG. 8 is a state chart diagram showing an example of a process of controlling a vehicle from the specific results of the driver consciousness level specifying unit 160 and the passenger behavior specifying unit 110 in the first embodiment.
First, when the driver gets on board, the load analysis unit 140 of the passenger behavior specifying unit 110 determines whether or not the passenger has boarded based on the value of the load detected by the load detection unit 121 (S10).

Further, the driver consciousness level specifying unit 160 grasps the driver's behavior, specifies the driver consciousness level, and gives the specified driver consciousness level to the driver state specifying unit 170 (S11).

In step S12, when there is a passenger, the passenger behavior specifying unit 110 grasps the behavior of the passenger, identifies the passenger behavior, and drives the passenger behavior information indicating the specified passenger behavior. It is given to the person state specifying unit 170 (S13).

If the driver is on board but the passenger is not on board, it means that there is no passenger in step S12, and the passenger behavior specifying unit 110 does not perform any processing, and the processing is the latter stage of S11. Continue to the step of. Here, it is assumed that the processing in the passenger behavior specifying unit 110 and the processing in the driver consciousness level specifying unit 160 are performed at the same time.

When there is a passenger in step S14, the driver state specifying unit 170 is a driver according to the driver consciousness level given by the driver consciousness level specifying unit 160 and the passenger behavior given by the passenger behavior specifying unit 110. The state is specified (S15). Then, the driver state specifying unit 170 provides the vehicle control unit 180 with the driver state information indicating the specified driver state.

When there is no passenger in step S14, the driver state specifying unit 170 specifies the driver state from the driver consciousness level given by the driver consciousness level specifying unit 160 (S16). Then, the driver state specifying unit 170 provides the vehicle control unit 180 with the driver state information indicating the specified driver state.

Then, after the process of step S15 or step S16, the vehicle control unit 180 controls the vehicle according to the driver state (S17).

FIG. 9 is a state chart diagram showing an example of processing in the passenger behavior specifying unit 110.
FIG. 9 shows the details of the process in step S13 of FIG.

First, the face image acquisition unit 123 acquires a passenger's face image from an image captured by a camera, which is an image pickup unit (not shown) (S20).
Next, the line-of-sight detection unit 124 detects the line-of-sight of the passenger from the acquired face image, generates line-of-sight data indicating the detected line-of-sight, and gives the line-of-sight data to the line-of-sight determination unit 125 (S21). .. The line-of-sight determination unit 125 gives line-of-sight data to the line-of-sight analysis unit 141.

The line-of-sight analysis unit 141 identifies the direction the passenger is looking at by analyzing the line-of-sight data (S22). Then, the line-of-sight analysis unit 141 generates line-of-sight information indicating the specified direction, and gives the line-of-sight information to the line-of-sight determination unit 125. The line-of-sight determination unit 125 gives the line-of-sight information to the specific unit 146.

Further, in the processing related to the heart rate data, the heart rate detection unit 126 detects the heart rate of the passenger and gives the heart rate information indicating the detected heart rate to the heart rate determination unit 127 (S23). The heart rate determination unit 127 gives the heart rate information to the specific unit 146.

Further, in the process related to the sweating amount data, the sweating amount detection unit 128 detects the sweating amount of the passenger and gives the sweating amount information indicating the detected sweating amount to the sweating amount determination unit 129 (S24). The sweating amount determination unit 129 gives the sweating amount information to the specific unit 146.

In the processing related to voice data, the voice detection unit 130 detects the voice of the passenger and generates voice data indicating the detected voice (S25). The voice detection unit 130 gives the generated voice data to the voice determination unit 131. The voice determination unit 131 gives the voice data to the language analysis unit 143.

The language analysis unit 143 identifies the content of the voice emitted by the passenger by analyzing the voice data (S26). Specifically, the language analysis unit 143 divides the voice indicated by the voice data into morphemes by performing morphological analysis. Then, the language analysis unit 143 gives the divided morphemes to the dictionary reference unit 145.
The dictionary reference unit 145 identifies the meaning of each morpheme given by the language analysis unit 143 by referring to the dictionary data stored in the dictionary storage unit 144, and gives the meaning to the language analysis unit 143.
The language analysis unit 143 identifies the content of the voice emitted by the passenger from the meaning of each morpheme, and generates voice information indicating the specified content.
Then, the language analysis unit 143 gives the voice information to the voice determination unit 131, and the voice determination unit 131 gives the voice information to the specific unit 146.

The identification unit 146 identifies the passenger behavior, which is the behavior of the passenger, based on the information given from the passenger monitoring unit 120 (S27).

FIG. 10 is a state chart diagram showing an example of processing of the vehicle control unit 180 in the first embodiment.
FIG. 10 is the process in step S17 of FIG.

First, the road condition acquisition unit 181 determines whether or not the driver state indicated by the driver state information given by the driver state specifying unit 170 is clear consciousness (S30).
Then, in step S31, if the driver state is clear consciousness, the process ends, and if the driver state is not clear consciousness, the process proceeds to step S32.

Next, the road condition acquisition unit 181 acquires the road condition indicating the condition of the surrounding road (S31). The road condition acquisition unit 181 gives the acquired road condition to the stop position determination unit 182.

Then, the stop position determination unit 182 determines a position where the vehicle can be safely stopped based on the given road condition (S33).
Finally, the stop unit 183 stops the vehicle at a position where it can be safely stopped (S34).

FIG. 11 is a schematic view showing an example of the inside of the cockpit 101 of a vehicle in which the vehicle control device 100 is mounted.
The cockpit 101 is provided with one sound collector 102 and one wide-angle imager 103.

The sound collector 102 is a sound collecting device that collects all the sounds from the driver's seat on the right side, the passenger seat as the passenger's seat on the left side, and the passenger's seat behind. The voice signal, which is a voice signal collected by the sound collector 102, is given to the voice detection unit 130. The voice detection unit 130 can identify who utters the voice from the reception direction of the voice signal. As a result, the voice detection unit 130 can identify the voice of the passenger from the voice inside the vehicle.

The wide-angle imager 103 is an image pickup device as an image pickup unit capable of capturing all images of the driver's seat on the right side, the passenger seat on the left side, and the passenger seat behind. The image data of the image captured by the wide-angle imager 103 is given to the face image acquisition unit 123. The face image acquisition unit 123 can identify the face of the passenger from the image shown by the image data.

In the present embodiment, the configuration is such that someone can be identified based on the data from one sound collector 102 and one wide-angle imager 103, but this is the number of sound collector 102 and wide-angle imager 103. This is to reduce costs by making each one one. If the cost is not a concern, the number of sound collectors and imagers may be multiple.

As described above, according to the first embodiment, the driver's state can be specified by the passenger behavior and the driver's consciousness level, and when the driver cannot drive the vehicle normally, the vehicle is safely placed in the stop position. Can be stopped at.

Embodiment 2.
FIG. 12 is a block diagram schematically showing the configuration of the vehicle control device 200 according to the second embodiment.
The vehicle control device 200 includes a passenger behavior specifying unit 110, a driver consciousness level specifying unit 160, a driver state specifying unit 270, a vehicle control unit 280, and a driver state storage unit 290.

The passenger behavior specifying unit 110 and the driver consciousness level specifying unit 160 of the vehicle control device 200 according to the second embodiment are the passenger behavior specifying unit 110 and the driver consciousness level specifying unit of the vehicle control device 100 according to the first embodiment. It is the same as the part 160.

The driver state specifying unit 270 is a driver state, which is a driver's state, based on a combination of the passenger behavior given by the passenger behavior specifying unit 110 and the driver consciousness level given by the driver consciousness level specifying unit 160. To identify.

In the second embodiment, when the amount of data stored in the driver state storage unit 290 is less than a predetermined threshold value and there is a passenger, the driver state identification unit 270 is shown in FIG. Driver Status Results Shown Driver status is identified based on a combination of passenger behavior and driver awareness level so that the driver status shown in Table 152 can be identified. Then, the driver state specifying unit 270 gives the specified driver state to the vehicle control unit 280, and associates the specified driver state with the combination of the passenger behavior and the driver awareness level, and the driver state. It is stored in the driver state storage unit 290 as related information.

Further, when the amount of data stored in the driver state storage unit 290 is less than a predetermined threshold value and there is no passenger, the driver state identification unit 270 is shown in FIG. The driver state is specified based on the driver awareness level so that the driver state shown in the driver state result table 152 can be specified. Then, the driver state specifying unit 270 gives the specified driver state to the vehicle control unit 280, and associates the specified driver state with the driver consciousness level as the driver state-related information. It is stored in the storage unit 290.

On the other hand, when the amount of data stored in the driver state storage unit 290 is equal to or greater than a predetermined threshold value and there is a passenger, the driver state identification unit 270 may perform passenger behavior and driver awareness. The driver state associated with the combination of levels is specified from the driver state related information stored in the driver state storage unit 290. Then, the driver state specifying unit 270 gives the specified driver state to the vehicle control unit 280.

Further, when the amount of data stored in the driver state storage unit 290 is equal to or higher than a predetermined threshold value and there is no passenger, the driver state identification unit 270 is associated with the driver awareness level. The driver state is specified from the driver state-related information stored in the driver state storage unit 290. Then, the driver state specifying unit 270 gives the specified driver state to the vehicle control unit 280.

The driver state storage unit 290 stores the driver state-related information given by the driver state identification unit 270.

The vehicle control unit 280 controls the vehicle according to the driver state given by the driver state specifying unit 270.
The vehicle control unit 280 in the second embodiment automatically sets a nearby destination such as an emergency hospital according to the driver's condition, and automatically drives the vehicle to the set destination.
The vehicle control unit 280 in the second embodiment may stop the vehicle in a safe place when the driver cannot normally drive the vehicle, as in the vehicle control unit 180 in the first embodiment.

FIG. 13 is a block diagram schematically showing the configuration of the vehicle control unit 280 according to the second embodiment.
The vehicle control unit 280 includes a destination automatic setting unit 284, a route search determination unit 285, and a travel control unit 286.

The destination automatic setting unit 284 identifies the position of the vehicle from the INS (Inertial Navigation System) or GPS (Global Positioning System) receiver installed in the vehicle, and maps the position. By referring to the information, the destination is set according to the driver's condition. For example, when the driver's state is the "state in which the driver does not awaken even if stimulated" shown in FIG. 5, the destination automatic setting unit 284 searches for an emergency hospital near the vehicle and searches for the emergency hospital. Set as a destination. Then, the destination automatic setting unit 284 gives the destination information indicating the set destination to the route search determination unit 285. In addition, it is assumed that the type of the destination to be set is predetermined in the destination automatic setting unit 284 for each driver state in which the destination should be set. Then, the destination automatic setting unit 284 may specify the type of the destination according to the driver's state, and select the destination closest to the vehicle from the destinations of the specified type in the map information.

The route search determination unit 285 searches for a route on which the vehicle should travel to the destination indicated by the destination information given by the destination automatic setting unit 284, and generates route information indicating the route. Then, the route search determination unit 285 gives the route information to the travel control unit 286.

The travel control unit 286 drives the vehicle to the destination according to the route information given by the route search determination unit 285.

A part or all of the vehicle control device 200 described above is also, for example, as shown in FIG. 7A, a memory 10 and a processor such as a CPU that executes a program stored in the memory 10. It can be configured by 11.
Further, a part or all of the vehicle control device 200 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or the like, as shown in FIG. 7 (B). It can also be configured by the processing circuit 12 of.
As described above, the vehicle control device 200 can be realized by the processing network.

FIG. 14 is a state chart diagram showing an example of a process of controlling a vehicle from the specific results of the driver consciousness level specifying unit 160 and the passenger behavior specifying unit 110 in the second embodiment.
Among the steps shown in FIG. 14, the steps that perform the same processing as the steps shown in FIG. 8 are designated by the same reference numerals as those in FIG. 8, and detailed description thereof will be omitted.

The process of steps S10 to S13 in FIG. 14 is the same as the process of steps S10 to S13 of FIG. However, after steps S11 and S13 in FIG. 14, the process proceeds to step S44.

In step S44, the driver state specifying unit 270 determines whether or not the amount of data stored in the driver state storage unit 290 is sufficient. If the amount of data is sufficient, the process proceeds to step S45, and if the amount of data is not sufficient, the process proceeds to step S48.

In step S45, the driver state specifying unit 270 determines whether or not there is a passenger. When there is a passenger in step S45, the driver state specifying unit 270 associates the driver consciousness level given by the driver consciousness level specifying unit 160 with the combination of the passenger behavior given by the passenger behavior specifying unit 110. The driver state is specified from the driver state-related information stored in the driver state storage unit 290 (S46). Then, the driver state specifying unit 270 provides the vehicle control unit 280 with the driver state information indicating the specified driver state.

When there is no passenger in step S45, the driver state specifying unit 270 stores the driver state associated with the driver consciousness level given by the driver consciousness level specifying unit 160 in the driver state storage unit 290. It is specified from the driver status related information (S47). Then, the driver state specifying unit 270 provides the vehicle control unit 280 with the driver state information indicating the specified driver state.

In step S48, the driver state specifying unit 270 determines whether or not there is a passenger. When there is a passenger in step S48, the driver state specifying unit 270 is a driver according to the driver consciousness level given by the driver consciousness level specifying unit 160 and the passenger behavior given by the passenger behavior specifying unit 110. The state is specified (S49). Here, the driver state specifying unit 270 identifies the driver state shown in the driver state result table 152 shown in FIG. Then, the driver state specifying unit 270 provides the vehicle control unit 280 with the driver state information indicating the specified driver state.

Next, the driver state specifying unit 270 identifies the driver consciousness level given by the driver consciousness level specifying unit 160 and the passenger behavior given by the passenger behavior specifying unit 110 in step S48. The driver state-related information associated with the state is generated, and the driver state-related information is stored in the driver state storage unit 290 (S50).

When there is no passenger in step S48, the driver state specifying unit 170 specifies the driver state from the driver consciousness level given by the driver consciousness level specifying unit 160 (S51). Here, the driver state specifying unit 270 identifies the driver state shown in the driver state result table 152 shown in FIG. Then, the driver state specifying unit 270 provides the vehicle control unit 280 with the driver state information indicating the specified driver state.

Next, the driver state specifying unit 270 generates driver state-related information in which the driver consciousness level given by the driver consciousness level specifying unit 160 is associated with the driver state specified in step S51. The driver state-related information is stored in the driver state storage unit 290 (S52).

Then, after the processing of step S46, step S47, step S50 or step S52, the vehicle control unit 280 controls the vehicle according to the driver state (S53).

FIG. 15 is a state chart diagram showing an example of processing of the vehicle control unit 280 in the second embodiment.
FIG. 15 is the process in step S53 of FIG.

First, the destination automatic setting unit 284 sets a nearby emergency hospital or the like as the destination according to the driver's condition (S60).

Then, the route search determination unit 285 searches for a route to the set destination and determines the route (S61).
Finally, the travel control unit 286 automatically drives the vehicle safely to the destination according to the determined route (S62).

As described above, according to the second embodiment, the driver state can be specified by the passenger behavior and the driver consciousness level, the destination is automatically set according to the driver state, and the destination is set. The vehicle can be driven automatically.
In the second embodiment, the method of specifying the driver state when the stored data amount of the driver state-related information becomes equal to or more than a predetermined threshold value is specified from the driver state-related information. The processing load can be reduced by switching to a method of identifying the driver state associated with the combination of the driver awareness level and the identified passenger behavior.

It should be noted that the embodiment is not limited to the above embodiment 1 or 2, and can be implemented in various embodiments within a range not deviating from the above gist.

100, 200 vehicle control device, 110 passenger behavior identification unit, 111 load identification unit, 112 line-of-sight identification unit, 113 heart rate identification unit, 114 sweating amount identification unit, 115 voice identification unit, 120 passenger monitoring unit, 121 load detection Unit, 122 load determination unit, 123 face image acquisition unit, 124 line-of-sight detection unit, 125 line-of-sight determination unit, 126 heart rate detection unit, 127 heart rate determination unit, 128 sweating amount detection unit, 129 sweating amount determination unit, 130 voice detection Unit, 131 voice judgment unit, 140 load analysis unit, 141 line-of-sight analysis unit, 142 voice analysis unit, 143 language analysis unit, 144 dictionary storage unit, 145 dictionary reference unit, 146 specific unit, 160 driver awareness level specific unit, 170 , 270 Driver status identification unit, 180, 280 vehicle control unit, 181 road condition acquisition unit, 182 stop position determination unit, 183 stop unit, 284 destination automatic setting unit, 285 route search determination unit, 286 travel control unit, 290 Driver state storage unit.

Claims (13)

1. The driver consciousness level identification unit that specifies the driver consciousness level, which is the level of consciousness of the driver of the vehicle,
   A passenger behavior specifying unit that specifies a passenger behavior that is a behavior of a passenger other than the driver in the vehicle, and
   From the combination of the driver consciousness level and the passenger behavior, the driver state specifying unit that specifies the driver state, which is the driver's state, and the driver state specifying unit.
   A vehicle control device including a vehicle control unit that controls the vehicle according to the driver's state.
2. Driving that associates the combination of the driver consciousness level, which is the level of consciousness of the driver of the vehicle, and the passenger behavior, which is the behavior of passengers other than the driver in the vehicle, with the driver state, which is the state of the driver. A driver state storage unit that stores information related to the person state,
   A driver consciousness level specifying unit that specifies the driver consciousness level from the driver,
   A passenger behavior identification unit that identifies the passenger behavior from the passenger,
   When the amount of data stored in the driver state storage unit is less than a predetermined threshold value, it is specified by the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specifying unit. The driver state is specified from the combination of the passenger behaviors, and the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specified by the passenger behavior specifying unit. The combination of the above and the specified passenger behavior are associated with each other, and the amount of data stored in the driver state storage unit as the driver state-related information is stored in the driver state storage unit. When the value is equal to or higher than a predetermined threshold value, the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specified by the passenger behavior specifying unit are used from the driver state-related information. The driver state specifying unit that specifies the driver state associated with the combination of
   A vehicle control device including a vehicle control unit that controls the vehicle according to the driver state specified by the driver state specifying unit.
3. The vehicle control device according to claim 1 or 2, wherein the passenger behavior is an behavior performed by the passenger by observing the driver.
4. The passenger behavior identification unit
   A load specifying unit that specifies the direction and magnitude of the load in the passenger's seat, a line-of-sight specifying unit that specifies the direction of the passenger's line of sight, a physical quantity specifying unit that specifies the physical quantity of the passenger's living body, and the above. At least one of the voice identification parts that specify the content of the voice emitted by the passenger, and
   It is characterized by including a specific unit that specifies the passenger behavior from the result specified by at least one of the load specifying unit, the line-of-sight specifying unit, the physical quantity specifying unit, and the voice specifying unit. The vehicle control device according to any one of claims 1 to 3.
5. The line-of-sight identification unit identifies the face of the passenger from the image of the inside of the vehicle captured by one imaging device, and specifies the direction of the line of sight of the passenger from the specified face. The vehicle control device according to claim 4.
6. The voice specifying unit is characterized in that the voice of the passenger is specified from the voice inside the vehicle acquired by one sound collecting device, and the content is specified from the specified voice. Item 4. The vehicle control device according to item 4.
7. The vehicle control unit is characterized in that the vehicle control unit controls the vehicle when the driver state specified by the driver state identification unit indicates that the driver cannot normally drive the vehicle. The vehicle control device according to any one of claims 1 to 6.
8. The vehicle control unit
   The road condition acquisition unit that acquires the road condition, which is the condition of the road on which the vehicle travels,
   A stop position determining unit that determines a stop position at which the vehicle can be safely stopped according to the road conditions, and a stop position determining unit.
   The vehicle control device according to any one of claims 1 to 7, further comprising a stop portion for stopping the vehicle at the stop position.
9. The vehicle control unit
   A destination automatic setting unit that sets a destination according to the driver state specified by the driver state specifying unit, and a destination automatic setting unit.
   The route search determination unit that determines the route to the destination,
   The vehicle control device according to any one of claims 1 to 6, further comprising a travel control unit for traveling the vehicle on the route.
10. Computer,
    Driver consciousness level identification unit, which specifies the driver consciousness level, which is the level of consciousness of the driver of the vehicle.
    Passenger behavior identification unit that specifies passenger behavior that is the behavior of passengers other than the driver in the vehicle,
    From the combination of the driver consciousness level and the passenger behavior, the driver state specifying unit that specifies the driver state, which is the driver's state, and the driver state specifying unit, and
    A program characterized by functioning as a vehicle control unit that controls the vehicle according to the driver's state.
11. Computer,
    Driving that associates the combination of the driver consciousness level, which is the level of consciousness of the driver of the vehicle, and the passenger behavior, which is the behavior of passengers other than the driver in the vehicle, with the driver state, which is the state of the driver. Driver state storage unit that stores person state related information,
    Driver consciousness level specifying unit that specifies the driver consciousness level from the driver,
    Passenger behavior identification unit that identifies the passenger behavior from the passenger,
    When the amount of data stored in the driver state storage unit is less than a predetermined threshold value, it is specified by the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specifying unit. The driver state is specified from the combination of the passenger behaviors, and the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specified by the passenger behavior specifying unit. The combination of the above and the specified passenger behavior are associated with each other, and the amount of data stored in the driver state storage unit as the driver state-related information is stored in the driver state storage unit. When the value is equal to or higher than a predetermined threshold value, the driver consciousness level specified by the driver consciousness level specifying unit and the passenger behavior specified by the passenger behavior specifying unit are used from the driver state-related information. The driver state specifying unit that identifies the driver state associated with the combination of, and
    A program characterized by functioning as a vehicle control unit that controls the vehicle according to the driver state specified by the driver state specifying unit.
12. Identify the driver consciousness level, which is the level of consciousness of the driver of the vehicle,
    Identifying passenger behavior, which is the behavior of passengers other than the driver in the vehicle,
    From the combination of the driver consciousness level and the passenger behavior, the driver state, which is the state of the driver, is specified.
    A vehicle control method comprising controlling the vehicle according to the driver state.
13. Identify the driver consciousness level, which is the level of consciousness of the driver of the vehicle,
    Identifying passenger behavior, which is the behavior of passengers other than the driver in the vehicle,
    From the combination of the specified driver awareness level and the specified passenger behavior, the driver state, which is the state of the driver, is specified.
    A vehicle control method for storing driver state-related information relating a combination of the specified driver awareness level and the specified passenger behavior to the specified driver state.
    When the amount of data of the stored driver state-related information exceeds a predetermined threshold value, the method for specifying the driver state is described by the specified driver awareness level and the specified passenger. A vehicle control method comprising switching to a method of identifying the driver state associated with a combination of actions.

Images