JP7361929B2 - Vehicle control device, program and vehicle control method - Google Patents

Description

The present disclosure relates to a vehicle control device, a program, and a vehicle control method.

Patent Document 1 describes a driving support device that determines whether or not a passenger in a vehicle is nervous when the passenger's line of sight is outside the vehicle or in front of the vehicle. This driving support device supports the driver's driving operations based on the degree of tension of the passenger.

Japanese Patent Application Publication No. 2014-75008

In the driving support device described in Patent Document 1, the direction of the passenger's line of sight is outside the vehicle or in front of the vehicle, and does not observe the driver. Unable to reflect status information.

Therefore, an object of the position or a plurality of aspects of the present disclosure is to use the behavior of a fellow passenger when determining the driver's condition, thereby making it possible to more accurately determine the driver's condition.

A vehicle control device according to an aspect of the present disclosure includes a driver consciousness level identification unit that identifies a driver consciousness level that is a consciousness level of a driver of a vehicle, and a driver consciousness level identification unit that identifies a driver consciousness level that is a consciousness level of a driver of a vehicle; a passenger behavior identifying unit that identifies a certain passenger behavior; a driver state identifying unit that identifies a driver state that is a state of the driver from a combination of the driver consciousness level and the passenger behavior; The present invention is characterized by comprising: a vehicle control section that controls the vehicle according to a driver's state.

A vehicle control device according to an aspect of the present disclosure provides a combination of a driver consciousness level, which is a level of consciousness of a driver of a vehicle, and fellow passenger behavior, which is a behavior of a fellow passenger other than the driver in the vehicle, and a driver state storage unit that stores driver state related information that associates the driver state with the driver state, a driver state storage unit that stores the driver state related information that associates the driver state with the driver state; When the amount of data stored in the passenger behavior identifying unit that identifies the passenger behavior and the driver state storage unit is less than a predetermined threshold, the driver consciousness level identifying unit identifies the passenger behavior. The driver condition is specified from the combination of the driver consciousness level and the passenger behavior specified by the passenger behavior identification section, and the driver consciousness level and the passenger behavior specified by the driver consciousness level identification section are determined. The combination of the passenger behavior specified by the passenger behavior identification unit and the identified passenger behavior are associated and stored in the driver status storage unit as the driver status related information, When the amount of data stored in the driver state storage unit is equal to or greater than a predetermined threshold, the driver consciousness level and a driver state identifying unit that identifies the driver state associated with the combination of passenger behaviors identified by the fellow passenger behavior identifying unit; and a driver state identified by the driver state identifying unit. Accordingly, the present invention is characterized by comprising a vehicle control section that controls the vehicle.

A program according to an aspect of the present disclosure includes a driver consciousness level identification unit that identifies a driver consciousness level, which is a consciousness level of a driver of a vehicle, and a driver consciousness level identification unit that identifies a driver consciousness level that is a consciousness level of a driver of a vehicle; a passenger behavior identification unit that identifies a certain passenger behavior; a driver state identification unit that identifies a driver state that is a state of the driver from a combination of the driver consciousness level and the passenger behavior; The present invention is characterized in that the vehicle control section functions as a vehicle control section that controls the vehicle according to the driver's state.

A program according to an aspect of the present disclosure causes a computer to determine a combination of a driver consciousness level, which is a level of consciousness of a driver of a vehicle, and a fellow passenger behavior, which is a behavior of a fellow passenger other than the driver in the vehicle, and the a driver state storage unit that stores driver state related information that associates the driver state with a driver state; a driver consciousness level identification unit that identifies the driver consciousness level from the driver; A passenger behavior identifying unit that identifies passenger behavior, and when the amount of data stored in the driver state storage unit is less than a predetermined threshold, the driving that is identified by the driver consciousness level identifying unit; The driver condition is specified from a combination of the passenger's consciousness level and the fellow passenger's behavior specified by the passenger's behavior specifying section, and the driver's consciousness level specified by the driver's consciousness level specifying section and the fellow passenger's condition are determined. The combination of the passenger behavior identified by the passenger behavior identification unit and the identified passenger behavior are associated and stored in the driver status storage unit as the driver status related information, When the amount of data stored in the driver state storage unit is equal to or greater than a predetermined threshold, the driver consciousness level and the fellow passenger identified by the driver consciousness level identification unit from the driver state related information are determined. a driver state identification unit that identifies the driver state associated with the combination of passenger behavior identified by the driver behavior identification unit; and and functions as a vehicle control section that controls the vehicle.

In a vehicle control method according to an aspect of the present disclosure, a computer identifies a driver consciousness level that is a consciousness level of a driver of a vehicle, and a passenger behavior that is a behavior of a fellow passenger other than the driver in the vehicle. The present invention is characterized in that a driver condition, which is a condition of the driver, is specified from a combination of the driver consciousness level and the passenger behavior, and the vehicle is controlled according to the driver condition. .

In a vehicle control method according to an aspect of the present disclosure, a computer identifies a driver consciousness level that is a consciousness level of a driver of a vehicle, and a passenger behavior that is a behavior of a fellow passenger other than the driver in the vehicle. from the combination of the identified driver consciousness level and the identified passenger behavior, the driver state that is the state of the driver is determined, and the driver state that is the state of the driver is determined. A vehicle control method for storing driver condition related information that associates a combination of passenger behaviors and the identified driver condition, wherein the computer stores a data amount of the stored driver condition related information. A method for identifying the driver state when the driver state exceeds a predetermined threshold is determined by determining the driver state associated with the combination of the identified driver consciousness level and the identified passenger behavior. The method is characterized by switching to a method for specifying.

According to one or more aspects of the present disclosure, by using the behavior of a fellow passenger when determining the driver's condition, the driver's condition can be determined more accurately.

1 is a block diagram schematically showing the configuration of a vehicle control device according to Embodiment 1. FIG. FIG. 2 is a block diagram schematically showing the configuration of a fellow passenger behavior identifying section in the first embodiment. FIG. 3 is a schematic diagram showing an example of a fellow passenger behavior result table for specifying fellow passenger behavior. FIG. 2 is a schematic diagram showing an example of a driver consciousness level result table for specifying a driver consciousness level. FIG. 3 is a schematic diagram showing an example of a driver condition result table for specifying a driver condition. FIG. 2 is a block diagram schematically showing the configuration of a vehicle control section in Embodiment 1. FIG. (A) and (B) are block diagrams showing examples of hardware configurations. 5 is a state chart diagram illustrating an example of a process for controlling a vehicle based on the identification results of a driver consciousness level identification unit and a fellow passenger behavior identification unit in the first embodiment; FIG. FIG. 7 is a state chart diagram illustrating an example of processing in a fellow passenger behavior identifying section. FIG. 3 is a state chart diagram showing an example of processing of a vehicle control unit in the first embodiment. FIG. 1 is a schematic diagram showing an example of the inside of a cockpit of a vehicle in which a vehicle control device is mounted. FIG. 2 is a block diagram schematically showing the configuration of a vehicle control device according to a second embodiment. FIG. 2 is a block diagram schematically showing the configuration of a vehicle control section in Embodiment 2. FIG. FIG. 7 is a state chart diagram showing an example of a process for controlling a vehicle based on the identification results of a driver consciousness level identification unit and a fellow passenger behavior identification unit in Embodiment 2; FIG. 7 is a state chart diagram illustrating an example of processing by a vehicle control unit in Embodiment 2. FIG.

Embodiment 1.
FIG. 1 is a block diagram schematically showing the configuration of a vehicle control device 100 according to the first embodiment.
The vehicle control device 100 includes a passenger behavior identification section 110, a driver consciousness level identification section 160, a driver state identification section 170, and a vehicle control section 180.

The fellow passenger behavior identification unit 110 identifies fellow passenger behavior that is the behavior of a fellow passenger other than the driver in the vehicle.
For example, the fellow passenger behavior identification unit 110 identifies fellow passenger behavior from information regarding the passenger such as line of sight information, voice information, or biological information of the fellow passenger, and provides the identified fellow passenger behavior to the driver state identification unit 170. .
Here, it is preferable that the fellow passenger's behavior is an action that the fellow passenger performs by observing the driver.

FIG. 2 is a block diagram schematically showing the configuration of fellow passenger behavior identifying section 110 in the first embodiment.
The fellow passenger behavior identification unit 110 includes a fellow passenger monitoring unit 120 , a load analysis unit 140 , a line of sight analysis unit 141 , a voice analysis unit 142 , and a identification unit 146 .

The fellow passenger monitoring unit 120 detects the condition of the fellow passenger.
The passenger monitoring unit 120 includes a load detection unit 121 , a load determination unit 122 , a face image acquisition unit 123 , a line of sight detection unit 124 , a line of sight determination unit 125 , a heart rate detection unit 126 , and a heart rate determination unit 127 , a sweat amount detection section 128 , a sweat amount determination section 129 , a voice detection section 130 , and a voice determination section 131 .

The load detection unit 121 detects the value of the load on the passenger seat from a signal detected by a load sensor (not shown) installed in the passenger seat of the vehicle. Then, the load detection section 121 generates load detection data indicating the value of the load, and provides the load detection data to the load determination section 122.

The load determination unit 122 provides the load detection data provided from the load detection unit 121 to the load analysis unit 140.
Moreover, the load determination unit 122 provides load information generated by the load analysis unit 140 to the identification unit 146, as described later.

The load analysis section 140 analyzes the load detection data provided from the load detection section 121 to detect the presence or absence of a fellow passenger. Further, when there is a fellow passenger, the load analysis unit 140 analyzes the load detection data provided from the load detection unit 121 to determine the position of the fellow passenger in the vehicle, the direction in which the load is applied by the fellow passenger, and The size of the load is specified, and load information indicating the specified position, direction, and size is generated. The load information is given to the load determining section 122, and from the load determining section 122 to the specifying section 146.

For example, when a fellow passenger gets on the vehicle, a load sensor installed in the fellow passenger's seat reacts, so the load analysis unit 140 can detect that the fellow passenger gets on the vehicle. Furthermore, even if the passenger shifts his or her center of gravity toward the driver's seat or the window, the load analysis unit 140 detects in which direction the passenger's center of gravity has shifted from the state of the load detected by the load sensor. Can be done.

When a fellow passenger gets on the vehicle, the face image acquisition unit 123 identifies the face of the fellow passenger from image data showing an image captured inside the vehicle, and generates face image data that is facial image data showing the identified face. get. The face image data is provided to the line of sight detection section 124.

The line of sight detection unit 124 detects the line of sight of the fellow passenger from the face image indicated by the face image data provided from the face image acquisition unit 123. The line-of-sight detection unit 124 generates line-of-sight data indicating the detected line-of-sight, and provides the line-of-sight data to the line-of-sight determination unit 125.

The line of sight determination unit 125 provides the line of sight data provided from the line of sight detection unit 124 to the line of sight analysis unit 141.
Then, the line-of-sight determination unit 125 provides line-of-sight information generated by the line-of-sight analysis unit 141 to the identification unit 146, as described later.

The line of sight analysis unit 141 specifies the direction in which the fellow passenger is looking by analyzing the line of sight data provided from the line of sight detection unit 124. Then, the line-of-sight analysis unit 141 generates line-of-sight information indicating the direction in which the fellow passenger is looking, and provides the line-of-sight information to the line-of-sight determination unit 125.

The heart rate detection unit 126 detects the heart rate of the passenger and generates heart rate information indicating the detected heart rate. The heart rate information is given to the heart rate determination section 127.
Heart rate determining section 127 provides heart rate information to identifying section 146 .

The sweat amount detection unit 128 detects the sweat amount of the passenger and generates sweat amount information indicating the detected sweat amount. The sweat amount information is provided to the sweat amount determining section 129.
The sweat amount determining section 129 provides sweat amount information to the specifying section 146.

The voice detection unit 130 generates voice data indicating the detected voice by detecting the voice emitted by the fellow passenger. The audio data is provided to the audio determination section 131.
The voice determination section 131 provides voice data to the voice analysis section 142.
Furthermore, the voice determination unit 131 provides the voice information generated by the voice analysis unit 142 to the identification unit 146, as will be described later.

The voice analysis unit 142 specifies the content of the voice uttered by the passenger by analyzing the voice data.
The speech analysis section 142 includes a language analysis section 143, a dictionary storage section 144, and a dictionary reference section 145.

The language analysis unit 143 performs morphological analysis on the speech represented by the speech data, thereby dividing the speech into morphemes. The language analysis unit 143 then provides the divided morphemes to the dictionary reference unit 145.

The dictionary storage unit 144 stores dictionary data indicating the meaning of each morpheme.
The dictionary reference unit 145 specifies the meaning of each morpheme provided from the language analysis unit 143 by referring to the dictionary data stored in the dictionary storage unit 144, and provides the meaning to the language analysis unit 143.

The language analysis unit 143 identifies the content of the voice uttered by the fellow passenger based on the meaning of each morpheme, and generates audio information indicating the identified content. The voice information is given to the voice determining section 131, and the voice determining section 131 gives the voice information to the specifying section 146. Here, the sounds emitted by the passenger include sounds that cannot be put into words, such as screams or sobs.

As described above, the load detection section 121, the load determination section 122, and the load analysis section 140 constitute the load identification section 111 that identifies the direction and magnitude of the load on the passenger's seat.
Furthermore, the face image acquisition section 123, the line of sight detection section 124, the line of sight determination section 125, and the line of sight analysis section 141 constitute a line of sight specifying section 112 that specifies the direction of the fellow passenger's line of sight.

The heart rate detection section 126 and the heart rate determination section 127 constitute a heart rate identification section 113 that identifies the heart rate of the passenger.
The sweat amount detection section 128 and the sweat amount determination section 129 constitute a sweat amount identification section 114 that identifies the sweat amount of the fellow passenger.
It is assumed that the heart rate identifying section 113 and the sweat amount identifying section 114 constitute a physical quantity identifying section that identifies a physical quantity related to the living body of the fellow passenger. In the first embodiment, the heart rate and the amount of perspiration are specified as the physical quantities related to the living body of the fellow passenger, but the first embodiment is not limited to such an example. For example, blood pressure or respiratory rate may be specified. The physical quantity specifying unit provides the specifying unit 146 with biological information indicating a physical quantity related to the living body of the fellow passenger.

Furthermore, the voice detection section 130, the voice determination section 131, and the voice analysis section 142 constitute a voice identification section 115 that identifies the content of the voice uttered by the fellow passenger.
Note that the passenger behavior identification unit 110 does not need to include all of the load identification unit 111, the line of sight identification unit 112, the heart rate identification unit 113, the sweat amount identification unit 114, and the audio identification unit 115; It is sufficient to have one of these.

The specifying unit 146 specifies fellow passenger behavior, which is the behavior of the fellow passenger, based on information provided from the fellow passenger monitoring unit 120. For example, the identification unit 146 identifies fellow passenger behavior as shown in FIG. 3 based on information provided from the fellow passenger monitoring unit 120.

FIG. 3 is a schematic diagram showing an example of a fellow passenger behavior result table for specifying fellow passenger behavior.
The passenger behavior result table 150 shown in FIG. 3 includes a corresponding column 150a, a PID (Passenger IDentification) column 150b, and a fellow passenger behavior column 150c.
The fellow passenger behavior column 150c stores fellow passenger behaviors.
The PID column 150b stores PIDs as fellow passenger behavior identification information, which is identification information for identifying fellow passenger behaviors.
The applicable column 150a stores whether or not the behavior corresponds to a fellow passenger's behavior, according to information provided from the fellow passenger monitoring unit 120.

For example, based on the load information, the identification unit 146 determines that "the passenger's center of gravity is leaning toward the driver," "the passenger's center of gravity is leaning toward the window," or "the passenger is standing up from the seat." ” or “a passenger is shaking the driver.” Specifically, the identification unit 146 determines that "the passenger's center of gravity is tilted toward the driver", "the passenger is standing up from the seat", or "the passenger is shaking the driver" does not apply. In this case, it may be determined that the passenger's center of gravity is leaning toward the window.

The specifying unit 146 specifies, based on the line-of-sight information, that "the fellow passenger's line of sight is facing forward" or that "the fellow passenger's line of sight is facing the driver." Specifically, the identification unit 146 may determine that "the fellow passenger's line of sight is facing forward" when "the fellow passenger's line of sight is facing the driver" does not apply.

The specifying unit 146 specifies, based on the heart rate information, that "the fellow passenger's heart rate is high", "the fellow passenger's heart rate is normal", or "the fellow passenger's heart rate is low". For example, the identifying unit 146 may make this determination using a predetermined threshold.

The specifying unit 146 specifies, based on the sweating amount information, that "the fellow passenger's sweating amount is large", "the fellow passenger's sweating amount is normal", or "the fellow passenger's sweating amount is low". For example, the identifying unit 146 may make this determination using a predetermined threshold.

Based on the audio information, the identification unit 146 determines whether "the fellow passenger is uttering an arbitrary word," "the fellow passenger is uttering a specific word," "the fellow passenger is screaming," or "the fellow passenger is screaming." Identify that the person is not speaking. Specifically, the identification unit 146 determines that if the following conditions do not apply: "The passenger is uttering specific words," "The passenger is screaming," or "The passenger is not speaking." , it can be determined that ``the passenger is uttering arbitrary words.''

The identification unit 146 provides the driver state identification unit 170 with passenger behavior information indicating the passenger behavior identified as described above. In this case, various data such as load, line of sight, heart rate, amount of sweat, and voice were used to judge the behavior of the passenger, but other data related to the passenger, such as pupil diameter or number of blinks, may also be used. may be used.

Here, the identifying unit 146 acquires information from all of the load identifying unit 111, the line of sight identifying unit 112, the heart rate identifying unit 113, the sweat amount identifying unit 114, and the audio identifying unit 115, and identifies the fellow passenger's behavior. However, the first embodiment is not limited to such an example. If the identification unit 146 can identify the fellow passenger's behavior from the results identified by at least one of the load identification unit 111, the line of sight identification unit 112, the heart rate identification unit 113, the sweat amount identification unit 114, and the sound identification unit 115, good.

Returning to FIG. 1, the driver consciousness level identification unit 160 identifies the driver consciousness level, which is the driver's consciousness level. For example, similar to the passenger behavior identifying unit 110, the driver consciousness level identification unit 160 determines the level of consciousness shown in FIG. Identify the driver awareness level shown in the table. The identified driver consciousness level is provided to the driver condition identifying section 170.

FIG. 4 is a schematic diagram showing an example of a driver consciousness level result table for specifying the driver consciousness level.
The driver awareness level result table 151 shown in FIG. 4 includes a corresponding column 151a, a DID (Driver IDentification) column 151b, and a driver awareness level column 151c.

The driver consciousness level column 151c stores driver consciousness levels.
The DID column 151b stores DIDs as driver consciousness level identification information for identifying the driver consciousness level.
The applicable column 151a stores information as to whether or not the driver's consciousness level corresponds to the driver's consciousness level determined by the driver's consciousness level specifying unit 160.

The driver state identification unit 170 determines the driver state, which is the state of the driver, from the combination of the passenger behavior given from the fellow passenger behavior identification unit 110 and the driver consciousness level given from the driver consciousness level identification unit 160. Identify.
Note that if there is no passenger in the vehicle, the driver condition identifying unit 170 may identify the driver condition based on the driver's consciousness level.
For example, the driver state identification unit 170 identifies the driver states shown in the table shown in FIG. 5 . The identified driver state is provided to vehicle control unit 180.

FIG. 5 is a schematic diagram showing an example of a driver condition result table for specifying the driver condition.
The driver condition result table 152 shown in FIG. 5 includes a corresponding column 152a, a JCS (Japan Coma Scale) column 152b, and a driver condition column 152c.
The JCS column 152b stores the consciousness level number designated by the JCS indicating the driver's condition.
The driver status column 152c stores driver status results corresponding to consciousness level numbers.
The applicable column 152a stores whether the driver condition corresponds to the driver condition determined by the driver condition identification unit 170.

Here, one or more passenger behaviors specified in the passenger behavior result table shown in FIG. 3 and one or more driving behaviors specified in the driver awareness level result table shown in FIG. It is assumed that one of the driver states specified in the driver state table shown in FIG. 5 is associated with the driver awareness level.
For example, in the passenger behavior result table shown in FIG. 3, the passenger behavior corresponds to two or more items among P01, P03, P04, P05, P07, P09, P11, P12, and P15, In addition, in the driver awareness level result table shown in FIG. If the item falls under one or more of the following items, it corresponds to "300" of the JCS shown in FIG. In this case, the driver state identification unit 170 determines that the driver is unconscious.

In this embodiment, JCS is adopted as the driver state, but this embodiment is not limited to such an example. For example, GCS (Glasgow Coma Scale) and other consciousness level indicators may be employed.
That is, the driver state specifying unit 170 outputs one of the driver states shown in FIG. 5 based on the combination of the fellow passenger's behavior and the driver's consciousness level.

Vehicle control section 180 controls the vehicle according to the driver condition given from driver condition identification section 170. Here, it is assumed that the vehicle control unit 180 safely controls the vehicle.
Specifically, the vehicle control unit 180 controls the vehicle when the driver condition identified by the driver condition identifying unit 170 indicates that the driver cannot normally drive the vehicle. Here, the vehicle control unit 180 stops the vehicle at a safe location.

FIG. 6 is a block diagram schematically showing the configuration of vehicle control section 180 in the first embodiment.
The vehicle control section 180 includes a road condition acquisition section 181 , a stop position determination section 182 , and a stop section 183 .

The road condition acquisition unit 181 acquires road conditions indicating road conditions. For example, although not shown, the road condition acquisition unit 181 acquires the road condition around the vehicle from a front camera, side camera, sonar, millimeter wave radar, LIDAR (Light Detection and Ranging), etc. installed on the vehicle. is acquired, and the road condition is provided to the stop position determination unit 182.

The stop position determination unit 182 determines a stop position at which the vehicle can be safely stopped, depending on the road condition given from the road condition acquisition unit 181. Then, the stop position determination unit 182 provides stop position information indicating the determined stop position to the stop unit 183.

The stopping unit 183 stops the vehicle at the stopping position indicated by the stopping position information provided from the stopping position determining unit 182.

Part or all of the vehicle control device 100 described above includes, for example, a memory 10 and a CPU (Central Processing) that executes programs stored in the memory 10, as shown in FIG. 7(A). It can be configured by a processor 11 such as a processor unit. Such a program may be provided through a network, or may be provided recorded on a recording medium. That is, such a program may be provided as a program product, for example. In such a case, vehicle control device 100 can be implemented by, for example, a computer.

In addition, part or all of the vehicle control device 100 may include, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific), as shown in FIG. 7(B). It can also be configured with a processing circuit 12 such as an integrated circuit (integrated circuit) or a field programmable gate array (FPGA).
As described above, the vehicle control device 100 can be realized by a processing circuit network.

FIG. 8 is a state chart diagram illustrating an example of a process for controlling the vehicle based on the identification results of the driver consciousness level identification unit 160 and passenger behavior identification unit 110 in the first embodiment.
First, when the driver gets on the vehicle, the load analysis section 140 of the fellow passenger behavior identification section 110 determines whether a fellow passenger gets on the vehicle based on the value of the load detected by the load detection section 121 (S10).

Further, the driver consciousness level identification unit 160 grasps the behavior of the driver, identifies the driver consciousness level, and provides the identified driver consciousness level to the driver state identification unit 170 (S11).

In step S12, if there is a fellow passenger, the fellow passenger behavior identifying unit 110 grasps the fellow passenger's behavior, identifies the fellow passenger's behavior, and transmits the passenger behavior information indicating the identified fellow passenger's behavior to the driver. The information is given to the user state specifying unit 170 (S13).

Note that if the driver gets in the car but no fellow passenger gets on board, it is determined in step S12 that there is no fellow passenger, and the fellow passenger behavior identifying unit 110 does not perform any processing, and the process is performed after S11. Continue to the next step. Here, it is assumed that the processing by the fellow passenger behavior identifying section 110 and the processing by the driver consciousness level identifying section 160 are performed simultaneously.

If there is a fellow passenger in step S14, the driver state specifying unit 170 determines whether the driver is in a state of The state is specified (S15). Then, driver condition identifying section 170 provides driver condition information indicating the identified driver condition to vehicle control section 180.

If there is no passenger present in step S14, the driver state identification unit 170 identifies the driver state based on the driver awareness level given from the driver awareness level identification unit 160 (S16). Then, driver condition identifying section 170 provides driver condition information indicating the identified driver condition to vehicle control section 180.

After the processing in step S15 or step S16, the vehicle control unit 180 controls the vehicle according to the driver's condition (S17).

FIG. 9 is a state chart diagram illustrating an example of processing in the fellow passenger behavior identifying unit 110.
FIG. 9 shows details of the process in step S13 in FIG.

First, the facial image acquisition unit 123 acquires a facial image of the passenger from an image captured by a camera, which is an image capturing unit (not shown) (S20).
Next, the line of sight detection unit 124 detects the line of sight of the fellow passenger from the acquired facial image, generates line of sight data indicating the detected line of sight, and provides the line of sight data to the line of sight determination unit 125 (S21). . The line of sight determination section 125 provides line of sight data to the line of sight analysis section 141 .

The line of sight analysis unit 141 identifies the direction in which the fellow passenger is looking by analyzing the line of sight data (S22). Then, the line-of-sight analysis unit 141 generates line-of-sight information indicating the specified direction, and provides the line-of-sight information to the line-of-sight determination unit 125. The line-of-sight determining unit 125 provides the line-of-sight information to the identifying unit 146.

In addition, in the process related to heart rate data, the heart rate detection unit 126 detects the heart rate of the passenger and provides heart rate information indicating the detected heart rate to the heart rate determination unit 127 (S23). Heart rate determining section 127 provides the heart rate information to identifying section 146 .

In addition, in the process related to sweat amount data, the sweat amount detection section 128 detects the sweat amount of the fellow passenger, and provides sweat amount information indicating the detected sweat amount to the sweat amount determination section 129 (S24). The sweat amount determining section 129 provides the sweat amount information to the specifying section 146.

In the process related to voice data, the voice detection unit 130 detects the voice of the fellow passenger and generates voice data indicating the detected voice (S25). The voice detection section 130 provides the generated voice data to the voice determination section 131. The speech determination section 131 provides the speech data to the language analysis section 143.

The language analysis unit 143 specifies the content of the voice uttered by the fellow passenger by analyzing the voice data (S26). Specifically, the language analysis unit 143 performs morphological analysis on the speech represented by the speech data, thereby dividing the speech into morphemes. The language analysis unit 143 then provides the divided morphemes to the dictionary reference unit 145.
The dictionary reference unit 145 specifies the meaning of each morpheme provided from the language analysis unit 143 by referring to the dictionary data stored in the dictionary storage unit 144, and provides the meaning to the language analysis unit 143.
The language analysis unit 143 identifies the content of the voice uttered by the fellow passenger based on the meaning of each morpheme, and generates audio information indicating the identified content.
The language analysis unit 143 then provides the audio information to the audio determination unit 131, and the audio determination unit 131 provides the audio information to the identification unit 146.

The specifying unit 146 specifies fellow passenger behavior, which is the behavior of the fellow passenger, based on information provided from the fellow passenger monitoring unit 120 (S27).

FIG. 10 is a state chart diagram illustrating an example of processing by vehicle control unit 180 in the first embodiment.
FIG. 10 shows the process in step S17 in FIG.

First, the road condition acquisition unit 181 determines whether the driver status indicated by the driver status information provided from the driver status identification unit 170 is clear (S30).
Then, in step S31, if the driver's state is clear, the process ends, and if the driver's state is not clear, the process advances to step S32.

Next, the road condition acquisition unit 181 acquires road conditions indicating the conditions of surrounding roads (S31). The road condition acquisition section 181 provides the acquired road condition to the stop position determination section 182.

Then, the stop position determining unit 182 determines a position where the vehicle can safely stop based on the given road conditions (S33).
Finally, the stopping unit 183 stops the vehicle at a position where it can be stopped safely (S34).

FIG. 11 is a schematic diagram showing an example of the inside of a cockpit 101 of a vehicle in which the vehicle control device 100 is mounted.
The cockpit 101 is provided with one sound collector 102 and one wide-angle imager 103.

The sound collector 102 is a sound collecting device that collects all sounds from the driver's seat on the right side, the front passenger seat as a passenger seat on the left side, and the rear passenger seat. The audio signal, which is the audio signal collected by the sound collector 102, is provided to the audio detection unit 130. It is assumed that the voice detection unit 130 can identify who uttered the voice based on the receiving direction of the voice signal. Thereby, the voice detection unit 130 can identify the voice of the fellow passenger from the voice inside the vehicle.

The wide-angle imager 103 is an imaging device that functions as an imaging unit that can capture images of the right driver's seat, the left front passenger seat, and the rear passenger seat. Image data of the image captured by the wide-angle imager 103 is provided to the face image acquisition unit 123. It is assumed that the face image acquisition unit 123 can identify the face of the fellow passenger from the image indicated by the image data.

In this embodiment, the configuration is such that it is possible to identify someone based on data from one sound collector 102 and one wide-angle imager 103. This is to reduce costs by using one for each. If cost is not a concern, a plurality of sound collectors and a plurality of imagers may be provided.

As described above, according to Embodiment 1, the driver's condition can be identified based on passenger behavior and the driver's awareness level, and when the driver is unable to drive the vehicle normally, the vehicle can be safely brought to a stopped position. can be stopped.

Embodiment 2.
FIG. 12 is a block diagram schematically showing the configuration of vehicle control device 200 according to the second embodiment.
The vehicle control device 200 includes a passenger behavior identifying section 110, a driver consciousness level identifying section 160, a driver state identifying section 270, a vehicle control section 280, and a driver state storage section 290.

Passenger behavior identifying section 110 and driver consciousness level identifying section 160 of vehicle control device 200 according to the second embodiment are identical to passenger behavior identifying section 110 and driver consciousness level identifying section 160 of vehicle control device 100 according to embodiment 1. This is similar to section 160.

The driver state identification unit 270 determines the driver state, which is the state of the driver, from the combination of the passenger behavior given from the fellow passenger behavior identification unit 110 and the driver consciousness level given from the driver consciousness level identification unit 160. Identify.

In the second embodiment, when the amount of data stored in driver state storage section 290 is less than a predetermined threshold and there is a passenger, driver state identification section 270 performs the process shown in FIG. The driver condition is specified based on the combination of passenger behavior and driver consciousness level so that the driver condition shown in the displayed driver condition result table 152 can be specified. Then, the driver state identification unit 270 provides the identified driver state to the vehicle control unit 280, and associates the identified driver state with the combination of passenger behavior and driver consciousness level to determine the driver state. It is stored in the driver state storage unit 290 as related information.

Further, when the amount of data stored in the driver state storage unit 290 is less than a predetermined threshold and there is no passenger, the driver state identification unit 270 The driver condition is specified based on the driver's consciousness level so that the driver condition shown in the driver condition result table 152 can be specified. Then, the driver state identification unit 270 provides the identified driver state to the vehicle control unit 280, associates the identified driver state with the driver consciousness level, and sets the driver state as driver state related information. The information is stored in the storage unit 290.

On the other hand, if the amount of data stored in the driver state storage unit 290 is greater than or equal to a predetermined threshold and there is a passenger, the driver state identification unit 270 determines the passenger behavior and driver consciousness. The driver condition associated with the combination of levels is specified from the driver condition related information stored in the driver condition storage section 290. Then, the driver state identification section 270 provides the identified driver state to the vehicle control section 280.

Further, if the amount of data stored in the driver state storage unit 290 is equal to or greater than a predetermined threshold and there is no passenger, the driver state identification unit 270 determines whether the data amount is associated with the driver consciousness level or not. The current driver state is specified from the driver state related information stored in the driver state storage unit 290. Then, the driver state identification section 270 provides the identified driver state to the vehicle control section 280.

The driver condition storage section 290 stores driver condition related information given from the driver condition identification section 270.

Vehicle control section 280 controls the vehicle according to the driver condition given from driver condition identification section 270.
The vehicle control unit 280 in the second embodiment automatically sets a nearby destination, such as an emergency hospital, according to the driver's condition, and automatically drives the vehicle to the set destination.
Note that similarly to vehicle control unit 180 in Embodiment 1, vehicle control unit 280 in Embodiment 2 may stop the vehicle in a safe place when the driver is unable to drive the vehicle normally.

FIG. 13 is a block diagram schematically showing the configuration of vehicle control section 280 in the second embodiment.
The vehicle control unit 280 includes an automatic destination setting unit 284, a route search determining unit 285, and a travel control unit 286.

The automatic destination setting unit 284 identifies the vehicle's position from an INS (Inertial Navigation System) or a GPS (Global Positioning System) receiver installed in the vehicle, and displays the location on a map. By referring to the information, a destination can be set according to the driver's condition. For example, when the driver's state is "not awakened even by stimulation" as shown in FIG. 5, the automatic destination setting unit 284 searches for an emergency hospital near the vehicle, Set as destination. Then, the automatic destination setting unit 284 provides destination information indicating the set destination to the route search determining unit 285. It is assumed that the automatic destination setting section 284 has predetermined the type of destination to be set for each driver state for which a destination is to be set. Then, the automatic destination setting unit 284 specifies the type of destination according to the driver state, and selects the destination closest to the vehicle from among the destinations of the specified type in the map information.

The route search determining unit 285 searches for a route on which the vehicle should travel to the destination indicated by the destination information given from the automatic destination setting unit 284, and generates route information indicating the route. The route search determining unit 285 then provides the route information to the travel control unit 286.

The travel control section 286 causes the vehicle to travel to the destination according to the route information given from the route search determining section 285.

Part or all of the vehicle control device 200 described above also includes a memory 10 and a processor such as a CPU that executes a program stored in the memory 10, as shown in FIG. 7(A), for example. 11.
Further, a part or all of the vehicle control device 200 may be implemented using a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, etc., as shown in FIG. 7(B), for example. The processing circuit 12 can also be configured as follows.
As described above, the vehicle control device 200 can be realized by a processing circuit network.

FIG. 14 is a state chart diagram illustrating an example of a process for controlling the vehicle based on the identification results of the driver consciousness level identification unit 160 and passenger behavior identification unit 110 in the second embodiment.
Among the steps shown in FIG. 14, the steps that perform the same processing as the steps shown in FIG. 8 are given the same reference numerals as in FIG. 8, and detailed explanation thereof will be omitted.

The processing from steps S10 to S13 in FIG. 14 is similar to the processing from steps S10 to S13 in FIG. 8. However, after steps S11 and S13 in FIG. 14, the process proceeds to step S44.

In step S44, driver condition identification section 270 determines whether the amount of data stored in driver condition storage section 290 is sufficient. If the amount of data is sufficient, the process proceeds to step S45, and if the amount of data is not sufficient, the process proceeds to step S48.

In step S45, the driver state identification unit 270 determines whether or not there is a fellow passenger. If there is a fellow passenger in step S45, the driver state identification unit 270 associates the driver state identification unit 270 with the combination of the driver awareness level given from the driver awareness level identification unit 160 and the fellow passenger behavior given from the fellow passenger behavior identification unit 110. The current driver state is specified from the driver state related information stored in the driver state storage unit 290 (S46). Then, driver condition identifying section 270 provides driver condition information indicating the identified driver condition to vehicle control section 280.

If there is no passenger present in step S45, the driver state identification unit 270 stores the driver state associated with the driver consciousness level given from the driver consciousness level identification unit 160 in the driver state storage unit 290. It is specified from the driver condition related information (S47). Then, driver condition identifying section 270 provides driver condition information indicating the identified driver condition to vehicle control section 280.

In step S48, the driver state identification unit 270 determines whether or not there is a fellow passenger. If there is a fellow passenger in step S48, the driver state specifying unit 270 determines whether the driver The state is specified (S49). Here, the driver state identification unit 270 identifies the driver state shown in the driver state result table 152 shown in FIG. Then, driver condition identifying section 270 provides driver condition information indicating the identified driver condition to vehicle control section 280.

Next, the driver state identification unit 270 converts the driver awareness level given from the driver awareness level identification unit 160 and the fellow passenger behavior given from the fellow passenger behavior identification unit 110 to the driver state identified in step S48. The driver state related information associated with the state is generated, and the driver state related information is stored in the driver state storage unit 290 (S50).

If there is no fellow passenger in step S48, the driver state identification unit 170 identifies the driver state based on the driver awareness level given from the driver awareness level identification unit 160 (S51). Here, the driver state identification unit 270 identifies the driver state shown in the driver state result table 152 shown in FIG. Then, driver condition identifying section 270 provides driver condition information indicating the identified driver condition to vehicle control section 280.

Next, the driver condition specifying unit 270 generates driver condition related information that associates the driver consciousness level given from the driver consciousness level specifying unit 160 with the driver condition specified in step S51. The driver condition related information is stored in the driver condition storage section 290 (S52).

After the processing in step S46, step S47, step S50, or step S52, the vehicle control unit 280 controls the vehicle according to the driver state (S53).

FIG. 15 is a state chart diagram illustrating an example of processing by vehicle control unit 280 in the second embodiment.
FIG. 15 shows the process at step S53 in FIG.

First, the automatic destination setting unit 284 sets a nearby emergency hospital or the like as the destination according to the driver's condition (S60).

Then, the route search determining unit 285 searches for a route to the set destination and determines the route (S61).
Finally, the travel control unit 286 causes the vehicle to safely and automatically travel to the destination according to the determined route (S62).

As described above, according to the second embodiment, it is possible to identify the driver's condition based on passenger behavior and the driver's awareness level, automatically set the destination according to the driver's condition, and Vehicles can be driven automatically up to the point in time.
Embodiment 2 describes a method for identifying the driver's condition from the driver's condition-related information when the data amount of the stored driver's condition-related information exceeds a predetermined threshold. By switching to a method of identifying the driver state associated with the combination of the driver consciousness level and the identified passenger behavior, the processing load can be reduced.

Note that the embodiment is not limited to the first or second embodiment described above, and can be implemented in various ways within the scope of the above gist.

100,200 vehicle control device, 110 passenger behavior identification unit, 111 load identification unit, 112 line of sight identification unit, 113 heart rate identification unit, 114 perspiration amount identification unit, 115 voice identification unit, 120 fellow passenger monitoring unit, 121 load detection 122 load determination unit, 123 face image acquisition unit, 124 line of sight detection unit, 125 line of sight determination unit, 126 heart rate detection unit, 127 heart rate determination unit, 128 sweat amount detection unit, 129 sweat amount determination unit, 130 voice detection 131 Voice determination unit, 140 Load analysis unit, 141 Line of sight analysis unit, 142 Voice analysis unit, 143 Language analysis unit, 144 Dictionary storage unit, 145 Dictionary reference unit, 146 Identification unit, 160 Driver consciousness level identification unit, 170 , 270 driver condition identification unit, 180, 280 vehicle control unit, 181 road condition acquisition unit, 182 stop position determination unit, 183 stop unit, 284 automatic destination setting unit, 285 route search determination unit, 286 travel control unit, 290 Driver status memory unit.

Claims (13)

a driver consciousness level identification unit that identifies a driver consciousness level that is a level of consciousness of the driver of the vehicle;
a passenger behavior identification unit that identifies a passenger behavior that is the behavior of a fellow passenger other than the driver in the vehicle;
a driver state identification unit that identifies a driver state that is a state of the driver from a combination of the driver consciousness level and the passenger behavior;
A vehicle control device comprising: a vehicle control unit that controls the vehicle according to the driver state. Driving in which a combination of a driver consciousness level, which is the level of consciousness of the driver of a vehicle, and passenger behavior, which is the behavior of a passenger other than the driver in the vehicle, and a driver state, which is the state of the driver. a driver status storage unit that stores driver status related information;
a driver consciousness level identification unit that identifies the driver consciousness level from the driver;
a fellow passenger behavior identification unit that identifies the fellow passenger behavior from the fellow passenger;
When the amount of data stored in the driver state storage unit is less than a predetermined threshold, the driver consciousness level specified by the driver consciousness level identification unit and the passenger behavior identification unit The driver condition is specified from the combination of the passenger behavior determined, and the driver consciousness level specified by the driver consciousness level identification unit and the fellow passenger behavior specified by the passenger behavior identification unit are identified. and the identified passenger behavior are stored in the driver state storage unit as the driver state related information, and the amount of data stored in the driver state storage unit is If the level is equal to or higher than a predetermined threshold, the driver consciousness level specified by the driver consciousness level specifying unit and the fellow passenger behavior specified by the fellow passenger behavior specifying unit from the driver condition related information. a driver state identifying unit that identifies the driver state associated with the combination;
A vehicle control device comprising: a vehicle control unit that controls the vehicle according to the driver condition specified by the driver condition identification unit. The vehicle control device according to claim 1 or 2, wherein the fellow passenger behavior is an action performed by the fellow passenger when the fellow passenger observes the driver. The passenger behavior identification unit includes:
a load identifying unit that identifies the direction and magnitude of the load on the seat of the fellow passenger; a line of sight identifying unit that identifies the direction of the passenger's line of sight; a physical quantity identifying unit that identifies a physical quantity related to the living body of the fellow passenger; At least one of a voice identifying section that identifies the content of the voice uttered by a fellow passenger;
and a specifying section that specifies the fellow passenger's behavior from the result specified by at least one of the load specifying section, the line of sight specifying section, the physical quantity specifying section, and the sound specifying section. The vehicle control device according to any one of claims 1 to 3. The line of sight identifying unit specifies the face of the passenger from an image of the interior of the vehicle captured by one imaging device, and specifies the direction of the passenger's line of sight from the identified face. The vehicle control device according to claim 4. A claim characterized in that the voice identifying unit identifies the voice of the passenger from the voice inside the vehicle acquired by one sound collection device, and identifies the content from the identified voice. Item 4. Vehicle control device according to item 4. The vehicle control unit controls the vehicle when the driver condition specified by the driver condition identification unit indicates that the driver cannot normally drive the vehicle. The vehicle control device according to any one of claims 1 to 6. The vehicle control unit includes:
a road condition acquisition unit that acquires a road condition that is a condition of a road on which the vehicle travels;
a stop position determination unit that determines a stop position where the vehicle can be safely stopped according to the road condition;
The vehicle control device according to any one of claims 1 to 7, further comprising: a stop unit that stops the vehicle at the stop position. The vehicle control unit includes:
an automatic destination setting unit that sets a destination according to the driver status specified by the driver status identification unit;
a route search determining unit that determines a route to the destination;
The vehicle control device according to any one of claims 1 to 6, further comprising: a travel control unit that causes the vehicle to travel on the route. computer,
a driver consciousness level identification unit that identifies a driver consciousness level that is the level of consciousness of the driver of the vehicle;
a passenger behavior identification unit that identifies passenger behavior that is the behavior of a fellow passenger other than the driver in the vehicle;
a driver state identifying unit that identifies a driver state that is a state of the driver from a combination of the driver consciousness level and the passenger behavior;
A program that functions as a vehicle control unit that controls the vehicle according to the driver state. computer,
Driving in which a combination of a driver consciousness level, which is the level of consciousness of the driver of a vehicle, and passenger behavior, which is the behavior of a passenger other than the driver in the vehicle, and a driver state, which is the state of the driver. a driver status storage unit that stores driver status related information;
a driver consciousness level identification unit that identifies the driver consciousness level from the driver;
a fellow passenger behavior identification unit that identifies the fellow passenger behavior from the fellow passenger;
When the amount of data stored in the driver state storage unit is less than a predetermined threshold, the driver consciousness level specified by the driver consciousness level identification unit and the passenger behavior identification unit The driver condition is specified from the combination of the passenger behavior determined, and the driver consciousness level specified by the driver consciousness level identification unit and the fellow passenger behavior specified by the passenger behavior identification unit are identified. and the identified passenger behavior are stored in the driver state storage unit as the driver state related information, and the amount of data stored in the driver state storage unit is If the level is equal to or higher than a predetermined threshold, the driver consciousness level specified by the driver consciousness level specifying unit and the fellow passenger behavior specified by the fellow passenger behavior specifying unit from the driver condition related information. a driver state identifying unit that identifies the driver state associated with the combination;
A program that functions as a vehicle control section that controls the vehicle according to the driver state specified by the driver state identification section. The computer is
Identify the driver consciousness level, which is the level of consciousness of the driver of the vehicle,
Identifying passenger behavior that is the behavior of a passenger other than the driver in the vehicle;
Identifying a driver state that is a state of the driver from a combination of the driver consciousness level and the passenger behavior,
A vehicle control method, comprising: controlling the vehicle according to the driver state. The computer is
Identify the driver consciousness level, which is the level of consciousness of the driver of the vehicle,
Identifying passenger behavior that is the behavior of a passenger other than the driver in the vehicle;
Identifying a driver condition that is a condition of the driver from a combination of the identified driver consciousness level and the identified passenger behavior,
A vehicle control method that stores driver condition related information that associates a combination of the specified driver consciousness level and the specified passenger behavior with the specified driver condition, the method comprising:
The computer determines a method for specifying the driver condition when the amount of data of the stored driver condition related information exceeds a predetermined threshold. A method for controlling a vehicle, comprising: switching to a method for specifying the driver state associated with a combination of passenger actions that have been performed.

Images