WO2020003392A1

WO2020003392A1 - Driving assistance device and driving mode assessment model generation device

Info

Publication number: WO2020003392A1
Application number: PCT/JP2018/024277
Authority: WO
Inventors: ▲イ▼ 景; 昌彦谷本
Original assignee: 三菱電機株式会社
Priority date: 2018-06-27
Filing date: 2018-06-27
Publication date: 2020-01-02
Also published as: JP6746043B2; JPWO2020003392A1

Abstract

Provided is a driving assistance device capable of assisting in the appropriate assessment of a driving mode taking into consideration both the driver status and the vehicle surrounding environment. Also provided is a driving mode assessment model generation device for generating a driving mode assessment model suitable for use in the driving assistance device. A driving assistance device 1 is equipped with: a surrounding environmental information output unit 13 for obtaining vehicle surrounding environmental information; a driver status information output unit 14 for obtaining driver status information corresponding to the surrounding environmental information; and a driving mode evaluation unit 15 for calculating an evaluation value representing the suitability of automatic driving and manual driving for a combination of the surrounding environmental information and the driver status information, the calculation carried out on the basis of such combination.

Description

Driving support device and driving mode judgment model generation device

The present invention relates to a driving support device that supports setting of a driving mode suitable for a situation in a vehicle having an automatic driving function, and a device that generates a driving mode determination model used for the driving support device.

When a vehicle having an automatic driving function is driven, it is necessary to appropriately switch between automatic driving and manual driving in various traffic conditions, that is, to select an appropriate driving mode.

As a technique for selecting an appropriate driving mode, for example, in Patent Document 1, it is determined whether a driver can return to manual driving based on a result of detecting a driver's action and a driver's state in a vehicle that is being driven automatically. A technique for performing this is disclosed. Similarly, Patent Literature 1 discloses a method of determining whether or not automatic driving can be continued in a vehicle that is being automatically driven based on travel position information, map data, and vehicle travel information of the vehicle.

JP 2018-5343 A

In Patent Literature 1, it is determined whether or not it is possible to return to the manual driving based only on the driver's state. It can be different. For example, if the driver is a little tired, it is possible to drive manually on a straight road with few pedestrians or other vehicles, but there are complicated lane branches and pedestrians and other In a situation where many vehicles exist, it is difficult to manually drive the vehicle unless the driver can pay close attention. Similarly, the determination as to whether or not to allow automatic driving may depend on not only the surrounding environment of the vehicle but also the state of the driver.
As described above, it is desirable to determine whether or not automatic driving and manual driving are possible, that is, to determine the driving mode based on a combination of the driver's state and the surrounding environment of the vehicle. In the case of automatic driving, the driving is performed based only on the environment around the vehicle.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and a first object of the present invention is to provide support for appropriately determining a driving mode based on both a driver's state and a surrounding environment of a vehicle. It is intended to obtain a driving support device capable of performing the following.
A second object is to provide a driving mode determination model generation device that generates a driving mode determination model suitable for use in the driving support device.

In the driving support device according to the present invention, a surrounding environment information output unit that outputs surrounding environment information of a vehicle, a driver state information output unit that outputs state information of a driver corresponding to the surrounding environment information, and a surrounding environment information And a driving mode evaluation unit that calculates an evaluation value indicating the suitability of the automatic driving and the manual driving for the combination based on the combination of the information and the driver state information.

The device of the present invention uses this evaluation value to calculate an evaluation value representing the suitability of automatic driving and manual driving for the combination based on the combination of the surrounding environment information of the vehicle and the driver state information, It can assist in determining an appropriate driving mode according to the surrounding environment and the driver's condition.

FIG. 1 is a configuration diagram illustrating a configuration of a driving support device and a driving mode determination model generation device according to a first embodiment. Schematic diagram illustrating learning data stored in a learning data storage unit provided in the driving mode determination model generation device. FIG. 1 is a configuration diagram illustrating a hardware configuration of a driving control device, an information server, a surrounding environment detection unit, and a driver state detection unit according to Embodiment 1. 4 is a flowchart showing the operation of the driving control device including the driving support device according to the first embodiment. Schematic diagram illustrating the surrounding environment of a vehicle in an example according to the first embodiment. Schematic diagram illustrating a numerical vector generated by a numerical vectorization unit provided in the driving support device according to the first embodiment. Schematic diagram illustrating driver feature values extracted by a driver state feature value extraction unit provided in the driving support device according to the first embodiment. Schematic diagram illustrating a driving mode judgment model stored in a judgment mode storage unit provided in the driving support device according to the first embodiment. Schematic diagram illustrating an evaluation value calculated by a driving mode evaluation unit provided in the driving support device according to the first embodiment. 5 is a flowchart illustrating the operation of the information server including the driving mode determination model generation device according to the first embodiment. Configuration diagram showing configurations of a driving support device and a driving mode determination model generation device according to Embodiment 2. 5 is a flowchart showing the operation of an operation control device including a driving assistance device according to Embodiment 2 and an information server including an operation mode determination model generation device.

Embodiment 1 FIG.
FIG. 1 is a configuration diagram showing a configuration of a driving support device 1 and a driving mode determination model generation device 2 according to Embodiment 1 for carrying out the present invention. The driving support device 1 is provided as a part of a driving control device 3 that controls driving of a vehicle, and the driving mode determination model generating device 2 is a part of an information server 4 that appropriately provides various information services to the vehicle. It is provided as. In the first embodiment, the operation control device 3 is mounted on a vehicle, and the information server 4 is installed at an arbitrary location and connected to the operation control device 3 mounted on the vehicle via a communication line.

The driving control device 3 includes a driving control unit 10 that performs overall driving control of the vehicle. The driving control unit 10 performs control of the vehicle based on input information, including switching between automatic driving and manual driving of the vehicle. Operation control is performed by controlling drive system devices such as deceleration and steering.
In addition, the driving control device 3 includes a driving support device 1, a driving mode candidate selection unit 7, a driving mode presenting unit 8, a driving mode input unit 9, a vehicle communication, as a configuration related to support for determining a driving mode according to the present invention. A section 11 is provided.
Further, a surrounding environment detection unit 5 and a driver state detection unit 6 are connected to the driving support device 1 via a communication line.

The surrounding environment detection unit 5 is mounted on a vehicle or installed in an urban area or a road on which the vehicle runs, and detects a surrounding environment such as the presence or absence and position of a person, a vehicle, a structure, or the like around the vehicle, and provides a driving assistance device. 1 Specifically, the surrounding environment detection unit 5 detects surrounding environment information of the vehicle, and outputs surrounding environment detection data representing the detected surrounding environment to the surrounding environment information output unit 13, and a plurality of surrounding environment detection units may be provided. . Examples of the surrounding environment detected by the surrounding environment detecting unit 5 include pedestrians and other vehicles, structures existing on roads, obstacles, and the like. The surrounding environment detecting unit 5 includes various sensors that detect these, for example, , A camera, Lidar (Light Detection and Ranging), Radar (Radio Detection and Ranging), and a communication device that outputs information detected by the sensor to the peripheral environment information output unit 13 as peripheral environment detection data. The surrounding environment detection unit 5 only needs to be able to detect the surrounding environment of the vehicle, and may be mounted on the vehicle or may be installed in a facility such as a traffic light installed on a road. For example, if the sensor provided in the surrounding environment detection unit 5 is a camera mounted on a vehicle, image information obtained by imaging the surroundings from the vehicle can be obtained. If the sensor is a Lidar, Radar, or the like installed on a traffic signal, It is also possible to obtain detection information of pedestrians, other traveling vehicles, obstacles, and the like around a traffic signal near a vehicle, and in particular, detection information of a target that cannot be imaged by being hidden by another object from a camera installed in the vehicle. .

The driver status detection unit 6 is mounted on the vehicle, detects the status of the driver, and informs the driving support device 1 of the detected status. Specifically, the driver status detection unit 6 detects the status of the driver, and outputs driver status detection data representing the detected driver status to the driver status information output unit 14. There may be. Examples of the driver state acquired by the driver state detection unit 6 include electrocardiogram, myoelectricity, eye movement, brain wave, breathing, pressure, sweating, and the like. The driver state detection unit 6 detects these. It is composed of a biosensor and a camera, and a communication device that outputs detected information to the driver status information output unit 14 as driver status detection data.

When driving a vehicle having an automatic driving function, the driving support device 1 takes into account the driver's state transmitted from the driver state detection unit 6 and the surrounding environment of the vehicle transmitted from the surrounding environment detection unit 5. A device that appropriately switches between automatic operation and manual operation, that is, a device that assists in selecting an appropriate operation mode. Specifically, it represents the suitability of each of automatic operation and manual operation according to the current situation. The evaluation value is calculated and presented. Since the calculated evaluation value is calculated based on the driver's state and the surrounding environment of the vehicle, the evaluation value is a useful judgment material when the driver or the system controlling the driving mode determines the driving mode. The detailed configuration of the driving support device 1 will be described later.

The driving mode candidate selection unit 7 selects an appropriate driving mode candidate based on the evaluation value of the driving mode calculated by the driving support device 1. The selected driving mode candidate and the driving control unit 10 apply the selected driving mode candidate. The current driving modes are compared, and if they match, the fact that the current driving mode is appropriate is presented to the driving mode presentation unit 8 as driving support information, and if they do not match, they are selected as driving support information. The driving mode presenting unit 8 presents the content that prompts the user to switch to the driving mode candidate. The driving mode presenting unit 8 includes, for example, a speaker that outputs audio, a display that displays a screen, and the like.
The driving mode candidate selecting unit 7 also provides a signal for recommending forcible switching to the driving control unit 10 when urgent switching of the driving mode is required, for example, when the vehicle is rapidly approaching a road facility or the like. Is output, whereby the operation control unit 10 can forcibly switch the operation mode.

The driving mode input unit 9 is for inputting whether or not the driver should switch the driving mode based on the driving mode candidates presented by the driving mode presentation unit 8. Here, when the driver inputs to switch the driving mode, a signal indicating the switching to the driving mode candidate is output to the driving control unit 10.

(4) When the signal output from the operation mode input unit 9 indicates that the operation mode is to be switched, the operation control unit 10 switches the operation mode and performs operation control according to the switched operation mode. The operation control unit 10 may forcibly switch the operation mode based on a signal from the operation mode candidate selection unit 7 that recommends the forcible switching.

The vehicle communication unit 11 communicates with the server communication unit 12 provided in the information server 4, and exchanges information between the operation control device 3 and the information server 4. In the present embodiment, particularly, a configuration is shown in which the operation mode determination model generated by the operation mode determination model generation device 2 is acquired and output to the determination model storage unit 16.

The operation control device 3 is configured as described above. Further, the configuration of the driving support device 1 of the present invention included in the driving control device 3 will be described.

As shown in FIG. 1, the driving support device 1 includes a surrounding environment information output unit 13, a driver state information output unit 14, a driving mode evaluation unit 15, and a judgment model storage unit 16.

The surrounding environment information output unit 13 generates surrounding environment information in a format suitable for determining the driving mode based on the surrounding environment detection data of the vehicle obtained from the surrounding environment detecting unit 5, and outputs the generated surrounding environment information to the driving mode evaluation unit 15. Is what you do. More specifically, the peripheral environment information output unit 13 includes a peripheral environment description unit 17, a morpheme extraction unit 18, and a numerical vectorization unit 19.

The surrounding environment description unit 17 uses the surrounding environment detection data obtained by the surrounding environment detection unit 5 to recognize pedestrians, other vehicles, obstacles, and the like, and to describe the surrounding environment as a character string in a natural language. This is for generating column information. The generated character string information is output to the morpheme extraction unit 18.

The morpheme extraction unit 18 performs a morpheme analysis on the character string information generated by the peripheral environment description unit 17, extracts morphemes, and outputs the extracted morphemes to the numerical vectorization unit 19. Here, the morphological analysis is a language processing technique that divides a text into morphemes, which are minimum units having a meaning in a language. For example, a character string "three pedestrians ahead" is divided into "", "", "", "", "", "", "".

The numerical vectorization unit 19 converts the morpheme extracted by the morpheme extraction unit 18 into a numerical vector. In the present embodiment, the One-Hot model is used as a method for converting into a numerical vector. The One-Hot model is a model in which the number of words in the corpus is the number of dimensions of a vector, and the word is expressed as 1 when a word appears in each dimension, and is expressed as 0 when no word appears in each dimension. As another vectorization method, for example, Word2Vector or the like can be used.
In the present embodiment, the morpheme is converted into a numeric vector by the numeric vectorization unit 19, but this is for the purpose of digitizing the morpheme. Therefore, the method of numerical conversion is not limited to vectorization, and for example, a method of converting into a higher-order tensor such as a scalar or a matrix may be used.
The vectorized morpheme obtained by the numerical vectorization unit 19 is output to the driving mode evaluation unit 15 as surrounding environment information.

The driver status information output unit 14 generates driver status information in a format suitable for determining a driving mode based on the driver detection data obtained from the driver status detection unit 6, and outputs the driving mode evaluation unit. 15 is output. More specifically, the driver status information output unit 14 includes a driver status feature amount extraction unit 20.
The driver state characteristic amount extraction unit 20 extracts the biological signal characteristic amount of the driver state used by the driving mode evaluation unit 15 using the driver state detection data obtained by the driver state detection unit 6. is there. As an example of the biological signal characteristic amount, for example, a heartbeat interval, a QRS width, and a QRS wave height are obtained as a biological signal characteristic amount from an electrocardiogram signal obtained as driver state detection data, and are obtained as driver state detection data. The pupil diameter is obtained from the image data of the obtained eyeball as a biological signal feature amount.

Then, the driving mode evaluation unit 15 outputs the surrounding environment information output from the surrounding environment information output unit 13, the driver state information output from the driver state information output unit 14, and the driving stored in the judgment model storage unit 16. Based on the mode determination model, an evaluation value representing a degree of suitability for manual operation and automatic operation is calculated, and the evaluation value is output to the operation mode candidate selection unit 7. Here, the driving mode determination model indicates the degree of influence of the surrounding environment of the vehicle and the state of the driver on each of the automatic driving and the manual driving, and by appropriately constructing this driving mode determination model, It is possible to more appropriately calculate the evaluation values of the automatic operation and the manual operation that are useful for selecting an appropriate operation mode. The operation mode determination model is generated in advance using prepared learning data and the like, and is stored in the determination model storage unit 16.

The judgment model storage unit 16 stores the driving mode judgment model acquired by the vehicle communication unit 11 from the information server 4 through communication with the server communication unit 12.

The driving support device 1 is configured as described above.
Next, the operation mode determination model generation device 2 of the present invention will be described. As described above, the driving mode determination model used in the driving support device 1 is generated in advance and stored in the determination model storage unit 16. The operation mode determination model generation device 2 generates the operation mode determination model. In the present embodiment, the driving mode determination model generation device 2 is provided in the information server 4 that provides various information services to vehicles, and the generated driving mode determination model appropriately controls driving of each vehicle. Provided to the device 3.

The information server 4 includes a judgment model storage unit 21 and a server communication unit 12 in addition to the operation mode judgment model generation device 2. Although the information server 4 provides various types of traffic information other than the driving mode determination model to the vehicle, it is omitted in the present embodiment.

The judgment model storage unit 21 stores the operation mode judgment model generated by the operation mode judgment model generation device 2, and upon receiving a request signal from a vehicle via the server communication unit 12, stores the stored operation mode. The mode determination model is provided to the requesting vehicle via the server communication unit 12.

The server communication unit 12 performs various communications with the vehicle communication unit 11, and in this embodiment, in particular, outputs the driving mode determination model acquired from the determination model storage unit 21 to the vehicle communication unit 11.

Next, the configuration of the operation mode determination model generation device 2 will be described. The driving mode determination model generation device 2 is a device that generates a driving mode determination model used when calculating an evaluation value by the driving support device 1 described above.
As shown in FIG. 1, the driving mode determination model generation device 2 includes a learning data storage unit 22, a surrounding environment description unit 23, a morphological extraction unit 24, a numerical vectorization unit 25, a driver state feature amount extraction unit 26, A mode determination model generator 27 is provided.

The learning data storage unit 22 stores learning data for generating a driving mode determination model. When the driving mode determination model is generated, the learning data storage unit 22 stores the learning data in the surrounding environment description unit 23 and the driver state feature amount extraction. The unit 26 outputs the learning data stored in the driving mode determination model generation unit 27.

Here, the learning data will be described. FIG. 2 shows an example of learning data stored in the learning data storage unit 22. In FIG. 2, one row indicates one record, and one record indicates a pair of individual surrounding environment detection data and driver state detection data, and an appropriate driving mode corresponding to each of these data pairs. It consists of operation mode data as operation mode information to be shown. The learning data is composed of a set of many records. An identifier represented by “No.” is assigned to the learning data.
Such learning data is obtained by, for example, actually measuring the surrounding environment and the driver's state when the vehicle is actually driven, and specifying an appropriate driving mode at that time. Specifically, a detection device similar to the surrounding environment detection unit 5 and the driver state detection unit 6 described with reference to FIG. 1 is attached to a vehicle, and the vehicle is driven under various actual environments. The data detected by the detecting devices at the same timing are acquired as surrounding environment detection data and driver state detection data. Furthermore, the evaluator who boarded the vehicle specified driving modes suitable for various situations during or after the vehicle was running, and the surrounding environment detection data and driver status detection data corresponded to those detection data and the appropriate driving mode. By doing so, learning data as shown in FIG. 2 is obtained. When generating the learning data, it is of course possible to associate the manual operation as the operation mode with the detection data when the manual operation is performed, but the detection data during the manual operation is also supported. If it is determined that automatic operation is suitable as the operation mode to be performed, automatic operation may be designated. Conversely, manual operation may be designated as a suitable operation mode even during automatic driving traveling.
The learning data obtained in this way is a large collection of data sets of various combinations of surrounding environments and driver states and appropriate driving modes corresponding to the combinations. By constructing a model that can be treated as a numerical value, it is possible to obtain an operation mode determination model indicating the degree of influence of the surrounding environment of the vehicle and the state of the driver on each of the automatic driving and the manual driving. In this analysis, the driving mode determination model generation device 2 digitizes the surrounding environment detection data and the driver state detection data among the learning data stored in the learning data storage unit 22, and uses a statistical method. An operation mode determination model is generated.

The surrounding environment description unit 23 uses the surrounding environment detection data input from the learning data storage unit 22 to generate character string information in which the surrounding environment is described by a character string in a natural language, and generates the generated character string information. Is output to the morpheme extraction unit 24.
The morpheme extracting unit 24 performs a morphological analysis on the character string information generated by the peripheral environment description unit 23, extracts morphemes, and outputs the extracted morphemes to the numerical vectorization unit 25.
The numerical vectorization unit 25 converts the morpheme extracted by the morpheme extraction unit 24 into a numerical vector and outputs the vector to the operation mode determination model generation unit 27.

In addition, the driver state feature amount extraction unit 26 extracts the biometric signal feature amount of the driver state using the driver state detection data input from the learning data storage unit 22, and outputs the driving mode determination model generation unit 27. Output to

The driving mode determination model generation unit 27 includes a numerical vector output by the numerical vectorization unit 25 as surrounding environment information, a driver's biological signal characteristic amount output by the driver state characteristic amount extraction unit 26 as driver state information, Then, using the surrounding environment information and the driving mode data corresponding to the driver state information stored in the learning data storage unit 22, the driving mode determination model is generated. For example, when a person is driving asleep, the vehicle should run in the automatic driving mode, but the heart rate decreases during the driving asleep. Therefore, in order to determine whether the automatic driving is in an appropriate driving mode, the heartbeat interval is important as a driver state feature, and the influence of the heartbeat interval corresponding to the label "automatic driving" of the driving mode determination model is large. Become. The driving mode determination model generation unit 27 performs the same processing as described above for all data included in the learning data, and for each driving mode, each component of the numerical value vector of the surrounding environment information and each of the driver state information. The degree of influence of the biological signal feature is optimized, and a driving mode determination model is generated. The generated operation mode determination model is output to the determination model storage unit 21.

Next, the hardware configuration of the present embodiment will be described.

FIG. 3 is a configuration diagram showing a hardware configuration of the operation control device 3, the information server 4, and the like shown in FIG. 1, and includes

processing devices

28 and 29 such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a ROM (Read Only Memory). A storage device 30 such as a hard disk device, a storage device 31, an input device 32 such as a sensor, an input device 33, an input device 34, an output device 35 such as a speaker or a display, a communication device 36, and a communication device 37 are bus-connected. It is the configuration that was done. Note that the CPU may have its own memory.

The surrounding environment detecting unit 5 shown in FIG. 1 is realized by the input device 32, the driver state detecting unit 6 is realized by the input device 33, the driving mode input unit 9 is realized by the input device 34, and the driving mode presenting unit 8 is realized by the output device 35. You.

The data stored in the judgment model storage unit 16 is stored in the storage device 30. In addition, the peripheral environment description unit 17, the morpheme extraction unit 18, the numerical vectorization unit 19, the driver state feature amount extraction unit 20, the operation mode evaluation unit 15, the operation mode candidate selection unit 7, and the operation control unit 10 are stored in the storage device 30. This is realized by executing the stored program in the processing device 28.

The processing device 28 appropriately reads and executes the program stored in the storage device 30 to execute the peripheral environment description unit 17, the morphological extraction unit 18, the numerical vectorization unit 19, the driver state feature amount extraction unit 20, The functions of the mode evaluation unit 15, the operation mode candidate selection unit 7, and the operation control unit 10 are realized. Note that the above functions are not limited to the combination of hardware and software, and the above programs may be implemented in the processing device 28 and realized by a single piece of hardware such as a system LSI (Large Scale Integrated Integrated Circuit).

The data stored in the judgment model storage unit 21 and the learning data storage unit 22 are stored in the storage device 31. The peripheral environment description unit 23, the morphological extraction unit 24, the numerical vectorization unit 25, the driver state feature amount extraction unit 26, and the driving mode judgment model generation unit 27 execute the program stored in the storage device 31 by the processing device 29. It is realized by doing.

The processing device 29 appropriately reads and executes the program stored in the storage device 31 to execute the surrounding environment description unit 23, the morphological extraction unit 24, the numerical vectorization unit 25, the driver state feature amount extraction unit 26, The function of the mode determination model generation unit 27 is realized. Note that the above functions are not limited to the combination of hardware and software, as in the case of the processing device 28, and the above-described program may be implemented in the processing device 29 and realized by hardware alone.

The vehicle communication unit 11 is realized by the communication device 36, and the server communication unit 12 is realized by the communication device 37.

Next, the operation in the present embodiment will be described.

The operation of the driving control device 3, the surrounding environment detecting unit 5, and the driver state detecting unit 6 in the present embodiment will be described with reference to the flowchart of FIG.

First, in step S1, the surrounding environment detecting unit 5 detects the surrounding environment of the vehicle using various sensors. Examples of the surrounding environment to be detected include pedestrians and other vehicles, structures existing on roads, obstacles, and the like, and various sensors are mounted on the vehicle and are cameras for imaging the front. The surrounding environment detection unit 5 outputs surrounding environment detection data representing the detected surrounding environment to the surrounding environment information output unit 13.

In step S2, when the peripheral environment information output unit 13 receives the peripheral environment detection data obtained by the peripheral environment detection unit 5 in step S1, the peripheral environment description unit 17 in the peripheral environment information output unit 13 transmits the received peripheral environment data. Character string information described as a character string in a natural language is generated from the environment detection data. For example, when image data obtained by imaging the front of the vehicle by the surrounding environment detection unit 5 is sent as the surrounding environment detection data, the surrounding environment description unit 17 converts the surrounding situation indicated by the captured image into a character string, that is, converts the surrounding situation into a character string. Generate character string information described by a character string in a natural language.
An example of the operation in step S2 will be described. FIG. 5 is a schematic diagram illustrating an example of the running vehicle 38 and the surrounding environment of the vehicle 38. In FIG. 5, the vehicle 38 is traveling on a general road 39 in the direction of the arrow, and two pedestrians 40 are at the corner of the right sidewalk near the intersection and one pedestrian 40 is at the left side corner near the intersection. Shows the situation in which Under this circumstance, the peripheral environment description unit 17 that has received the image data of the front imaged from the vehicle 38 performs image recognition processing, and generates character string information representing the captured image in a natural language. The image processing and the character string conversion are performed by using, for example, a convolutional neural network (CNN) or a recurrent neural network (RNN), which is a model of machine learning. As a result, in the example of FIG. 5, for example, character string information described as a character string “Three pedestrians are ahead while traveling on a general road” is generated. The generated character string information is output to the morpheme extraction unit 18.

When the character string information generated by the peripheral environment description unit 17 in step S2 is output to the morpheme extraction unit 18, the morpheme extraction unit 18 performs morphological analysis on the sent character string information in step S3. Extract morphemes. The morphological analysis is a process of segmenting a character string into morphemes. For example, when a character string "running on a general road and there are three pedestrians ahead" is morphologically analyzed, "" It is divided into morphemes “running”, “forward”, “ni”, “pedestrian”, “ga”, “3”, “people”, and “is”. The morpheme extraction unit 18 outputs the extracted morpheme to the numerical vectorization unit 19.

In step S4, the morpheme extracted by the morpheme extraction unit 18 in step S3 is converted into a numerical vector by the numerical vectorization unit 19. Here, an example will be described in which the One-Hot model is used to perform numerical vectorization as a numerical vectorization method. FIG. 6 shows the morphemes 41 such as "" general road ",""," running "," forward "," ni "," pedestrian "," ga "," 3 "," people "and" is "obtained in step S3. 9 is a correspondence table showing correspondence between dimensions, morphemes, and obtained vector values when vectorized. Although the correspondence table of FIG. 6 shows up to a portion having 16 dimensions, it is considered that the greater the number of dimensions, the finer the granularity for expressing the situation. Further, in the example of FIG. 6, particles are not handled in correspondence with one dimension of a morpheme, but particles may be handled.
The numerical vectorization unit 19 outputs the generated numerical vector to the operation mode evaluation unit 15.

Acquisition and processing of the surrounding environment information are performed in steps S1 to S4 described above. In parallel with these operations, the operation of the driver state information acquisition and processing starting from step S5 is performed. In step S5, the driver state detecting unit 6 detects the driver's state with a biological sensor or a camera at a timing corresponding to when the surrounding environment detecting unit 5 detects the surrounding environment of the vehicle with various sensors in step S1. As an example of detecting the driver's state, there is a case where an electrocardiogram is obtained by an electrocardiograph and eyeball image data is obtained by a camera as described above.
The driver status detection unit 6 outputs the detected driver status information to the driver status information output unit 14 as driver status detection data.

In step S6, the driver state feature amount extraction unit 20 in the driver state information output unit 14 uses the driver state detection data sent from the driver state detection unit 6 in step S5, and the living state of the driver is displayed. Extract the signal features. For example, the driver state characteristic amount extraction unit 20 extracts a heartbeat interval, a QRS width, and a QRS wave height as characteristic amounts from the electrocardiogram signal as the driver state detection data, and extracts an image of an eyeball as the driver state detection data. The pupil diameter is extracted from the data as a feature amount. FIG. 7 shows an example of the extracted driver state feature quantity. The driver state characteristic amount extraction unit 20 outputs the extracted driver state characteristic amount to the driving mode evaluation unit 15.
In these steps S5 and S6, the operation of the processing for obtaining the driver state information is performed.

Then, as a result of the operations in steps S1 to S4, the generated numerical vector is output to the driving mode evaluation unit 15, and as a result of the operations in steps S5 to S6, the driver state feature amount is output to the driving mode evaluation unit 15. Will be. After these operations, the process proceeds to step S7. In step S7, the driving mode evaluation unit 15 determines the numerical value vector input from the numerical value vectorizing unit 19 and the driver state characteristic input from the driver state characteristic amount extracting unit 20. Based on the amount and the driving mode judgment model stored in the judgment model storage unit 16, the evaluation values of the suitability of manual driving and automatic driving with respect to the situation where the surrounding environment information and the driver state information are obtained in steps S1 and S5 are calculated. I do.

Hereinafter, a method of calculating the evaluation value will be described. FIG. 8 is a schematic diagram showing an example of the concept of an operation mode determination model used for calculating the evaluation value. The driving mode determination model indicates the degree of influence on the automatic driving and the manual driving for each component of the numerical vector that is the surrounding environment information and each driver state feature amount that is the driver state information, and is shown in FIG. In the operation mode determination model, the degree of influence is represented as a numerical value. In addition, here, the degree of influence of each of the surrounding environment information and the driver state information indicated in the driving mode determination model on the automatic driving and the manual driving is represented by a numerical value. What is necessary is just to represent the correspondence between the automatic operation and the manual operation of each of the environmental information and the driver state information, and may be represented by a function, for example. In this driving mode determination model, a unit in which the influence of each component of the numerical value vector and each driver state feature amount is integrated for each driving mode is called an influence vector.

The driving mode evaluation unit 15 first combines the numerical value vector input from the numerical value vectorization unit 19 and the driver state characteristic amount input from the driver state characteristic amount extraction unit 20 to obtain both the surrounding environment and the driver state. Is generated. Here, for example, the vector A input from the numerical vectorization unit 19 is an n-dimensional vector, and its component is represented as A = (a ₁ , a ₂ ,..., _An ), and the driver state feature amount is extracted. When there are m driver state feature amounts input from the unit 20 and their values are b ₁ , b ₂ ,..., B _m , the generated feature amount vector C is an (n + m) -dimensional vector. And its components are represented as C = (a ₁ , a ₂ ,..., _An , b ₁ , b ₂ ,..., B _m ).
Next, the operation mode evaluation unit 15 calculates an evaluation value for each operation mode. Specifically, the inner product of the feature quantity vector and the influence vector is calculated according to equation (1). The denominator in Expression (1) is a normalization constant. At this time, it is assumed that the feature value vector used for input has been converted into a binary value of 0 or 1.

Here, X is the feature quantity vectors, Y _i represents a manual operation when in operation mode automatic operation when i = 1, the i = 2. P (Y _i | X) is an evaluation value of Y _i when X is input, and W _i is an influence vector of each operation mode. When i = 1, automatic operation is performed, and when i = 2, manual operation is performed. Represents the influence vector. The suffix _i of Wi does not represent a vector component, but is merely a suffix for distinguishing between an influence vector for automatic operation and an influence vector for manual operation.

Further, the formula for calculating the evaluation value for each driving mode is not limited to this, and may be any formula that reflects the degree of influence of each feature based on the driving mode determination model. It may be calculated.

The meaning of each character is the same as in equation (1).

FIG. 9 is a diagram showing an example in which the evaluation values of the suitability of the manual driving and the automatic driving for the combination of the surrounding environment information and the driver state information are calculated. In the situation shown in this example, the evaluation value of the suitability of the manual driving is shown. Is calculated to be 0.2, and the evaluation value of the suitability for automatic driving is calculated to be 0.8.
The obtained evaluation value is output to the driving mode candidate selection unit 7.

In step S <b> 8, the driving mode candidate selection unit 7 selects a driving mode candidate appropriate for the current surrounding environment and the driver state based on the evaluation value obtained by the driving mode evaluation unit 15. In the present embodiment, a threshold value is set for the difference between the evaluation values of the automatic operation and the manual operation, and when the difference exceeds the threshold value, the operation mode having the larger evaluation value is set as an appropriate operation mode candidate. For example, when the threshold value is set to “0.5”, the evaluation value result in FIG. 9 exceeds the threshold value because the difference between the evaluation value of the automatic operation and the evaluation value of the manual operation is 0.6. Is selected as an appropriate driving mode candidate. If the threshold value is not exceeded, the currently applied operation mode is selected as a candidate. Here, the method of setting the threshold value is used, but more simply, the operation mode with the larger final evaluation value may be selected as an appropriate operation mode candidate.

In step S9, a comparison is made as to whether the operation mode candidate selected by the operation mode candidate selection unit 7 in step S8 matches the current operation mode. This operation is also performed by the operation mode candidate selection unit 7. If they match, the process proceeds to step S10, in which the operation mode presentation unit 8 presents that the current operation mode is appropriate, and also issues a signal indicating that the current operation mode is to be continuously applied. Output to the unit 10. Thereafter, the process proceeds to step S11, and the operation control unit 10 receiving the signal instructing to continuously apply the current operation mode performs the operation control by continuously applying the current operation mode.
If the driving mode candidate does not match the current driving mode in step S9, the process proceeds to step S12, in which the driving mode candidate selecting unit 7 displays the content for urging the user to switch to the selected driving mode candidate. To be presented.
When the selected driving mode candidate is presented to the driving mode presentation unit 8, the driver determines whether to switch to the presented driving mode candidate.

Next, proceeding to step S13, the operation control unit 10 waits whether the driver has issued an instruction to switch the operation mode via the operation mode input unit 9, and if there is an instruction to switch the operation mode, proceeds to step S14. The operation control unit 10 performs operation control by switching the operation mode to the operation mode candidate selected in step S8. Further, when the driver does not give an instruction to switch the driving mode in step S13, for example, when an instruction to continue the current driving mode is input, or presents a content urging the switching of the driving mode candidate to the driving mode presentation unit 8. If nothing has been input for a certain period of time after this, the operation proceeds to step S11, where the operation control unit 10 continues to apply the current operation mode and performs operation control.

The driving support device 1, the driving control device 3, the surrounding environment detecting unit 5, and the driver state detecting unit 6 repeat the operations shown in steps S1 to S14 to perform driving support.

The suitability of the automatic driving and the manual driving for the combination of the vehicle surrounding environment information and the driver state information by the operation of the driving support device 1, the driving control device 3, the surrounding environment detecting unit 5, and the driver state detecting unit 6 as described above. Since the evaluation value representing the degree is calculated, it is possible to use the evaluation value to present a suitable driving mode candidate according to the surrounding environment and the driver state to the driver, and the driver can select an appropriate driving mode. Can help you decide.

Although the operation of the vehicle communication unit 11 is omitted in the flowchart of FIG. 4, the vehicle communication unit 11 communicates with the server communication unit 12 at an arbitrary timing before step S <b> 7 and transmits a driving mode determination model from the information server 4. To get.

Next, the operation of the information server 4 will be described using the flowchart of FIG.
First, in step S101, the surrounding environment description unit 23 generates character string information in which the surrounding environment is described by a character string in a natural language from the surrounding environment detection data stored in the learning data storage unit 22. The generated character string information is output to the morpheme extraction unit 24.

Next, in step S102, the morpheme extraction unit 24 performs morpheme analysis on the character string information generated by the peripheral environment description unit 23 in step S101, and extracts morphemes. The extracted morpheme is output to the numerical vectorization unit 25.

In step S103, the morpheme obtained by the morpheme extraction unit 24 in step S102 is converted into a numeric vector by the numeric vectoring unit 25. The converted numerical vector is output to the operation mode determination model generation unit 27.

In step S104, the driver state characteristic amount extraction unit 20 extracts the biological signal characteristic amount of the driver state from the driver state detection data stored in the learning data storage unit 22. The extracted driver state feature amount is output to the driving mode determination model generation unit 27.

In step S105, based on the numerical value vector input from the numerical value vectorization unit 25, the driver state characteristic amount input from the driver state characteristic amount extraction unit 26, and the driving mode data input from the learning data storage unit 22, The operation mode determination model generation unit 27 generates an operation mode determination model by a statistical method. For example, it can be generated using the maximum entropy method. Specifically, as shown in

Equations

1 and 2, a parameter W _i that characterizes the function form of the probability distribution function of the realization probability P is introduced, and the form of the probability distribution function realized by each operation mode is assumed, it is a method for optimizing the parameter W _i. Thus a impact vector in the optimized W _i is the operation mode determination model, realized probability P at that time is an evaluation value for each operation mode. Alternatively, the probability distribution function of the realization probability of each of the automatic driving and the manual driving with respect to each component of the numerical vector that is the surrounding environment information and each feature amount of the driver state information is obtained, and the joint probability is used as an evaluation value for each driving mode. Good. Here, it is important that the generation of the judgment model and the calculation of the evaluation value of each operation mode are performed by a statistical method, and are not limited to the above method.
The generated operation mode determination model is stored in the determination model storage unit 21.
Steps S101 to S103 described above are the same operations as steps S2 to S4 in the flowchart described with reference to FIG. 4 as the operation of driving assistance by the driving assistance device 1 in FIG. 1, and similarly, the operation in step S104 is step S6. This is the same operation as.

When the server communication unit 12 receives the request signal for the driving mode determination model from the vehicle communication unit 11, the server communication unit 12 outputs the driving mode determination model stored in the determination model storage unit 21 to the vehicle communication unit 11.

By the operation of the information server 4 as described above, the driving mode determination model is provided to the driving support apparatus 1, and the driving support apparatus 1 stores the provided driving mode determination model in the determination model storage unit 16, thereby providing driving support. Can be used to evaluate the operation mode.
In addition, the operation mode determination model generation device 2 appropriately constructs the operation mode determination model from a large amount of learning data by a statistical method, and thereby each of the evaluation values of the automatic operation and the manual operation useful for selecting the operation mode. Can be calculated more appropriately.

Further, in the present embodiment, the surrounding environment information is converted into a character string in a natural language by the surrounding environment description unit 17, and then the morpheme in the character string is extracted by the morpheme extracting unit 18. This advantage will be described. In the real world, the objects that appear in the surrounding environment of a vehicle are diverse, and the combinations and positional relationships of the objects often change.Therefore, when creating a model that represents the surrounding environment, setting the surrounding environment individually is extremely difficult. It is troublesome.
In the present embodiment, the surrounding environment set in the previous model can be expressed by a combination of morphemes defined in the natural language database without setting the surrounding environment individually. Work can be simplified. In addition, in the conventional models, the surrounding environment, which could not be set by the person designing the model without thinking, can be set by combining the morphemes.
At this time, an existing natural language database may be used, or a natural language database may be newly created for an operation mode determination model.

メリット The advantage of describing the surrounding environment with a character string in a natural language is not only that a natural language database can be used. By performing logical estimation on a character string described in a natural language using a knowledge structure database, it is possible to add information on not only the surrounding environment acquired by the sensor but also a situation outside the sensor range. For example, consider a case where a soccer ball is captured in an image acquired by a camera. When a soccer ball is shown in the picture, it is possible that a child is playing nearby outside the image, and if the child suddenly jumps out of the way to get the soccer ball, it will collide with the vehicle there is a possibility. The logic estimation using the knowledge structure database makes it possible to infer the possibility that a child will jump out of the image of the soccer ball, and presents it to the driver so that the driver can avoid collisions in advance. You can drive.

場合 In this example, inference and information presentation are performed in the following procedure. First, when it is recognized that a soccer ball exists in the image, a character string "soccer ball" is described. If the character string "soccer ball" is associated with "child" in the knowledge structure database, information based on "child" is presented, and there is a possibility that "child" is invisible to the driver The driver should carefully check that the child does not jump out. If it is recognized from further detailed information, for example, that an image or a moving image indicates that "soccer ball is rolling down the road", a character string "soccer ball is rolling down the road" is described. Then, as described above, it can be inferred that the “child” will “jump out” from the “rolling” of the “soccer ball”, and more detailed information can be provided to the driver.

In addition, in the evaluation of the driving mode, the value of the component of the numerical vector corresponding to the specific morpheme becomes 1 and the evaluation value fluctuates, so that the driving mode more suitable for the surrounding environment can be evaluated. For example, recognizing that a soccer ball is rolling down the road from an image or video, etc., and describing it in a character string, it can be said that a child jumps out because a soccer ball is rolling. Can be guessed, the possibility is high. In this case, in addition to the originally recognized morphemes such as "soccer ball" and "rolling", the morphemes of "child" and "jump" are added by inference, and the components of the numerical vector corresponding to "child" and "jump" When the value of is 1, for example, while the evaluation value of manual driving decreases, the evaluation value of automatic driving increases, and even if the driver does not notice sudden child jumping out or the reaction of the driver is likely to be delayed, By automatically driving, the possibility of collision between the vehicle and the child can be avoided.

In the present embodiment described above, the driving support device 1 is configured to include the judgment model storage unit 16, but the driving support device 1 does not include the judgment model storage unit 16, and the driving mode evaluation unit 15 is The configuration may be such that the judgment model storage unit 21 in the information server 4 is appropriately referred to by the communication via the server communication unit 12 and.

The driving support apparatus in the present embodiment uses the driving mode determination model to calculate an evaluation value indicating the suitability of the automatic driving and the manual driving for the combination of the surrounding environment information and the driver state information. An evaluation value indicating the suitability of the automatic driving and the manual driving may be calculated based on the combination of the information and the driver state information, and a configuration without using the driving mode determination model may be used. For example, in the surrounding environment information, if the number of pedestrians increases by one, the evaluation value of automatic driving is increased by 1, while the evaluation value of manual driving is decreased by 1. For example, a threshold value is set for the heart rate in the driver state information. If the threshold value is exceeded, the evaluation value of the automatic driving is increased by 10, and so on.

Embodiment 2
Next, a driving support device 1 and a driving mode determination model generation device 2 according to Embodiment 2 of the present invention will be described. In the second embodiment, in addition to the configuration of the first embodiment, the information server 4 includes a learning data adding unit 42. According to the driver's judgment input to the driving mode input unit 9, By changing whether or not the learning data is added by the learning data adding unit 42, the accuracy of the judgment model is improved.

FIG. 11 is a configuration diagram showing a configuration of a driving assistance device 1 and a driving mode determination model generation device 2 according to a second embodiment for carrying out the present invention. It is similar.

In addition to the functions described in the first embodiment, the driving mode input unit 9 outputs a signal indicating that the driving mode is not switched when the driver does not switch to the presented driving mode. The data is output to the learning data adding unit 42 via the communication unit 11 and the server communication unit 12. Here, the time when the driver does not switch to the presented driving mode is not only when the driver has input to the driving mode input unit 9 to continue running in the current driving mode, but also This includes the case where there is no input from the driver to the operation mode input unit 9 for a certain period of time after the content prompting the switching of the operation mode is presented to the operation mode presentation unit 8.

When a signal indicating that switching of the driving mode is not performed is input from the driving mode input unit 9 to the learning data adding unit 42, the surrounding environment information output unit 13 via the server communication unit 12 and the vehicle communication unit 11. The learning data is collected from the driver state information output unit 14 and the driving control unit 10, and the learning data is added to the learning data storage unit 22.
The learning data to be added is, for example, a data set of the surrounding environment detection data, the driver state detection data, and the driving mode data. The driving mode data is not a driving mode candidate presented by the driving mode presentation unit 8 but a current one. This is operation mode data as operation mode information in which an operation mode being applied is recorded. The reason is that the case where the driver has not switched the driving mode is determined by the driver that the driving mode candidate presented by the driving mode presenting unit 8 is not appropriate for the current surrounding environment and driver state. The driving mode data corresponding to the surrounding environment detection data and the driver state detection data used to select the presented driving mode, that is, the learning mode data to be added includes the presented driving mode. This is because the candidates are inappropriate and the currently applied operation mode is appropriate.

In the present embodiment, the learning data adding unit 42 includes the peripheral environment detection data from the peripheral environment description unit 17 provided in the peripheral environment information output unit 13 and the driver state characteristics provided in the driver state information output unit 14. Driver state detection data is obtained from the quantity extraction unit 20. The surrounding environment description section 17 includes a memory for temporarily storing the surrounding environment detection data input from the surrounding environment detecting section 5. The input of the surrounding environment detection data is given an identifier indicated by “No.” in FIG. 2, and when the driver does not switch to the presented driving mode, the learning data adding unit 42 Can be read from the peripheral environment description unit 17 by designating the corresponding peripheral environment detection data using this identifier. Similarly, the driver state characteristic amount extraction unit 20 is provided with a memory for temporarily storing the driver state detection data input from the driver state detection unit 6, and the input driver state right data includes the memory shown in FIG. The identifier indicated by “No.” is given. When the driver does not switch to the presented driving mode, the learning data adding unit 42 specifies the corresponding driver state detection data using this identifier, and thereby the driver state feature amount extracting unit 20. Can be read from

(4) The learning data addition unit 42 acquires from the operation control unit 10 operation mode data as operation mode information indicating the current operation mode. Here, the current operation mode is an operation mode that is already applied in the operation control unit 10 when the driver switches or inputs the operation mode.

In the present embodiment, the learning data adding unit 42 converts the surrounding environment detection data from the surrounding environment description unit 17, the driver state detection data from the driver state feature amount extraction unit 20, and the driving mode data from the driving control unit 10. However, the configuration for data acquisition is not limited to the above as long as each data can be acquired. For example, the surrounding environment detection data from the surrounding environment detection unit 5 and the driver state detection data from the driver state detection unit 6 Thus, the configuration may be such that the operation mode data is acquired from the operation mode candidate selection unit 7.

The configurations of the driving control device 3 and the information server 4 other than those described above in the present embodiment, and the configurations of the surrounding environment detection unit 5 and the driver state detection unit 6 are the same as those in the first embodiment. The driving support device 1 and the driving mode determination model generation device 2 in the embodiment are configured as described above.

The hardware configuration in the present embodiment is the same as that shown in FIG. 3, and the learning data adding unit 42 added in the second embodiment executes the program stored in the storage device 31 by the processing device 29. This function is not limited to the combination of hardware and software, and the above-described program may be implemented in the processing device 29 and may be realized by hardware alone.

Next, the operation in the present embodiment will be described.
In the second embodiment, the same operation as in the flowchart of FIG. 4 shown in the first embodiment is performed. However, in the second embodiment, operations after step S9 in FIG. 4 are different, and operations after step S8 are shown in FIG. 4 is different from FIG. 4 in that the operation of steps S201 and S202 is performed before step S11 when the operation proceeds to “No” in the branch of step S13.
After proceeding to “No” in step S13, in step S201, the learning data addition unit 42 collects learning data via the vehicle communication unit 11 and the server communication unit 12. The learning data in the present embodiment is a data set of surrounding environment detection data, driver state detection data, and operation mode data representing the current operation mode. The surrounding environment detection data is output from the surrounding environment description unit 17, the driver state detection data is output from the driver state feature amount extraction unit 20, and the operation mode data is output from the operation control unit 10.

In step S202, the learning data addition unit 42 outputs the learning data collected in step S201 to the learning data storage unit 22, and the learning data storage unit 22 stores the input learning data.
After step S202 is performed, the process proceeds to step S11. The operation in step S11 is the same as the operation in the first embodiment.

By the above operation, in the second embodiment, the data set when the driving mode candidates that do not match the driver's judgment are presented are set together with the driving mode determined by the driver, and the learning data is used as the learning data. By adding to the storage unit 22, the accuracy of the driving mode determination model generated next time can be improved, and driving assistance more suitable for the driver can be performed.

In the present embodiment, the surrounding environment detection data and the driver state detection data are assumed as the learning data to be added, but the surrounding environment information and the driver state information at the timing when the driver refuses to switch the driving mode are described. As long as it is a data set, it is not necessary to use these detection data. For example, a numerical vector may be stored as surrounding environment information, and driver state feature amount information may be stored as driver state information as a data set.

The driving support device and the driving mode determination model generation device according to the present invention are applicable to an automatic driving system for a vehicle.

1 driving support device, 2 driving mode judgment model generating device, 3 driving control device, 4 information server, 5 surrounding environment detecting unit, 6 driver status detecting unit, 7 driving mode candidate selecting unit, 8 driving mode presenting unit, 9 driving Mode input unit, 10 driving control unit, 11 vehicle communication unit, 12 server communication unit, 13 surrounding environment information output unit, 14 driver status information output unit, 15 driving mode evaluation unit, 16 judgment model storage unit, 17 surrounding environment description Unit, 18 morpheme extraction unit, 19 numerical vectorization unit, 20 driver state feature quantity extraction unit, 21 judgment model storage unit, 22 learning data storage unit, 23 peripheral environment description unit, 24 morpheme extraction unit, 25 numeric vectorization Unit, 26 driver state feature quantity extraction unit, 27 driving mode judgment model generation unit, 28 processing unit, 29 processing unit 30 storage device, 31 storage device, 32 input device, 33 input device, 34 input device, 35 output device, 36 communication device, 37 communication device, 38 vehicle, 39 general road, 40 pedestrian, 41 morpheme, 42 learning data Additional part

Claims

A surrounding environment information output unit that outputs surrounding environment information representing the surrounding environment of the vehicle;
A driver state information output unit that outputs driver state information indicating a state of the driver corresponding to the surrounding environment;
Based on a combination of the surrounding environment information output by the surrounding environment information output unit and the driver state information output by the driver state information output unit,
A driving support device comprising: a driving mode evaluation unit that calculates an evaluation value indicating aptitude of automatic driving and manual driving with respect to a combination of the surrounding environment information and the driver state information.
Each of the surrounding environment of the vehicle and the state of the driver includes a judgment model storage unit that stores a driving mode judgment model indicating the degree of influence on each of the automatic driving and the manual driving,
The driving mode evaluation unit stores the influence degree corresponding to each of the surrounding environment information output by the surrounding environment information output unit and the driver state information output by the driver state information output unit in the determination model storage unit. 2. The evaluation value according to claim 1, wherein the evaluation value is calculated based on a combination of the degree of influence, the surrounding environment information, and the driver state information with reference to the stored driving mode determination model. 3. Driving support device.
The surrounding environment information output unit, based on surrounding environment detection data obtained by detecting the surrounding environment of the vehicle, a surrounding environment description unit that generates character string information describing the surrounding environment in a natural language,
The driving support apparatus according to claim 2, further comprising: a morpheme extraction unit that morphologically analyzes the character string information generated by the surrounding environment description unit, extracts a morpheme, and outputs the morpheme as the surrounding environment information. .
The surrounding environment information output unit, based on surrounding environment detection data obtained by detecting the surrounding environment of the vehicle, a surrounding environment description unit that generates character string information describing the surrounding environment in a natural language,
A morpheme extraction unit that extracts a morpheme by morphologically analyzing the character string information generated by the surrounding environment description unit;
The driving assistance device according to claim 2, further comprising: a numerical vector conversion unit that converts the morpheme extracted by the morpheme into a numerical vector and outputs the converted vector as the surrounding environment information.
For learning by associating surrounding environment information indicating a surrounding environment of a vehicle, driver state information indicating a driver state corresponding to the surrounding environment, and driving mode information indicating a driving mode corresponding to the driver state with the surrounding environment. A learning data storage unit for storing as data,
Based on the surrounding environment information, the driver state information, and the driving mode information stored in the learning data storage unit, the degree of influence of the surrounding environment of the vehicle and the state of the driver on each of the automatic driving and the manual driving is determined. An operation mode determination model generation device, comprising: an operation mode determination model generation unit that generates the operation mode determination model shown in FIG.
The driver state information stored in the learning data storage unit is driver state characteristic amount information indicating a driver state by a characteristic amount, and the driving mode determination model generation unit generates a driving mode determination model. The driving mode determination model generation device according to claim 5, wherein the driver state feature amount information is used as the driver state information at the time.
The driver state information stored in the learning data storage unit is driver state detection data in which the state of the driver is detected by a sensor,
Based on the driver state detection data stored in the learning data storage unit, a driver state feature amount information extraction unit that extracts driver state feature amount information indicating the state of the driver by a feature amount,
The driving mode determination model generation unit is based on the driver state characteristic amount information extracted by the driver state characteristic amount information extraction unit, the surrounding environment information and the driving mode information stored in the learning data storage unit. The driving mode determination model generation device according to claim 5, wherein the driving mode determination model is generated.
The surrounding environment information stored in the learning data storage unit is surrounding environment detection data in which a surrounding environment of the vehicle is detected by a sensor,
A surrounding environment description unit that generates character string information describing the surrounding environment in a natural language based on the surrounding environment detection data stored in the learning data storage unit;
A morpheme extraction unit that extracts a morpheme by morphologically analyzing the character string information generated by the surrounding environment description unit,
The driving mode determination model generation unit generates the driving mode determination model based on the morpheme extracted by the morpheme extraction unit, the driver state information stored in the learning data storage unit, and the driving mode information. The operation mode determination model generation device according to claim 5, wherein
A numerical vectorization unit that converts the morpheme extracted by the morphological extraction unit into a numerical vector,
The driving mode determination model generation unit is configured to execute the driving mode determination model based on the numerical value vector generated by the numerical vectorization unit, the driver state information and the driving mode information stored in the learning data storage unit. The operation mode determination model generation device according to claim 8, wherein
A surrounding environment information output unit that outputs surrounding environment information representing the surrounding environment of the vehicle;
A driver state information output unit that outputs driver state information indicating a state of the driver corresponding to the surrounding environment;
A judgment model storage unit that stores a driving mode judgment model indicating the degree of influence on the driving mode of each of the automatic driving and the manual driving of the vehicle surrounding environment and the driver state,
The driving mode determination in which the degree of influence corresponding to each of the peripheral environment information output by the peripheral environment information output unit and the driver state information output by the driver state information output unit is stored in the determination model storage unit. An evaluation representing the suitability of automatic driving and manual driving for the combination of the surrounding environment information and the driver state information based on a combination of the degree of influence, the surrounding environment information, and the driver state information with reference to a model. An operation mode evaluation unit for calculating a value,
A driving mode candidate selection unit that selects an appropriate driving mode candidate for the surrounding environment information and the driver state information based on the evaluation value calculated by the driving mode evaluation unit,
A driving mode presenting unit that presents driving assistance information to the driver based on the driving mode candidate selected by the driving mode candidate selecting unit,
A driving mode input unit that inputs whether or not the driver switches the driving mode based on the driving support information presented by the driving mode presentation unit,
When a signal indicating that switching is not performed is input to the operation mode input unit, the surrounding environment information output by the surrounding environment information output unit and the driver state information output by the driver state information output unit The driving mode determination model according to any one of claims 5 to 9, further comprising: a learning data adding unit that adds driving mode information indicating a current driving mode to the learning data storage unit. Generator.