JP2024068004A - Information processing method, program, and information processing system - Google Patents

Abstract

[Problem] A method for generating an appropriate user interface for a vehicle includes a driving information acquisition step for acquiring information about a driver's driving operation, a separate task request step for requesting the driver to execute a predetermined process different from the driving operation while the driver is driving, a driver state information acquisition step for acquiring information about at least one of the driver's driving operation, the predetermined process, and the driver's state while the driver is executing the predetermined process, an evaluation step for evaluating the driver's cognitive function based on the information acquired in the driver state information acquisition step, and a generation step for generating a user interface for a vehicle based on the evaluation result of the driver's cognitive function determined in the evaluation step. [Selected Figure] Figure 1

Description

The present invention relates to an information processing method, a program, and an information processing system.

Cited Document 1 discloses a display control device. The display control device disclosed in this document controls the display on a display means such as a liquid crystal display that is visually recognized by a vehicle occupant, and changes the display mode of icons or marks on a map displayed on the display means according to the degree of recommendation to the occupant.

JP 2017-045285 A

In recent years, the user interface for driving a vehicle has increasingly been shifting from physical buttons to devices using touch screens, due to factors such as an increase in vehicle functionality. When operating a touch screen, the display may change dynamically, and the driver must look closely at the touch screen when operating it, which can be dangerous. In particular, if the driver's cognitive function (also known as cognitive ability, hereinafter referred to as "cognitive function") while driving a vehicle is impaired, the driver may be distracted by operating the touch screen. Therefore, there is room for improvement in generating an appropriate user interface for a vehicle.

In consideration of the above, the present invention aims to provide an information processing method, program, and information processing system that can generate an appropriate user interface for a vehicle.

According to an information processing method according to one embodiment, a method for generating a user interface for a vehicle is executed by an information processing device, and includes a driving information acquisition step for acquiring information about a driver's driving operation, a separate task request step for requesting the driver to execute a predetermined process different from the driving operation while the driver is driving, a driver state information acquisition step for acquiring information about at least one of the driver's driving operation, the execution of the predetermined process, and the driver's state while the driver is executing the predetermined process, an evaluation step for evaluating the cognitive function of the driver based on the information acquired in the driver state information acquisition step, and a generation step for generating a user interface for a vehicle based on the evaluation result of the driver's cognitive function determined in the evaluation step.

According to a program of one embodiment, a method for generating a user interface for a vehicle is executed by an information processing device, and the method causes a computer to execute the following steps: a driving information acquisition step for acquiring information about a driver's driving operation; a separate task request step for requesting the driver to execute a predetermined process different from the driving operation while the driver is driving; a driver state information acquisition step for acquiring information about at least one of the driver's driving operation, the execution of the predetermined process, and the driver's state while the driver is executing the predetermined process; an evaluation step for evaluating the cognitive function of the driver based on the information acquired in the driver state information acquisition step; and a generation step for generating a user interface for a vehicle based on the evaluation result of the driver's cognitive function determined in the evaluation step.

According to an information processing system according to one embodiment, the information processing system includes a driving information acquisition device that acquires driving operation information for driving a vehicle, a separate task execution device that requests the driver who is driving to execute a predetermined process other than vehicle driving, accepts the execution, and acquires the execution result, an evaluation device that evaluates the cognitive function of the driver from at least one of information about the driver, the driving operation information by the driving information acquisition device, and information about the execution of the predetermined process acquired by the separate task execution device, and a generation device that generates a user interface for a vehicle based on the evaluated cognitive function, and executes the following steps: a driving information acquisition step that acquires information about the driving operation of the driver, a separate task request step that requests the driver to execute a predetermined process different from the driving operation while the driver is driving, a driver state information acquisition step that acquires information about at least one of the driving operation of the driver, the execution of the predetermined process, and the state of the driver while the driver is executing the predetermined process, an evaluation step that evaluates the cognitive function of the driver based on the information acquired in the driver state information acquisition step, and a generation step that generates a user interface for a vehicle based on the evaluation result of the cognitive function determined in the evaluation step.

According to one embodiment, an appropriate vehicle user interface can be generated.

1 is a diagram illustrating an example of a configuration of an information processing system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a simulator device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a different task execution device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of an evaluation device and a generation device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a simulator device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a different task execution device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of an evaluation device and a generation device according to the first embodiment. 4 is a flowchart illustrating an example of processing executed by the information processing system according to the first embodiment. FIG. 1A is an example of a display of a vehicle user interface in a normal state generated by the information processing system of the first embodiment, and FIG. 1B is a figure showing an example of a display of a vehicle user interface generated differently from (A) based on evaluation results. FIG. 11 is a diagram illustrating an example of a configuration of an information processing system according to a second embodiment. FIG. 11 is a diagram illustrating an example of a functional configuration of an evaluation device and a generation device according to a second embodiment. 10 is a flowchart illustrating an example of processing executed by an information processing system according to a second embodiment. FIG. 13A is an example of a display of a vehicle user interface in a normal state generated by an information processing system according to the second embodiment, and FIG. 13B is a figure showing an example of a display of a vehicle user interface generated differently from (A) based on evaluation results.

First Embodiment
A first embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 9. Note that the same or equivalent components and parts in each figure are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

(System Overview)
First, an overview of the information processing system 10 according to the present embodiment will be described. The information processing system 10 according to the present embodiment is a system for evaluating the cognitive functions required for driving a vehicle by a driver using a simulator device as a driving information acquisition device that executes a driving simulation, and generating a vehicle user interface suitable for the driver based on the evaluation result of the cognitive functions. The system can be used in any situation, such as customization of a vehicle used by a driver and academic research. The cognitive functions of a driver are determined from the appearance of the driver who is answering calculation test questions, memory test questions, and reaction time test questions as separate tasks that are predetermined processing different from the driving operation, the state of the driving operation, the processing status of the separate tasks, etc., when the driver is performing a driving operation. In other words, the cognitive functions of the driver are determined from the appearance of the driver and the result of a multitasking operation that executes a driving operation and a separate task process, and a vehicle user interface is generated based on the evaluation result of the cognitive functions. The vehicle user interface in this embodiment is a user interface for operating a vehicle and peripheral equipment of the vehicle that need to be operated when driving a vehicle. Specifically, it is an interface that is mainly displayed on an LCD display or the like and is used to operate vehicle functions, provide road guidance, provide business communication screens for commercial drivers, play videos, and operate functions linked to mobile devices, and the like, and the display content can be changed according to the control of the information processing system 10.

(System configuration)
Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a simulator device 12, a separate task execution device 14, and an evaluation device 16, which are communicably connected to each other via a network N, a wiring cable C, or the like. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these.

The simulator device 12 is an example of an information processing device (computer) that controls a simulation, and is used to perform a driving simulation by having a driver operate a driving operation means in a simulated driving situation, without having the driver drive an actual vehicle. The specific configuration and operation of the simulator device 12 will be described later.

The separate task execution device 14 is an example of an information processing device that controls the execution of separate task processes. The separate task execution device 14 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the separate task execution device 14 will be described later.

The evaluation device 16 is an example of an information processing device that evaluates the cognitive function of the driver, which will be described later, and controls the generation of a vehicle user interface accordingly. The evaluation device 16 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. In this embodiment, the evaluation device 16 also serves as a generation device that generates a vehicle user interface based on the evaluated cognitive function of the driver. The specific configuration and operation of the evaluation device 16 will be described later.

(Hardware configuration - simulator device)
2 is a block diagram showing a hardware configuration of the simulator device 12. The simulator device 12 includes a processor 20, a memory 22, a storage 24, a communication I/F 26, an input/output I/F 28, a display device 30, and a driving operation input device 32, which are communicatively connected to each other via a bus B.

The processor 20 controls each component of the simulator device 12 and realizes the functions of the simulator device 12 by expanding various programs stored in the storage 24 into the memory 22 and executing them. The programs executed by the processor 20 include, but are not limited to, an OS (Operating System) and various programs described below. The processor 20 executes these programs to realize a part of the information processing method according to this embodiment. The processor 20 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

The memory 22 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination of these. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination of these. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination of these.

Storage 24 stores the OS, various programs described below, and various data. Storage 24 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

The communication I/F 26 is an interface for connecting the simulator device 12 to external devices including the separate task execution device 14 and the evaluation device 16 via a network N, a wiring cable, etc., and for controlling communication. The communication I/F 26 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (e.g., Fibre Channel), but is not limited to these.

The input/output I/F 28 is an interface for connecting the display device 30 and the driving operation input device 32 to the simulator device 12. The display device 30 is a display, a projector, or a combination of these. As shown in FIG. 1, the display device 30 is disposed in a position where it can be seen by a driver (not shown) who drives in the simulation.

The driving operation input device 32 is configured to include an operating handle 36 and foot pedals 38. The driving operation input device 32 is disposed in a position where it can be operated by a driver seated in a seat (not shown). The driving operation input device 32 is also provided with operation switches (not shown) for operating the transmission, turn signals, wipers, headlights, etc. The driving operation input device 32 acquires operation information indicating the operation of the operating handle 36, foot pedals 38, and operation switches by the driver.

(Hardware configuration - separate task execution device)
3 is a block diagram showing a hardware configuration of the separate task execution device 14. The separate task execution device 14 includes a processor 20, a memory 22, a storage 24, a communication I/F 26, an input/output I/F 28, a display device 40, and an input device 42, which are communicatively connected to each other via a bus B. In this embodiment, the display device 40 and the input device 42 are, as an example, an integrally configured touch panel display. As shown in FIG. 1, the display device 40 and the input device 42 are arranged at a position (as an example in this embodiment, a position near the operation handle 36) where a driver seated in a seat not shown in the figure can operate them during a driving simulation.

(Hardware configuration - evaluation device)
4 is a block diagram showing a hardware configuration of the evaluation device 16. The evaluation device 16 includes a processor 20, a memory 22, a storage 24, a communication I/F 26, an input/output I/F 28, an imaging device 44, a display device 46, and an input device 48, which are communicatively connected to each other via a bus B. As shown in FIG. 1, the imaging device 44 is a camera provided to capture an image of the driver, and is directed toward the driver's face to acquire video information capturing the direction of movement of the driver's line of sight and the state of the driver. Note that the imaging device 44 is not limited to a single camera, and may be composed of multiple cameras.

In this embodiment, as an example, the display device 46 and the input device 48 are an integrally configured touch panel display. The display device 46 and the input device 48 are disposed in a position where they can be seen and operated by the administrator who performs the cognitive function test.

(Functional configuration - simulator device)
Next, the functional configuration of the simulator device 12 will be described. Fig. 5 is a diagram showing an example of the functional configuration of the simulator device 12. When executing various programs, the simulator device 12 realizes various functions using the above-mentioned hardware resources. The simulator device 12 has a storage unit 50, a processing unit 52, and a communication unit 54 as functional configurations realized by the simulator device 12. Each functional configuration is realized by the processor 20 reading and executing a simulation program 56 and a generation program 58 stored in the memory 22 or the storage 24.

The memory unit 50 stores a simulation program 56, a generation program 58, and a simulation DB 60.

The simulation DB 60 is a database for storing various data required for carrying out a driving simulation. In this embodiment, it may include stationary objects and moving object objects. Stationary objects may include, for example, roads, sidewalks, buildings, guardrails, traffic lights, road signs, and the sky. Moving objects may include, for example, people, automobiles, bicycles, motorbikes, and wheelchairs. In addition, it includes data on interior components such as the dashboard as viewed from inside the automobile driven by the driver, the rearview mirror, and the left and right side mirrors as objects representing the automobile driven by the driver.

The processing unit 52 has a test control information processing unit 53, a driving operation information acquisition unit 62, a simulator processing unit 63, and a display processing unit 55. The test control information processing unit 53 acquires information on the driver's personal attributes (such as age and driving experience, for example), and sets a driving simulation scenario for testing the driver's cognitive function based on the information. As an example, this scenario is set up to include multiple scenes, such as general road scenes, highway scenes, and mountain road scenes, and is set up so that events specific to each scene occur in the simulation.

The driving operation information acquisition unit 62 acquires the driver's driving operations input via the driving operation input device 32.

The simulator processing unit 63 performs processing to reproduce the driving operations acquired from the driving operation information acquisition unit 62 on the driving simulation scenario set by the test control information processing unit 53.

The display processing unit 55 generates a simulation video to be presented to the driver based on the information processed by the simulator processing unit 63, and processes it to display it on the display device 30.

The communication unit 54 communicates with various devices via a network N that includes a mobile phone communication network, a wired LAN (Local Area Network), a wireless LAN, the Internet, etc. In this embodiment, the communication unit 54 communicates with various devices via the network N, a wiring cable C, etc. The communication unit 54 transmits data provided by the processing unit 52 to other devices, and provides data received from other devices to the processing unit 52.

(Functional configuration - Task execution device)
Next, the functional configuration of the other task execution device 14 will be described. Fig. 6 is a diagram showing an example of the functional configuration of the other task execution device 14. When executing various programs, the other task execution device 14 realizes various functions using the above-mentioned hardware resources. The other task execution device 14 has a storage unit 64, a processing unit 66, and a communication unit 68 as the functional configuration realized by the other task execution device 14. Each functional configuration is realized by the processor 20 reading and executing the calculation program 70 and the generation program 58 stored in the memory 22 or the storage 24.

The storage unit 64 stores a calculation program 70, a generation program 58, and a question DB 72. The question DB 72 stores at least one of calculation test questions, memory test questions, and reaction time test questions for causing the driver to execute processing as a separate task. Note that in this embodiment, at least one of calculation test questions, memory test questions, and reaction time test questions is stored in the question DB 72, but this is not limited to this, and other separate tasks may also be stored.

The processing unit 66 has a test control information processing unit 67, an input information acquisition unit 74, a separate task processing unit 69, and a display processing unit 71. The test control information processing unit 67 acquires information related to the personal attributes of the driver, and sets questions to be posed as separate tasks for testing the driver's cognitive functions based on the information and operation information for selecting the type of separate task, etc.

The input information acquisition unit 74 acquires operation information of the driver when executing another task process. That is, when at least one of a calculation test question, a memory test question, and a reaction time test question is presented as a separate task, operation information of the driver who inputs an answer (processing result) to the question is acquired. Specifically, as an example, in the case of a calculation test question, a formula for mental calculation is presented, and then answer candidates of the calculation result are presented, and the driver is prompted to selectively input whether the answer candidate is correct or not. The input information acquisition unit 74 acquires at least one of this input result and the time from the question being presented to the input (i.e., the driver's processing time) as operation information.

The separate task processing unit 69 executes a process to output a question based on information from the test control information processing unit 67 and the input information acquisition unit 74. When an answer to a question is input, the separate task processing unit 69 executes a process to output a new question a predetermined number of times. When the separate task processing unit 69 has output the question a predetermined number of times, it ends the output of the question as a separate task.

The display processing unit 71 generates a question screen and a processing result input screen to be presented to the driver based on the information processed by the separate task processing unit 69, and processes them to be displayed on the display device 40.

The communication unit 68 communicates with various devices via a network N that includes a mobile phone communication network, a wired LAN (Local Area Network), a wireless LAN, the Internet, etc. In this embodiment, the communication unit 68 communicates with various devices via the network N, a wiring cable C, etc. The communication unit 68 transmits data provided by the processing unit 66 to other devices, and provides data received from other devices to the processing unit 66.

(Functional configuration - evaluation device)
Next, the functional configuration of the evaluation device 16 will be described. Fig. 7 is a diagram showing an example of the functional configuration of the evaluation device 16. When executing various programs, the evaluation device 16 realizes various functions using the above-mentioned hardware resources. The evaluation device 16 has a storage unit 76, a processing unit 78, and a communication unit 80 as functional configurations realized by the evaluation device 16. Each functional configuration is realized by the processor 20 reading and executing the generation program 58 stored in the memory 22 or the storage 24.

The memory unit 76 stores the generation program 58, the learning model 82, the user DB 84, and the result DB 86.

The learning model 82 is a learning model that has been trained in advance by machine learning to estimate and output the driver's cognitive function in response to input of driver information, driving operation information, and other task operation information described below. The learning process and re-learning process of the learning model 82 are performed, for example, by the evaluation device 16, but are not limited to this. The learning process and re-learning process of the learning model 82 may be performed by a device other than the evaluation device 16 and provided to the evaluation device 16. The learning model 82 is, for example, a neural network such as a deep neural network (DNN), a recurrent neural network (RNN), a convolutional neural network (CNN), or a long short-term memory (LSTM), but is not limited to this. The learning model 82 may be a learning model such as a decision tree or a support vector machine (SVM). In addition, the learning method of the learning model 82 is, for example, a gradient descent method, a stochastic gradient descent method, or an error backpropagation method, but is not limited to this.

User DB84 is a database for storing information about users (i.e., drivers) who undergo cognitive function tests. In this embodiment, ID information that identifies the user and various information linked to the ID information are stored. The various information includes personal attribute information of the user, etc.

Result DB86 is a database that stores and accumulates the results of users who have taken cognitive function tests using the services provided by the information processing system 10 according to this embodiment. Result DB86 stores, in association with each other, information such as the user's name or ID information, the date the user took the test, the user's test results, information about the vehicle user interface generated based on the test results, and other reference test results.

The processing unit 78 has a driver information acquisition unit 88, an inspection control unit 90, a driving operation information acquisition unit 92, a separate task operation information acquisition unit 94, a judgment unit 96, and a display processing unit 81. The driver information acquisition unit 88 acquires information on the driver's personal attributes and video information capturing the driver's behavior while driving. Specifically, the video information is intended to be information primarily for acquiring the movement and direction of the driver's head and body, the position and movement of the driver's eyes, and the like. In particular, with regard to the line of sight, the main information is the direction of the eyes, i.e., whether the driver is looking outside the vehicle (simulator screen) or at the separate task execution device 14.

The test control unit 90 controls the test for evaluating the cognitive function of the driver. That is, the test control unit 90 controls the simulator device 12 and the separate task execution device 14 so as to prompt the driver to perform a separate task processing different from the driving operation while the driver is performing the driving operation using the simulator device 12.

The driving operation information acquisition unit 92 acquires driving operation information input through the simulator device 12 during the simulation.

The separate task operation information acquisition unit 94 acquires the driver's answers to questions posed as separate tasks for testing the driver's cognitive function and operation information for providing the answers via the separate task execution device 14.

The determination unit 96 uses the output from the learning model 82 to determine the cognitive function of the driver based on various information acquired from the driver information acquisition unit 88, the driving operation information acquisition unit 92, and the separate task operation information acquisition unit 94. That is, if the direction of the driver's head or body acquired from the driving operation information acquisition unit 92 continues to be directed toward the separate task execution device 14 when processing the separate task, the cognitive function is determined to be low. Also, if the line of sight of the driver acquired from the driving operation information acquisition unit 92 continues to be directed toward the separate task execution device 14 when processing the separate task, the cognitive function is determined to be low. Furthermore, if the processing time when executing the separate task acquired from the separate task operation information acquisition unit 94 is longer than a predetermined time, the cognitive function is determined to be low. Furthermore, if the execution results of the separate task acquired from the separate task operation information acquisition unit 94 are more inaccurate than a predetermined number, the cognitive function is determined to be low. For these determinations, the cognitive function may be determined based on any one piece of information, or multiple pieces of information may be combined to determine the cognitive function. The determination unit 96 stores the determined cognitive function in the result DB 86.

The determination unit 96 also generates a vehicle user interface based on the driver's cognitive function. That is, as an example, when the driver's cognitive function is above average, the determination unit 96 maintains a preset vehicle user interface. On the other hand, when the driver's cognitive function is lower than a predetermined value, a vehicle user interface with improved operability is generated according to the driver's cognitive function. As an example, in this embodiment, a vehicle user interface for operating each function of the vehicle displayed on a center display provided in the center of the vehicle's instrument panel is generated based on the driver's cognitive function. Specifically, when the driver's cognitive function is above average, a normal vehicle user interface that is preset in consideration of the appearance design is maintained as shown in FIG. 9(A). On the other hand, when the driver's cognitive function is lower than a predetermined value, the wiper switch 97 and the wiper speed switch 99 as the operation unit are disposed at a position away from the steering wheel 89, and the size of each switch is adjusted to a size that is easy to press, as shown in FIG. 9(B). As a result, since the switches are not arranged near the steering wheel 89, the driver can be prevented from thinking that the switches are located near the back side of the steering wheel 89 and looking into the vicinity of the back side of the steering wheel 89, i.e., from looking away while driving. In other words, in the case of a driver with a relatively low cognitive function, the driver is distracted when operating each switch and the time spent in the inattentive driving state is relatively long, so a vehicle user interface is generated in which at least one of the position, shape, and size of the operation unit is adjusted so as to shorten the time spent in the inattentive driving state. In this embodiment, the vehicle user interface is a wiper operation screen, but is not limited to this, and may be a user interface for realizing other functions such as a vehicle operation screen, a car navigation screen, an audio screen, etc. In addition, the operation unit is not limited to the wiper switch 97 and the wiper speed switch 99, and may be something other than these. Furthermore, if the driver's cognitive function is lower than a predetermined value, a vehicle user interface that cannot be operated in the first place may be generated.

Furthermore, the determination unit 96 may generate a vehicle user interface based on at least one of the driving speed, the road surrounding conditions, and the number of accidents, as well as the driver's cognitive function. That is, if the number of accidents caused by the driver in the simulation while performing another task exceeds a predetermined threshold, a vehicle user interface that cannot be operated or does not display anything is generated while the driver is driving. In addition, in the simulation, when there are people around the road or in complex traffic conditions such as passing an intersection, the driving control tends to be inappropriate, such as not slowing down the driving speed, or when the operation timing tends to be inappropriate, such as when trying to perform processing of another task, a vehicle user interface that cannot be operated or does not display anything is generated while the driver is driving, even if the driving is not safe. Furthermore, for a driver whose line of sight is in a distracted driving state for more than a predetermined time when performing another task under complex road conditions in the simulation, a vehicle user interface that cannot be operated or does not display anything is generated while the driver is driving. (A) Note that, not limited to the above example, the vehicle user interface may be generated so as to be changed depending on the driver who is estimated to have concerns about safe driving and the driving conditions. As an example, when a driving speed is high or a complex road condition such as an intersection is detected, a vehicle user interface is generated in which no operation is possible or nothing is displayed until the condition ends or a predetermined time has passed. In addition, when the driver's cognitive function is relatively low, a greater distance from the vehicle in front is generally required for safe driving, and a vehicle user interface may be generated that dynamically changes so that a vehicle user interface in which no operation is possible or nothing is displayed is displayed according to the vehicle distance and the driver's cognitive function. Furthermore, a vehicle user interface may be generated that dynamically changes according to factors other than the vehicle distance and the driver's cognitive function.

The display processing unit 81 performs processing to display at least one of the result of the driver's cognitive function determined by the determination unit 96 and the vehicle user interface. The display processing unit 81 may transfer the vehicle user interface generated by the determination unit 96 to an actual vehicle, or may make the data of the generated vehicle user interface available for download. Furthermore, the generated vehicle user interface may be displayed on the display device 40 for research into vehicle user interfaces.

The communication unit 80 communicates with various devices via a network N, which may include a mobile phone communication network, a wired LAN (Local Area Network), a wireless LAN, and the Internet, or via a wiring cable C. In this embodiment, the communication unit 80 communicates with various devices via the network N. The communication unit 80 transmits data provided by the processing unit 78 to other devices, and provides data received from other devices to the processing unit 78.

(flowchart)
Next, a process executed by the information processing system 10 according to the present embodiment will be described. Fig. 8 is a flowchart showing an example of a process executed by the information processing system. The processor 20 reads out the generation program according to the present embodiment from the memory 18 or the storage 24, expands it in the memory 18, and executes it, thereby performing a process based on the generation program according to the present embodiment.

The processor 20 accepts input of information related to the personal attributes of the driver (step S100). The processor 20 then determines whether or not an operation to perform a test of the driver's cognitive function has been performed (step S102). If an operation to perform the test has not been performed (step S102: NO), the processor 20 repeats the process of step S102. On the other hand, if an operation to perform the test has been performed (step S102: YES), the processor 20 performs a driving simulation (step S104) and starts acquiring operation information of the driver's simulator device 12 in the simulation (step S106). The processor 20 also starts photographing the driver (step S108).

The processor 20 requests the driver to answer the question output to the separate task execution device 14 as a separate task (step S110). The processor 20 then determines whether the driver has answered the question (step S112). If the driver has not answered the question (step S112: NO), the processor 20 transitions to step S110. On the other hand, if the driver has answered the question (step S110: YES), the processor 20 acquires the driver's operation information inputting the answer to the question (step S114).

The processor 20 determines whether or not the execution of at least one of the simulation and the other task has ended (step S116). If the execution has not ended (step S116: NO), the processor 20 proceeds to step S114. On the other hand, if the execution has ended (step S116: YES), the processor 20 stores various information such as driving operation information and operation information of the other task in the memory unit 76 (step S118).

The processor 20 determines the driver's cognitive function based on various information such as driving operation information and operation information for other tasks (step S120). The processor 20 then generates a vehicle user interface based on the determined cognitive function (step S122).

The processor 20 determines whether or not an end operation has been performed for the cognitive function test or the generation of the vehicle user interface (step S124). If an end operation has not been performed (step S124: NO), the processor 20 proceeds to step S100. On the other hand, if an end operation has been performed (step S124: YES), the processor 20 ends the processing based on the generation program.

(Actions and Effects of the First Embodiment)
Next, the operation and effects of this embodiment will be described.

In this embodiment, the driver's cognitive function is evaluated based on information about at least one of the driver's driving operation, the processing of the answer, and the driver's state when answering questions posed as separate tasks during driving operations in a simulation. In other words, because the driver's cognitive function is tested and evaluated in a multitasking state, it is possible to test the cognitive function under conditions that are more stressful than a single-tasking cognitive function test and that are closer to the actual driving state of a vehicle. Therefore, the driver's cognitive function can be used as a realistic and highly effective indicator.

In addition, since a vehicle user interface is generated based on the evaluation results of the cognitive function of the vehicle being driven, a vehicle user interface that also takes into account the appearance design can be applied to drivers with high cognitive function, while a vehicle user interface with higher operability that takes safety into account can be applied to drivers with low cognitive function. In other words, it is possible to achieve both improved marketability and improved safety depending on the driver. This makes it possible to generate an appropriate vehicle user interface.

Furthermore, the information processing system 10 acquires driving information using a simulator device 12 that executes a driving simulation, and thus can acquire driving information without actually entering a dangerous driving state. This allows a vehicle user interface to be generated that is appropriate for the driver in a safe state.

Furthermore, the information processing system 10 acquires posture information including at least the direction of the driver's head as the state of the driver in the driver information acquisition unit 88. Therefore, it is possible to accurately detect the so-called inattentive driving state seen in drivers with relatively low cognitive function, in which the head is turned toward the separate task execution device 14 for a long period of time while processing another task. This makes it possible to accurately determine the driver's detection function, and furthermore to generate a vehicle user interface appropriate for the driver.

The information processing system 10 also acquires gaze information, which is information related to the driver's gaze, as the driver's state using the driver information acquisition unit 88. Therefore, it is possible to accurately detect a distracted driving state in which the driver's gaze is not directed in the direction required for driving, but is directed toward the separate task execution device 14 for a long period of time. This allows the driver's detection function to be accurately determined, making it possible to generate a vehicle user interface that is more appropriate for the driver.

Furthermore, the information processing system 10 acquires information related to at least the processing time when executing another task as the driver's operation information when executing another task processing in the input information acquisition unit 74. In general, if the processing time is long, the driver's cognitive function is relatively low, and if the processing time is short, the driver's cognitive function is relatively high. Therefore, the driver's cognitive function can be evaluated from the processing time. This makes it possible to generate an appropriate vehicle user interface according to the driver.

Furthermore, the information processing system 10 acquires at least information related to the execution result of another task as the driver's operation information when executing another task process in the input information acquisition unit 74. In general, if the execution result is accurate, the driver's cognitive function is relatively high, and if the execution result is inaccurate, the driver's cognitive function is relatively low. Therefore, the driver's cognitive function can be evaluated from the information related to the execution result. This makes it possible to generate an appropriate vehicle user interface according to the driver.

In addition, the information processing system 10 generates a vehicle user interface after adjusting at least one of the position, shape, and size of the operation unit (wiper switch 97 and wiper speed switch 99) that accepts operational input in the vehicle user interface based on the evaluation results of the driver's cognitive function. Therefore, the ease of operation of the operation unit of the vehicle user interface can be changed according to the driver. In other words, a vehicle user interface that is more appropriate for the driver can be generated.

Second Embodiment
Next, an information processing system 100 according to a second embodiment of the present invention will be described with reference to Figures 10 to 13. The information processing system 100 according to the second embodiment has a basic configuration similar to that of the first embodiment, and is characterized in that the information processing system 100 generates a user interface for a vehicle based on housing information and surrounding information of a display device that displays the user interface for a vehicle.

(System configuration)
10, the information processing system 100 is a system mounted on the vehicle 102 to evaluate the cognitive functions of a driver required for driving the vehicle 102 and generate a vehicle user interface suitable for the driver. The information processing system 100 according to this embodiment includes a vehicle operation device 104, a separate task execution device 14, and an evaluation device 106, which are communicatively connected to each other via a harness (not shown).

(Hardware configuration - vehicle operation device)
The vehicle operation device 104 is a device for operating the vehicle 102, and specifically corresponds to a steering wheel 108, an accelerator pedal 110, a brake pedal 112, a shift lever 114, a turn signal lever 116, a light switch (not shown), etc. Operation information of these devices can be acquired as appropriate and transmitted to the separate task execution device 14 and the evaluation device 106.

(Hardware configuration - evaluation device)
The evaluation device 106 includes a processor 20, a memory 22, a storage 24, a communication I/F 26, an input/output I/F 28, an imaging device 122, a display device 124, and an input device 126, which are communicatively connected to each other via a bus B. The imaging device 122 includes a driver's camera 128 that is provided to capture an image of the driver, and a driving monitoring camera 130 that monitors the driving state of the vehicle. The driver's camera 128 acquires video information that captures the driver's appearance with the driver's face as the center. The driving monitoring camera 130 acquires video information that captures the driving state of the vehicle 102.

In this embodiment, the display device 124 and the input device 126 are, as an example, an integrated touch panel display. The display device 124 and the input device 126 are arranged in a position where they can be seen and operated by the driver who is performing the cognitive function test. The display device 124 is a device that displays a vehicle user interface, and housing information, which is information about the housing of the display device 124, and surrounding information of the display device 124 are stored as part of the operation unit information 132 described below. The housing information specifically includes information such as the size of the housing constituting the display device 124 and its position and orientation within the vehicle 102. The surrounding information specifically includes information about objects (such as the steering wheel 108, as an example) present around the display device 124.

(Functional configuration - evaluation device)
Next, the functional configuration of the evaluation device 106 will be described. Fig. 11 is a diagram showing an example of the functional configuration of the evaluation device 106. When executing various programs, the evaluation device 106 realizes various functions using the above-mentioned hardware resources. The evaluation device 106 has a storage unit 134, a processing unit 136, and a communication unit 80 as functional configurations realized by the evaluation device 106. Each functional configuration is realized by the processor 20 reading and executing the generation program 58 stored in the memory 22 or the storage 24.

The memory unit 134 stores the generation program 58, the learning model 82, the user DB 84, the result DB 86, and the operation unit information 132. The operation unit information 132 includes operation information of the vehicle operation device 104 operated by the driver, housing information of the display device 124, and surrounding information of the housing.

The processing unit 136 has a driver information acquisition unit 88, an examination control unit 140, a driving operation information acquisition unit 142, a separate task operation information acquisition unit 94, a judgment unit 144, and a display processing unit 145. The examination control unit 140 controls an examination for evaluating the cognitive function of the driver. That is, when the driver is driving the vehicle 102 using the vehicle operation device 104, the examination control unit 140 controls the separate task execution device 14 to prompt the driver to perform separate task processing that is different from the driving operation.

The driving operation information acquisition unit 142 acquires driving operation information input through the vehicle operation device 104 while driving the vehicle 102, and information on the driving state of the vehicle 102 input through the driving monitoring camera 130. Examples of information on the driving state include the position of the vehicle 102 in the lane, and information on its positional relationship with other vehicles, including light vehicles such as bicycles, and pedestrians.

The determination unit 144 determines the driver's cognitive function based on various information acquired from the driver information acquisition unit 88, the driving operation information acquisition unit 142, and the separate task operation information acquisition unit 94. The determination unit 144 stores the determined cognitive function in the result DB 86.

The determination unit 144 also generates a vehicle user interface based on the driver's cognitive function and the housing information of the display device 124. That is, as an example, as shown in FIG. 13(A), when the driver's cognitive function is average, the determination unit 144 maintains a preset vehicle user interface. In contrast, when the driver's cognitive function is lower than a predetermined value, the determination unit 144 generates a vehicle user interface with improved operability according to the driver's cognitive function. Specifically, as shown in FIG. 13(B), the determination unit 144 estimates a position where the driver can easily place his/her hands (hereinafter, simply referred to as a "hand placement position"; corresponding to point P in the figure) from the housing information of the display device 124 and the surrounding information of the display device 124. This hand placement position P is determined from the surrounding information to be a position where the driver can easily place his/her hands without interfering with objects (for example, the steering wheel 108) around the display device 124. The determination unit 144 then generates a vehicle user interface that positions the wiper switch 97 and the wiper speed switch 99 as operation units along the hand placement position P and the outer edge of the housing of the display device 124 from the hand placement position P.

The display processing unit 145 performs processing to display the vehicle user interface generated by the determination unit 144 on the display device 124.

(flowchart)
Next, a process executed by the information processing system 100 according to the present embodiment will be described. Fig. 12 is a flowchart showing an example of a process executed by the information processing system 100. The processor 20 reads out the generation program according to the present embodiment from the memory 18 or the storage 24, expands it in the memory 18, and executes it, thereby performing a process based on the generation program according to the present embodiment. Note that the same processes as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

After processing step S100, the processor 20 determines whether or not an operation has been performed to perform a judgment run to test the driver's cognitive function (step S200). If an operation to perform a judgment run has not been performed (step S200: NO), the processor 20 repeats the processing of step S200. On the other hand, if an operation to perform a judgment run has been performed (step S200: YES), the processor 20 transitions to the processing of step S106 to acquire various information during the judgment run.

After processing step S114, the processor 20 determines whether or not the execution of at least one of the judgment driving and the other task has ended (step S202). If the execution has not ended (step S202: NO), the processor 20 proceeds to step S114. On the other hand, if the execution has ended (step S202: YES), the processor 20 stores various information such as the driving operation information and the operation information of the other task in the memory unit 76 (step S118).

(Actions and Effects of the Second Embodiment)
Next, the operation and effects of the second embodiment will be described.

The above configuration is similar to the information processing system 10 of the first embodiment, except that the vehicle user interface is generated based on the housing information and surrounding information of the display device that displays the vehicle user interface, and therefore the same effects as the first embodiment can be obtained. In addition, the information processing system 100 acquires housing information of the display device 124 that displays the vehicle user interface and surrounding information of the display device 124, and generates the vehicle user interface based on the housing information and surrounding information. Therefore, the vehicle user interface can be generated so that the driver can operate the operation unit 146 with his/her hands placed at the hand placement position P where the driver can easily place his/her hands or on the outer edge of the housing of the display device 124. As a result, the driver can grasp the position of the operation unit 146 based on the hand placement position P or the outer edge of the housing by feeling around without staring at the display device 124, and then operate it. Therefore, for a driver with reduced cognitive function who generally tends to stare at the display device 124, a vehicle user interface can be provided that allows the driver to operate the wiper switch 97, the wiper speed switch 99, and the like without staring at the display device 124.

In addition, since the vehicle user interface is generated based on the housing information and surrounding information of the display device 124, it is possible to generate a vehicle user interface that takes into account the relative position of the evaluation device 124 within the vehicle cabin. In other words, since it is possible to generate an appropriate vehicle user interface that takes into account how the display device 124 appears to the driver, it is possible to provide an even more appropriate vehicle user interface.

In the first and second embodiments described above, the vehicle user interface is generated after adjusting at least one of the position, shape, and size of the operation unit (wiper switch 97 and wiper speed switch 99) that accepts operation input in the vehicle user interface based on the evaluation results of the driver's cognitive function. However, this is not limited to this, and the vehicle user interface may also be generated after adjusting at least one of the position, shape, and size of parts other than the operation unit, such as parts that display information in the vehicle user interface.

In addition, the vehicle user interface is a user interface displayed on a liquid crystal display or the like, but is not limited to this. It may also be a user interface including physical switches that is generated based on the results of the cognitive function assessment.

<Additional Notes>
The present embodiment includes the following disclosure.

(Appendix 1)
A method for generating a user interface for a vehicle executed by an information processing device, comprising:
A driving information acquisition step of acquiring information regarding a driver's driving operation;
A separate task request step of requesting the driver to execute a predetermined process different from the driving operation during the driving operation of the driver;
a driver state information acquisition step of acquiring information regarding at least one of the driving operation of the driver, the predetermined process, and a state of the driver while the driver is performing the predetermined process;
an evaluation step of evaluating a cognitive function of the driver based on the information acquired in the driver state information acquisition step;
a generating step of generating a user interface for a vehicle based on an evaluation result of the cognitive function of the driver determined in the evaluating step;
An information processing method comprising:

(Appendix 2)
The driving information acquisition step acquires information in a driving simulation that displays a driving video that is viewed by the driver and in which a situation changes based on a driving operation of the driver.
2. The information processing method according to claim 1.

(Appendix 3)
The information processing device acquires, in the driver state information acquisition step, posture information including at least a direction of the head of the driver as the state of the driver.
2. The information processing method according to claim 1.

(Appendix 4)
The information processing device acquires, in the driver state information acquisition step, gaze information that is information regarding at least the driver's gaze as the state of the driver.
2. The information processing method according to claim 1.

(Appendix 5)
The information processing device acquires, in the driver state information acquisition step, information regarding at least a processing time required for executing the predetermined processing as the information regarding the predetermined processing.
2. The information processing method according to claim 1.

(Appendix 6)
In the driver state information acquisition step, the information processing device acquires at least information regarding a result of execution of the predetermined processing as the information regarding the predetermined processing.
2. The information processing method according to claim 1.

(Appendix 7)
and generating the vehicle user interface by adjusting at least one of a position, a shape, and a size of an operation unit that accepts an operation input in the vehicle user interface based on the evaluation result.
2. The information processing method according to claim 1.

(Appendix 8)
the information processing device acquires case information, which is information related to a device case that displays the vehicular user interface, and surrounding information of the case, and in the generating step, generates the vehicular user interface based on the case information and the surrounding information.
2. The information processing method according to claim 1.

(Appendix 9)
the information processing device acquires case information, which is information related to a device case that displays the vehicular user interface, and surrounding information of the case, and in the generating step, generates the vehicular user interface based on the case information and the surrounding information.
2. The information processing method according to claim 1.

(Appendix 10)
A driving information acquisition device for acquiring driving operation information for driving a vehicle;
a separate task execution device that requests a driver who is driving a vehicle to execute a predetermined process other than driving a vehicle, accepts the execution, and acquires the execution result;
an evaluation device for evaluating a cognitive function of the driver based on at least one of information about the driver, the driving operation information obtained by the driving information acquisition device, and information about the predetermined processing obtained by the separate task execution device;
Equipped with
A driving information acquisition step of acquiring information regarding a driver's driving operation;
A separate task request step of requesting the driver to execute a predetermined process different from the driving operation during the driving operation of the driver;
a driver state information acquisition step of acquiring information regarding at least one of the driving operation of the driver, the predetermined process, and a state of the driver while the driver is performing the predetermined process;
an evaluation step of evaluating a cognitive function of the driver based on the information acquired in the driver state information acquisition step;
a generating step of generating a user interface for a vehicle based on an evaluation result of the cognitive function of the driver determined in the evaluating step;
An information processing system that executes the above.

The embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the claims, not by the meaning described above, and is intended to include all modifications within the scope of the claims. Furthermore, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

10 Information processing system 12 Driving information acquisition device 14 Other task execution device 16 Evaluation device (generation device)
97 Wiper switch (operation part)
99 Wiper speed switch (operation part)
100 Information processing system 106 Evaluation device (generation device)

According to an information processing method of one embodiment, a method for generating a user interface for a vehicle is executed by an information processing device, the method for generating a user interface for a vehicle being executed by an information processing device includes a driving information acquisition step of acquiring information regarding a driver's driving operation, a separate task request step of requesting the driver to execute a specified process different from the driving operation while the driver is performing the driving operation, a driver state information acquisition step of acquiring information regarding the driver's driving operation and information regarding at least one of the execution of the specified process and the driver's state while the driver is performing the specified process, an evaluation step of evaluating the cognitive function of the driver based on the information acquired in the driver state information acquisition step, and a generation step of generating a user interface for a vehicle based on the evaluation result of the driver's cognitive function determined by the evaluation step.

According to a program of one embodiment, a method for generating a user interface for a vehicle is executed by an information processing device, and the method causes a computer to execute the following steps: a driving information acquisition step for acquiring information regarding a driver's driving operation; a separate task request step for requesting the driver to perform a specified process other than the driving operation while the driver is performing the driving operation; a driver state information acquisition step for acquiring information regarding the driver's driving operation and information regarding at least one of the execution of the specified process and the driver's state while the driver is performing the specified process; an evaluation step for evaluating the cognitive function of the driver based on the information acquired in the driver state information acquisition step; and a generation step for generating a user interface for a vehicle based on the evaluation result of the driver's cognitive function determined by the evaluation step.

According to one embodiment of the information processing system, the information processing system includes a driving information acquisition device that acquires driving operation information for driving a vehicle; a separate task execution device that requests a driver who is driving to execute a predetermined process other than vehicle driving, accepts the execution, and acquires the execution result; an evaluation device that evaluates the cognitive function of the driver from at least one of information about the driver, the driving operation information by the driving information acquisition device, and information about the execution of the predetermined process acquired by the separate task execution device; and a generation device that generates a user interface for a vehicle based on the evaluated cognitive function. The information processing system executes the following steps: a driving information acquisition step that acquires information about the driving operation of the driver; a separate task request step that requests the driver to execute a predetermined process different from the driving operation while the driver is performing the predetermined process; a driver state information acquisition step that acquires information about the driver's driving operation and information about at least one of the execution of the predetermined process and the driver's state while the driver is performing the predetermined process; an evaluation step that evaluates the cognitive function of the driver based on the information acquired in the driver state information acquisition step; and a generation step that generates a user interface for a vehicle based on the evaluation result of the cognitive function determined in the evaluation step.

Claims (10)

A method for generating a user interface for a vehicle executed by an information processing device, comprising:
A driving information acquisition step of acquiring information regarding a driver's driving operation;
A separate task request step of requesting the driver to execute a predetermined process different from the driving operation during the driving operation of the driver;
a driver state information acquisition step of acquiring information regarding at least one of a driving operation of the driver, the execution of the predetermined process, and a state of the driver while the driver is executing the predetermined process;
an evaluation step of evaluating a cognitive function of the driver based on the information acquired in the driver state information acquisition step;
a generating step of generating a user interface for a vehicle based on an evaluation result of the cognitive function of the driver determined in the evaluating step;
An information processing method comprising: The driving information acquisition step acquires information in a driving simulation that displays a driving video that is viewed by the driver and in which a situation changes based on a driving operation of the driver.
The information processing method according to claim 1 . The information processing device acquires, in the driver state information acquisition step, posture information including at least a direction of the head of the driver as the state of the driver.
The information processing method according to claim 1 . The information processing device acquires, in the driver state information acquisition step, gaze information that is information regarding at least the driver's gaze as the state of the driver.
The information processing method according to claim 1 . The information processing device acquires, in the driver state information acquisition step, information regarding at least a processing time required for executing the predetermined processing as the information regarding the predetermined processing.
The information processing method according to claim 1 . In the driver state information acquisition step, the information processing device acquires at least information regarding a result of execution of the predetermined processing as the information regarding the predetermined processing.
The information processing method according to claim 1 . and generating the vehicle user interface by adjusting at least one of a position, a shape, and a size of an operation unit that accepts an operation input in the vehicle user interface based on the evaluation result.
The information processing method according to claim 1 . the information processing device acquires case information, which is information related to a device case that displays the vehicular user interface, and surrounding information of the case, and in the generating step, generates the vehicular user interface based on the case information and the surrounding information.
The information processing method according to claim 1 . A driving information acquisition step of acquiring information regarding a driver's driving operation;
A separate task request step of requesting the driver to execute a predetermined process different from the driving operation during the driving operation of the driver;
a driver state information acquisition step of acquiring information regarding at least one of a driving operation of the driver, the execution of the predetermined process, and a state of the driver while the driver is executing the predetermined process;
an evaluation step of evaluating a cognitive function of the driver based on the information acquired in the driver state information acquisition step;
a generating step of generating a user interface for a vehicle based on an evaluation result of the cognitive function of the driver determined in the evaluating step;
A program that causes a computer to execute the following. A driving information acquisition device that acquires driving operation information for driving a vehicle;
a separate task execution device that requests a driver who is driving to execute a predetermined process other than vehicle driving, accepts the execution, and acquires the execution result;
an evaluation device for evaluating a cognitive function of the driver based on at least one of information about the driver, the driving operation information obtained by the driving information acquisition device, and information about the execution of the predetermined process obtained by the separate task execution device;
a generating device for generating a user interface for a vehicle based on the evaluated cognitive functions;
Equipped with
A driving information acquisition step of acquiring information regarding the driving operation of the driver;
A separate task request step of requesting the driver to execute a predetermined process different from the driving operation during the driving operation of the driver;
a driver state information acquisition step of acquiring information regarding at least one of a driving operation of the driver, the execution of the predetermined process, and a state of the driver while the driver is executing the predetermined process;
an evaluation step of evaluating a cognitive function of the driver based on the information acquired in the driver state information acquisition step;
a generating step of generating a user interface for a vehicle based on the evaluation result of the cognitive function determined in the evaluating step;
An information processing system that executes the above.

Images