CN116594858A - A human-computer interaction evaluation method and system for an intelligent cockpit - Google Patents

Abstract

The invention provides a human-computer interaction evaluation method and system for an intelligent cockpit, which provides a virtual driving scene through an XR head-mounted device, and collects multi-modal data through a virtual simulation driver to realize eye control, brain control and other interactive methods for operators to carry out Interactive, simulate real driving scenarios, and obtain test data close to real scenarios while ensuring safety. Introduce multi-modal data of human factors, based on vital sign data and operator behavior data generated during the interaction process, calculate and form a multi-dimensional score including cognitive load score, comfort score and behavioral efficiency score during the interaction process, and build a smart The all-round evaluation plan for cockpit design, fully automatic evaluation, improves the evaluation efficiency and reduces the influence of individual subjective factors, multi-modal data quantitative evaluation, higher reliability.

Description

A human-computer interaction evaluation method and system for an intelligent cockpit

technical field

The invention relates to the technical field of intelligent driving, in particular to a human-computer interaction evaluation method and system for an intelligent cockpit.

Background technique

With the continuous innovation of intelligent driving technology, the human-computer interaction functions of the intelligent cockpit are becoming more and more abundant, so as to realize the monitoring of occupant status and safe driving assistance during driving. At the same time, the innovative application of AI, big data, 5G and other technologies has also brought more and more diversified multi-modal forms of human-computer interaction in the smart cockpit, and the interaction scenarios have become more and more refined. For example, smart car DMS system (driver monitoring system). The DMS system can realize the real-time monitoring of the driver's driving fatigue, distraction and other dangerous behaviors (such as making phone calls, eating, etc.), and can make timely warnings for abnormal states.

On this basis, the demand for human-computer interaction evaluation of intelligent cockpit design is also getting higher and higher. The design evaluation of intelligent cockpit traditionally relies on the experience of experts or engineers/designers, subjective evaluation is the main method, and the test environment is also relatively complicated. On the one hand, it cannot guarantee safety, on the other hand, it lacks objective and systematic evaluation technology, and lacks perfection. And the automated evaluation scheme of the system, the efficiency is low, and it also affects the safety of later products and the experience of operators.

Contents of the invention

In view of this, the embodiment of the present invention provides a smart cockpit human-computer interaction evaluation method and system to eliminate or improve one or more defects in the prior art, and solve the problem that the prior art cannot provide intelligent cockpit that can simulate An automated evaluation solution for real scenarios.

Technical scheme of the present invention is as follows:

On the one hand, the present invention provides an intelligent cockpit human-computer interaction evaluation method, the method is executed on the cockpit human-computer interaction design evaluation recommendation subsystem, and the cockpit human-computer interaction design evaluation recommendation subsystem is connected to the simulated driver module. The simulated driver module is connected to the XR cockpit interaction module, and the simulated driver module provides a virtual driving scene for the operator to interact with by the XR head-mounted device, including:

Acquiring interactive data, the interactive data is collected by the interactive data acquisition component of the simulated driver module from the operator in the cockpit during the interaction process; the XR cockpit interactive module is based on the interactive data based on the preset The interactive control scheme makes interactive decisions and feeds back to the simulated driver module to achieve continuous interaction;

Based on the multimodal human factor data collected by the interactive data collection component, the human-computer interaction behavior is calculated in multiple scoring dimensions, and the weighted average is calculated to obtain the strategy score of the interaction control scheme; the scoring dimension includes recognition. cognitive load score, comfort score, and behavioral efficiency score; wherein, the multimodal human factor data includes: EEG data, eye movement data, physiological data, near-infrared data, and operator behavior data.

In some embodiments, the EEG data includes EEG signal time domain, frequency domain and nonlinear indicators; the eye movement data includes pupil diameter, fixation point coordinates and their corresponding fixation duration; the physiological data includes Skin temperature data, electrodermal data, and respiratory data; the near-infrared data includes blood oxygen data, total hemoglobin concentration, and heart rate variability data; the respiratory data includes respiratory rate, tidal volume, and vital capacity; the operator behavior data Including the completion rate of driving behavior and the duration of stimulus response.

In some embodiments, the cognitive load score is obtained by evaluating the first type of interaction data, and the first type of interaction data includes the EEG signal time domain, frequency domain and nonlinear indicators, the pupil diameter, The blood oxygen data and the total hemoglobin concentration value;

The comfort score is obtained by evaluating the second type of interaction data, and the second type of interaction data includes the heart rate variability data, the skin temperature data, and the electrodermal data;

The behavioral efficiency score is obtained by evaluating the third type of interaction data, and the third type of interaction data includes the execution completion rate and the stimulus response duration.

In some embodiments, the method includes:

Mapping the first type of interaction data to the cognitive load score using a pre-trained first neural network, the cognitive load score comprising a first set number of grade scores;

using a pre-trained second neural network to map the second type of interaction data to the comfort score, where the comfort score includes a second set number of grade scores;

A pre-trained third neural network is used to map the third type of interaction data to the behavior efficiency score, and the behavior efficiency score includes a third set number of grade scores.

In some embodiments, the first neural network, the second neural network and the third neural network are decision trees or BP neural networks.

In some embodiments, the first neural network, the second neural network and the third neural network each comprise a convolutional neural network.

In some embodiments, before calculating the weighted average to obtain the strategy score of the interaction control scheme, it also includes:

Normalize the grade scores of each scoring dimension.

In some embodiments, the method further includes: calculating strategy scores corresponding to multiple interaction control schemes, and selecting the interaction control scheme with the highest strategy score as the optimal interaction control scheme.

On the other hand, the present invention also provides an intelligent cockpit human-computer interaction evaluation system, including:

The simulated driver module includes a simulated cockpit and an interactive data acquisition component, the cockpit is used to provide a hardware environment for simulated driving, and the interactive data acquisition component includes an XR head-mounted device for brain-controlled and eye-controlled interaction, a setting The skin temperature sensor and near-infrared sensor on the steering wheel, the breathing sensor on the seat belt, and the interactive behavior sensor in the cockpit; the EEG data and eye movement data collected by the XR head-mounted device , the skin temperature skin electricity sensor collects skin temperature data and skin electricity data, the near infrared data collected by the near infrared sensor, the respiratory sensor collects respiratory data, and the interactive behavior sensor is used to collect operator behavior data;

The XR cockpit interaction module is used to control the XR head-mounted device to present a virtual driving scene, and to analyze the EEG data, the eye movement data, the skin temperature data, and the skin electricity based on a preset interactive control scheme. data, the near-infrared data, the respiration data and the operator behavior data to make an interactive decision, and feed back the interactive decision to the simulation value module for execution;

The cockpit human-computer interaction design evaluation and recommendation subsystem is used to implement the above-mentioned intelligent cockpit human-computer interaction evaluation method to obtain the strategy score of the interaction control scheme.

In some embodiments, the system is also wired or wirelessly connected to the cloud server, and uploads the strategy score of the interaction control scheme.

The beneficial effects of the present invention are at least:

The intelligent cockpit human-computer interaction evaluation method and system of the present invention provides a virtual driving scene through an XR head-mounted device, and collects multi-modal data through a simulated driver to realize eye control, brain control and other interactive modes for operators to interact. Simulate real driving scenarios and obtain test data close to real scenarios while ensuring safety. Introduce human factor multimodal data, based on the vital sign data and operator behavior data generated during the interaction process, calculate and form a multi-dimensional score including cognitive load score, comfort score and behavioral efficiency score during the interaction process, and build a smart The all-round evaluation plan of the cockpit design, conducts automatic evaluation, improves the evaluation efficiency and eliminates the influence of human subjective factors, and has higher reliability.

Additional advantages, objects, and features of the present invention will be set forth in part in the following description, and will be partly apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

It will be understood by those skilled in the art that the objects and advantages that can be achieved by the present invention are not limited to the above specific ones, and the above and other objects that can be achieved by the present invention will be more clearly understood from the following detailed description.

Description of drawings

The drawings described here are used to provide further understanding of the present invention, constitute a part of the application, and do not limit the present invention. In the attached picture:

Fig. 1 is a schematic structural diagram of the intelligent cockpit human-computer interaction evaluation system according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of the calculation structure of the cockpit human-computer interaction design scheme score in the intelligent cockpit human-computer interaction evaluation method according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of the evaluation logic of the interactive control scheme by the intelligent cockpit human-computer interaction evaluation method according to an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with the embodiments and accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

Here, it should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only the structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and the related Other details are not relevant to the invention.

It should be emphasized that the term "comprising/comprising" when used herein refers to the presence of a feature, element, step or component, but does not exclude the presence or addition of one or more other features, elements, steps or components.

The smart cockpit can be defined as an intelligent service system that can proactively gain insight into and understand the needs of operators and meet the needs of operators; starting from the needs of end consumers and application scenarios, passengers not only do not need to worry about driving and travel, but can also Get a comfortable experience in the smart cockpit. The car based on the smart cockpit will bid farewell to the role of only a travel tool, and the goal is to realize the intelligent interaction between the cockpit and people, vehicles and roads. Compared with the current cockpit, this form of smart cockpit will be more intelligent and humanized. The intelligent cockpit can provide a complete set of interactive control solutions, and can generate and execute interactive decisions based on the physiological characteristic data and operator behavior data generated by the operator during driving, including but not limited to emergency braking and steering. Driving assistance, driver fatigue monitoring and warnings about dangerous driving behaviors, and intelligent control of the driving environment in the car. For example, the smart car DMS system can detect the driver's driving fatigue, distraction and other dangerous behaviors, and it can also adjust the temperature of the air conditioner in the car based on the driver's body temperature and heart rate changes. However, whether the interactive control scheme provided by the intelligent cockpit is suitable for the scene requirements or whether it can achieve the expected effect needs to be accurately evaluated.

On the one hand, the present invention provides an intelligent cockpit human-computer interaction evaluation method, the method is executed on the cockpit human-computer interaction design evaluation recommendation subsystem, the cockpit human-computer interaction design evaluation recommendation subsystem is connected to the simulated driver module, and the simulated driver The module is connected to the XR cockpit interaction module, and the simulated driver module is provided with a virtual driving scene by the XR head-mounted device for the operator to interact with, including steps S101-S102:

Step S101: Obtain interactive data, which is collected by the interactive data acquisition component of the simulated driver module from the operator in the cockpit during the interaction process; the XR cockpit interactive module makes a decision based on the interactive data based on the preset interactive control scheme Interaction decisions are made and fed back to the simulated driver module for continuous interaction.

Step S102: Based on the multi-modal human factor data collected by the interaction data collection component, calculate grade scores for human-computer interaction behavior in multiple scoring dimensions, and calculate the weighted average to obtain the strategy score of the interaction control scheme; the score dimension includes cognitive load score , comfort score and behavioral efficiency score; among them, multimodal human factors data include: EEG data, eye movement data, physiological data, near-infrared data and operator behavior data.

In this embodiment, the simulated driver module includes a cockpit and an interactive data collection component, wherein the cockpit is used to provide a driving environment and required hardware equipment, such as: seats, steering wheel, air conditioning system, audio-visual system, central control equipment and other control keys etc. Interactive data acquisition components, including XR head-mounted equipment for brain-controlled and eye-controlled interactions, skin-to-skin electrical sensors and near-infrared sensors set on the steering wheel, breathing sensors set on seat belts, and set in the cockpit The interactive behavior sensor in the device; the EEG data and eye movement data collected by the XR headset, the skin temperature data and skin electricity data collected by the skin temperature sensor, the near infrared data collected by the near infrared sensor, and the respiratory data collected by the respiratory sensor. Behavior sensors are used to collect operator behavior data.

The XR cockpit interaction module responds to the physiological characteristic data collected by the interactive data component and the behavior of the operator according to the preset interactive control scheme, forms an interaction strategy and provides it to the operator, and continuously adjusts the strategy during the interaction process to form the continuity of the interaction .

In step S102, the cockpit human-computer interaction design evaluation and recommendation subsystem acquires the physiological characteristic data and operator behavior data collected by the interactive data collection component, and evaluates the interactive control scheme in multiple scoring dimensions, including the operator's cognition load, comfort, and behavioral efficiency. Among them, the evaluation of cognitive load can reflect whether some intelligent interaction schemes are easy to be accepted by customers, whether there are obstacles to understanding, and whether learning costs are required. The evaluation of comfort can reflect the experience of the operator, such as whether the provided temperature control and interior light control strategies are appropriate. The evaluation of the efficiency of behavior is mainly to evaluate the response time and accuracy required for the operator to perform corresponding actions on the basis of providing driving assistance. Perform driving maneuvers with accuracy and reaction time. The present invention calculates the weighted average value for the scores of these evaluation dimensions to obtain the strategy score of the interactive control strategy.

It should be noted that the present invention is based on the virtual driving scene provided by the XR head-mounted device for the operator to interact with, combined with the hardware equipment in the cockpit, it can realize the simulation of real road conditions in the virtual scene. No road test is required, which ensures the safety of the detection process, especially the detection of functions related to intelligent assisted driving, which can ensure zero accidents.

In some embodiments, EEG data includes EEG signal time domain, frequency domain, and nonlinear indicators; eye movement data includes pupil diameter, fixation point coordinates, and corresponding fixation duration; physiological data includes skin temperature data, skin electrical Data and respiratory data; near-infrared data includes blood oxygen data, total hemoglobin concentration and heart rate variability data; respiratory data includes respiratory rate, tidal volume, and lung capacity; operator behavior data includes the completion rate of driving behavior and the duration of stimulus response. It should be understood by those skilled in the art that the parameters that can be cited in the present invention not only include the above-mentioned content, but also can be used as a parameter for evaluating cognitive load, comfort and behavioral efficiency for any other physiological characteristic data and operator interaction data. parameter.

In some embodiments, the cognitive load score is obtained by evaluating the first type of interactive data, the first type of interactive data includes EEG signal time domain, frequency domain and nonlinear indicators, pupil diameter, blood oxygen data and total hemoglobin concentration value .

The comfort score is obtained by evaluating the second type of interactive data, which includes heart rate variability data, skin temperature data, and skin electrical data.

The behavioral efficiency score is obtained by evaluating the third type of interaction data, which includes execution completion rate and stimulus response time.

In some embodiments, the method includes:

A pre-trained first neural network is used to map the first type of interaction data to a cognitive load score, and the cognitive load score includes a first set number of grade scores.

The second type of interaction data is mapped to a comfort score by using a pre-trained second neural network, and the comfort score includes a second set number of grade scores.

A pre-trained third neural network is used to map the third type of interaction data to a behavioral efficiency score, and the behavioral efficiency score includes a third set number of ratings.

Specifically, in this embodiment, it is first necessary to pre-train the first neural network, the second neural network, and the third neural network based on the existing data, and use the data in the existing database to add labels manually. A set of data that adds ratings on cognitive load, comfort, and behavioral efficiency. The scoring can be divided into multiple grades. For example, it can be divided into 5 grades from high to low, with excellent grades of 5 points, good grades of 4 points, average grades of 3 points, poor grades of 2 points and poor grades of 1 point.

In some embodiments, the first neural network, the second neural network and the third neural network are decision trees or BP neural networks. In other embodiments, the first neural network, the second neural network and the third neural network each include a convolutional neural network.

In some embodiments, before calculating the weighted average value to obtain the strategy score of the interaction control scheme, it further includes: normalizing the grade scores of each scoring dimension.

In some embodiments, the method further includes: calculating strategy scores corresponding to multiple interaction control schemes, and selecting the interaction control scheme with the highest strategy score as the optimal interaction control scheme.

On the other hand, the present invention also provides an intelligent cockpit human-computer interaction evaluation system, referring to Figure 1, including:

The simulated driver module includes a simulated cockpit and interactive data acquisition components. The cockpit is used to provide a hardware environment for simulated driving. The interactive data acquisition components include XR headsets for brain-controlled and eye-controlled interactions, and the steering wheel. Skin-to-skin electrical sensors and near-infrared sensors, breathing sensors installed on seat belts, and interactive behavior sensors installed in the cockpit; EEG data and eye movement data collected by XR headsets, skin-to-skin electrical sensors to collect skin Temperature data and electrodermal data, near-infrared data collected by near-infrared sensors, respiratory data collected by respiratory sensors, and interactive behavior sensors are used to collect operator behavior data.

The XR cockpit interaction module is used to control the XR head-mounted device to present virtual driving scenes, and based on the preset interactive control scheme to analyze EEG data, eye movement data, skin temperature data, skin electricity data, near-infrared data, respiratory data and operations Human behavior data makes interactive decisions, and the interactive decisions are fed back to the simulation value module for execution.

The cockpit human-computer interaction design evaluation and recommendation subsystem is used to execute the intelligent cockpit human-computer interaction evaluation method described in the above steps S101-S102, and obtain the strategy score of the interactive control scheme.

In some embodiments, the system is also wired or wirelessly connected to the cloud server, and uploads the policy score of the interactive control scheme.

Describe below in conjunction with specific embodiment:

Referring to FIG. 1 , this embodiment provides an XR cockpit interactive control system, which consists of an XR cockpit interactive module and a simulated driver module. The system is used for the design of the human-computer interaction interface in the cockpit, the realization of the design of the human-computer interaction control method, the presentation of the driving scene, and the collection and recording of various driving behavior data and physiological data of the driver.

Among them, the XR cockpit interaction module: used to present the design scheme of the cockpit interaction interface, the design of the human-computer interaction control method and the presentation of the driving scene, equipped with a variety of intelligent and natural human-computer interaction control methods, including but not limited to: brain-controlled interaction , eye control interaction, electrodermal interaction, skin temperature interaction, breathing interaction, myoelectric interaction, gesture interaction, etc. EEG data includes time domain, frequency domain and nonlinear indicators of EEG signals; eye movement data includes pupil diameter, fixation point coordinates and corresponding fixation duration; near-infrared data includes blood oxygen data, total hemoglobin concentration and heart rate variability Respiratory data include respiratory rate, tidal volume, and lung capacity; operator behavior data include execution completion rate of driving behavior and stimulus response time. It should be understood by those skilled in the art that the parameters that can be cited in the present invention not only include the above-mentioned content, but also can be used as a parameter for evaluating cognitive load, comfort and behavioral efficiency for any other physiological characteristic data and operator interaction data. parameter. The interactive command is fed back to the simulator to drive the operation of the simulated driver.

The design of the human-computer interaction control mode is selected by the operator according to the design scheme. The implementation methods of each control command include but are not limited to the following two:

First, custom threshold: the operator can set the threshold of the control instruction according to the needs. For example: eye control interaction, the output of commands can be realized by setting the gaze duration threshold to 2s.

Second, the algorithm adaptively adjusts the threshold value: the algorithm adaptively adjusts the threshold value according to the recognition of the operator's habits, driving scenes, and vehicle status. For example: eye control interaction, when it is recognized that the vehicle is driving at a high speed, the eye movement fixation duration threshold can be adaptively reduced. When it is recognized that the driving scene is more complex and requires more attention and cognitive resource consumption of the driver, the eye movement fixation duration threshold for interface control can be adaptively reduced.

Simulated driver module: used to implement interaction strategies, and collect and record various human-computer interaction data and physiological data of the driver. Collection methods include but are not limited to: Brain control and eye control interaction data are collected by XR headsets, skin temperature and electrodermal data are collected by the steering wheel of the driving simulator, breathing data are collected by the seat belt of the driving simulator, gestures Interaction data is collected by sensors built into the simulated driver.

Cockpit human-computer interaction design evaluation and recommendation subsystem: the system inputs multi-channel driver physiological data, human-computer interaction and other data, and evaluates the grade of cockpit human-computer interaction design according to the scoring dimension. The grades are divided into 5 levels (excellent 5, good 4, medium 3, poor 2, poor 1).

The evaluation dimensions include: the state or performance of the driver when completing the driving task in a specific driving scene, interactive interface and interaction mode, and each state/performance is scored on a 5-level scale. Including but not limited to cognitive load, comfort and efficiency.

Referring to Figure 2, the calculation method of the grade of the cockpit human-computer interaction design scheme is: the weighted average of the grade scores of each evaluation dimension.

Among them, S is the score of the cockpit human-computer interaction design scheme; W is the weight of each dimension; X is the rating of each dimension; n is the number of evaluation dimensions. The S value is rounded up and judged as the level of the cockpit plan.

The corresponding schemes of the evaluation dimensions and each sub-indicator include but are not limited to: building models through machine learning or deep learning algorithms (such as decision trees, neural networks, etc.), establishing specific operators’ physiological characteristic indicators, operators’ behavior data and cognition Mapping relationship of load score, comfort score and behavioral efficiency score. Correspond the operator's subjective score (1-5) on each dimension with its objective physiological index.

After normalizing the scores of each sub-indicator (the range is 0-1), take the weighted average (the range is 0-1), divide the values of 0-1 into quintiles, and map them to the grade score (1 ~5). The value of the weight is defined by experts.

Taking the above three evaluation dimensions as an example, the features selected for rating 1 of cognitive load (L) include but are not limited to: time domain, frequency domain and nonlinear indicators (theta/beta) of EEG signals during task completion, Near-infrared (oxygenation, deoxygenation, total hemoglobin concentration value), eye movement index (pupil diameter); the characteristics selected for rating comfort (H) include but not limited to: HRV frequency domain index (LF/HF), skin electrical index (SC); Efficiency (Q) ratings include but are not limited to: driving behavior indicators (accuracy and reaction time to complete tasks). Referring to Figure 3, the three elements of the cockpit interaction interface, cockpit interaction method, and driving scene constitute a complete interactive control scheme. In terms of cognitive load, evaluation is based on EEG indicators and eye movement indicators; in terms of comfort, evaluation is based on physiological indicators. Evaluation; Efficiency is evaluated based on driving behavior indicators.

This embodiment provides an automatic evaluation method for cockpit human-computer interaction design based on the combination of cockpit interaction interface and cockpit interaction mode in a specific driving scene, which helps to adapt the optimal human-computer interaction scheme under different driving conditions.

To sum up, the intelligent cockpit human-computer interaction evaluation method and system of the present invention provides virtual driving scenes through XR head-mounted devices, and realizes interactive methods such as eye control and brain control by collecting multi-modal data through simulated drivers for The operator interacts, simulates the real driving scene, and obtains test data close to the real scene while ensuring safety. Introduce human factor multimodal data, based on the vital sign data and operator behavior data generated during the interaction process, calculate and form a multi-dimensional score including cognitive load score, comfort score and behavioral efficiency score during the interaction process, and build a smart The all-round evaluation plan of the cockpit design, conducts automatic evaluation, improves the evaluation efficiency and eliminates the influence of human subjective factors, and has higher reliability.

Those of ordinary skill in the art should understand that each exemplary component, system and method described in conjunction with the embodiments disclosed herein can be implemented by hardware, software or a combination of the two. Whether it is implemented in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments employed to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted over transmission media or communication links by data signals carried in carrier waves. "Machine-readable medium" may include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. Code segments may be downloaded via a computer network such as the Internet, an Intranet, or the like.

It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps may be performed in the order mentioned in the embodiment, or may be different from the order in the embodiment, or several steps may be performed simultaneously.

In the present invention, features described and/or exemplified for one embodiment can be used in the same or similar manner in one or more other embodiments, and/or can be combined with features of other embodiments or replace other Features of the implementation.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, various modifications and changes may be made to the embodiments of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. The method is executed on a cabin man-machine interaction design evaluation recommendation subsystem, the cabin man-machine interaction design evaluation recommendation subsystem is connected with a simulation driver module, the simulation driver module is connected with an XR cabin interaction module, and the simulation driver module provides a virtual driving scene for operators to interact by XR head-mounted equipment, and the method comprises the following steps:

acquiring interaction data, wherein the interaction data are acquired by an operator in a cockpit in an interaction process by an interaction data acquisition component of the simulated driver module; the XR cabin interaction module makes an interaction decision based on a preset interaction control scheme according to the interaction data and feeds back the interaction decision to the simulation driver module so as to realize continuous interaction;

calculating grade scores of human-computer interaction behaviors in a plurality of score dimensions based on the multi-modal human-computer interaction data acquired by the interaction data acquisition component, and calculating a weighted average value to obtain strategy scores of the interaction control scheme; the scoring dimension includes a cognitive load score, a comfort score, and a behavioral efficiency score; wherein the multi-modal human factor comprises: electroencephalogram data, eye movement data, physiological data, near infrared data, and operator behavioral data.

2. The intelligent cockpit human-computer interaction assessment method according to claim 1, wherein the electroencephalogram data comprises electroencephalogram EEG signal time domain, frequency domain and nonlinear indexes; the eye movement data comprise pupil diameters, fixation point coordinates and corresponding fixation time lengths; the physiological data includes skin temperature data, skin electrical data, and respiratory data; the near infrared data comprises blood oxygen data, total hemoglobin concentration values and heart rate variability data; the respiratory data includes respiratory rate, tidal volume, and vital capacity; the operator behavior data includes an execution completion rate of driving behavior and a stimulus response time period.

3. The intelligent cabin human-computer interaction evaluation method according to claim 2, wherein the cognitive load score is obtained by evaluating first-class interaction data, wherein the first-class interaction data comprises the electroencephalogram EEG signal time domain, the electroencephalogram EEG signal frequency domain, nonlinear indexes, the pupil diameter, the blood oxygen data and the total hemoglobin concentration value;

the comfort score is obtained by evaluating second-class interaction data, wherein the second-class interaction data comprises heart rate variability data, skin temperature data and skin electricity data;

the behavior high-efficiency score is obtained by evaluating third-class interaction data, wherein the third-class interaction data comprises the execution completion rate and the stimulus response duration.

4. A method of intelligent cockpit human-computer interaction assessment according to claim 3, wherein the method comprises:

mapping the first type of interaction data to the cognitive load scores by using a pre-trained first neural network, wherein the cognitive load scores comprise a first set number of grade scores;

mapping the second type of interaction data to the comfort scores using a pre-trained second neural network, the comfort scores comprising a second set number of rank scores;

mapping the third type of interaction data to the behavioral efficiency scores using a pre-trained third neural network, the behavioral efficiency scores comprising a third set number of rank scores.

5. The intelligent cabin human-computer interaction assessment method according to claim 4, wherein the first neural network, the second neural network and the third neural network are decision trees or BP neural networks.

6. The intelligent cockpit human-computer interaction assessment method of claim 4 wherein the first neural network, the second neural network and the third neural network each comprise a convolutional neural network.

7. The method for evaluating human-computer interaction of intelligent cabin according to claim 1, wherein before calculating a weighted average to obtain a policy score of the interaction control scheme, further comprises:

and normalizing the grade scores of the score dimensions.

8. The intelligent cockpit human-computer interaction assessment method of claim 1, further comprising:

and calculating strategy scores corresponding to the interaction control schemes, and selecting the interaction control scheme with the highest strategy score as the optimal interaction control scheme.

9. An intelligent cabin man-machine interaction evaluation system, which is characterized by comprising:

the simulation driver module comprises a simulation driver cabin and an interaction data acquisition component, wherein the driver cabin is used for providing a hardware environment for simulation driving, and the interaction data acquisition component comprises XR head-mounted equipment for brain control and eye control interaction, a skin temperature skin electric sensor and a near infrared sensor which are arranged on a steering wheel, a respiration sensor arranged on a safety belt and an interaction behavior sensor arranged in the driver cabin; the XR headset equipment acquires electroencephalogram data and eye movement data, the skin temperature and skin electricity sensors acquire skin temperature data and skin electricity data, the near infrared sensors acquire near infrared data, the respiration sensors acquire respiration data, and the interaction behavior sensors are used for acquiring operator behavior data;

the XR cabin interaction module is used for controlling the XR head-mounted equipment to present a virtual driving scene, making an interaction decision on the electroencephalogram data, the eye movement data, the skin temperature data, the near infrared data, the breathing data and the operator behavior data based on a preset interaction control scheme, and feeding back the interaction decision to the simulation value module for execution;

the cabin man-machine interaction design evaluation recommendation subsystem is used for executing the intelligent cabin man-machine interaction evaluation method according to any one of claims 1 to 8 to obtain the strategy score of the interaction control scheme.

10. The intelligent cockpit human-computer interaction assessment system of claim 9, wherein the system is further connected to a cloud server in a wired or wireless manner and uploads the policy scores of the interaction control scheme.