CN111580641A - VR-based military decision efficiency simulation monitoring and early warning system - Google Patents

VR-based military decision efficiency simulation monitoring and early warning system Download PDF

Info

Publication number
CN111580641A
CN111580641A CN202010179892.1A CN202010179892A CN111580641A CN 111580641 A CN111580641 A CN 111580641A CN 202010179892 A CN202010179892 A CN 202010179892A CN 111580641 A CN111580641 A CN 111580641A
Authority
CN
China
Prior art keywords
decision
military
monitoring
simulated
efficiency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010179892.1A
Other languages
Chinese (zh)
Other versions
CN111580641B (en
Inventor
王晓霞
欧阳谭特
唐浩
梁明威
刘浩
刘斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Army Medical University
Original Assignee
Army Medical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Army Medical University filed Critical Army Medical University
Priority to CN202010179892.1A priority Critical patent/CN111580641B/en
Publication of CN111580641A publication Critical patent/CN111580641A/en
Application granted granted Critical
Publication of CN111580641B publication Critical patent/CN111580641B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Neurology (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Dermatology (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Neurosurgery (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

The invention provides a simulation military decision efficiency monitoring and early warning system based on VR, and relates to the technical field of VR. The invention develops a simulated convoy decision task based on a virtual reality situation, provides a strong-adaptability, reliable and effective monitoring system for military decision efficiency, dynamically monitors the numerical value of an evaluation index of military personnel when the military personnel make a decision according to a military decision efficiency evaluation index system through feedback information in an individual decision process, and provides a feedback signal to the military personnel when intervention training is needed so as to improve the military decision efficiency and further ensure the adaptability of the system.

Description

VR-based military decision efficiency simulation monitoring and early warning system
Technical Field
The invention relates to the technical field of VR (virtual reality), in particular to a simulation military decision efficiency monitoring and early warning system based on VR.
Background
The Military Decision Performance (MDP) is a core component of military operation capability, refers to a process of military personnel performing operation situation analysis, designing an operation scheme and finally selecting reasonable operation, and is a key element for military combat capability generation. The military decision efficiency monitoring and early warning system under the simulated mission condition is constructed, which is beneficial to selecting fighters, evaluating decision effect and making a training strategy, thereby providing possibility for obtaining decision advantages.
The military decision efficiency monitoring and early warning system under the condition of the existing simulated military decision task comprises: the subjective self-rating scale of adult decision-making capability is used for measuring and evaluating the decision-making capability of young and middle-aged decision-making persons.
However, the inventor of the present applicant finds that the existing military decision performance monitoring and early warning system under the condition of simulating the military decision task is suitable for the static decision problem, and cannot accurately describe the continuous decision process in which the individual action causes the environmental state change and further affects the subsequent action, but most of the military decision tasks are continuous decision tasks, so that the existing military decision performance monitoring and early warning system has poor adaptability.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides a VR-based military decision efficiency simulation monitoring and early warning system, which solves the technical problem of poor adaptability of the conventional military decision efficiency monitoring and early warning system.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
the invention provides a VR-based military simulation decision efficiency monitoring and early warning system, which comprises a simulation decision system, an evaluation system, a monitoring and early warning system and wearable equipment, wherein the wearable equipment can be communicated and interacted with the simulation decision system, the evaluation system and the monitoring and early warning system; the wearable device comprises a VR device for presenting simulated convoy decision-making tasks of virtual reality situations to military personnel receiving monitoring;
compiling a simulated convoy decision task based on a virtual reality situation through the simulated decision system, presenting the simulated convoy decision task to military personnel receiving monitoring through VR equipment, and acquiring feedback information made by the military personnel based on the simulated convoy decision task through wearable equipment and transmitting the feedback information to the evaluation system;
the evaluation system obtains the numerical value of the evaluation index of the military personnel decision-making efficiency according to the military decision-making efficiency evaluation index system and the feedback information;
and the monitoring and early warning system acquires decision efficiency in the decision process of military personnel according to the numerical value of the evaluation index, and provides an early warning signal when the decision does not reach the optimal efficiency or is lower than the acceptable minimum efficiency.
Preferably, the evaluation index includes: flexibility of decision, accuracy of decision, and cognitive status of decision.
Preferably, the method for evaluating flexibility of decision includes:
the flexibility of the decision is evaluated with weights based on model reinforcement learning and model-free reinforcement learning.
Preferably, the method for evaluating the accuracy of the decision comprises:
and evaluating the accuracy of the decision in real time by using the difference between the actual decision and the optimal decision of military personnel.
Preferably, the method for evaluating the cognitive state of the decision comprises:
cognitive status of decisions is assessed in real time using the variability of military personnel in reaction times when performing simulated convoy decision tasks.
Preferably, the feedback information includes a behavior index and a physiological index.
Preferably, the monitoring and early warning system is a military decision efficiency monitoring and early warning system based on a biofeedback principle, and comprises:
a decision performance evaluation system based on the behavior index;
a decision performance evaluation system based on the physiological index;
and the monitoring and early warning system is used for monitoring and early warning the military decision efficiency based on the cognitive state calculation of the behavior index and the physiological index of the efficiency evaluation system.
Preferably, the physiological index includes: eye movement index and EEG index.
Preferably, the wearable device further comprises:
a head band for collecting EEG index and an eye movement instrument for collecting eye movement index.
(III) advantageous effects
The invention provides a VR-based military decision efficiency simulation monitoring and early warning system. Compared with the prior art, the method has the following beneficial effects:
the invention relates to a VR-based military simulation decision efficiency monitoring and early warning system, which comprises a simulation decision system, an evaluation system, a monitoring and early warning system and wearable equipment, wherein the wearable equipment can be communicated and interacted with the simulation decision system, the evaluation system and the monitoring and early warning system; the wearable device comprises a VR device for presenting simulated convoy decision-making tasks of virtual reality situations to military personnel receiving monitoring; compiling a simulated convoy decision task based on a virtual reality situation through a simulated decision system, presenting the simulated convoy decision task to military personnel receiving monitoring through VR equipment, and acquiring feedback information made by the military personnel based on the simulated convoy decision task through wearable equipment and transmitting the feedback information to an evaluation system; the evaluation system obtains the numerical value of the evaluation index of the military personnel decision efficiency according to the military decision efficiency evaluation index system and the feedback information; the monitoring and early warning system obtains decision-making efficiency in the decision-making process of military personnel according to the numerical value of the evaluation index, and provides early warning signals when the decision does not reach the optimal efficiency or is lower than the acceptable minimum efficiency. The invention develops a simulated convoy decision task based on a virtual reality situation, provides a strong-adaptability, reliable and effective monitoring system for military decision efficiency, dynamically monitors the numerical value of an evaluation index of military personnel when the military personnel make a decision according to a military decision efficiency evaluation index system through feedback information in an individual decision process, and provides a feedback signal to the military personnel when intervention training is needed so as to improve the military decision efficiency and further ensure the adaptability of the system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a block diagram of an embodiment of the present invention;
FIG. 2 is a flow chart of a simulated convoy decision task in an embodiment of the present invention;
FIG. 3 is a diagram illustrating a partially observable Markov decision process in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of monitoring thresholds for determining cognitive states in an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating whether intervention training is required in the cognitive status monitoring system based on the evaluation index according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides a VR-based military decision efficiency simulation monitoring and early warning system, solves the problem of poor adaptability of the existing military decision efficiency monitoring and early warning system, and provides a reliable and effective monitoring and early warning system with strong adaptability for military decision efficiency
In order to solve the technical problems, the general idea of the embodiment of the application is as follows:
the embodiment of the invention develops a simulation convoy decision task based on a virtual reality situation, provides a strong-adaptability, reliable and effective monitoring system for military decision efficiency, dynamically monitors the numerical value of an evaluation index of military personnel when the military personnel make a decision according to an evaluation index system of the military decision efficiency through feedback information in an individual decision process, and provides a feedback signal to the military personnel when intervention training is needed, so that the military decision efficiency is improved, and the adaptability of the system is further ensured.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
The embodiment of the invention provides a VR-based military simulation decision efficiency monitoring and early warning system, which comprises a simulation decision system, an evaluation system, a monitoring and early warning system and wearable equipment, wherein the wearable equipment can be communicated and interacted with the simulation decision system, the evaluation system and the monitoring and early warning system; the wearable device comprises a VR device for presenting simulated convoy decision-making tasks of virtual reality situations to military personnel receiving monitoring;
compiling a simulated convoy decision task based on a virtual reality situation through a simulated decision system, presenting the simulated convoy decision task to military personnel receiving monitoring through VR equipment, and acquiring feedback information made by the military personnel based on the simulated convoy decision task through wearable equipment and transmitting the feedback information to an evaluation system;
the evaluation system obtains the numerical value of the evaluation index of the military personnel decision efficiency according to the military decision efficiency evaluation index system and the feedback information;
the monitoring and early warning system obtains decision-making efficiency in the decision-making process of military personnel according to the numerical value of the evaluation index, and provides early warning signals when the decision does not reach the optimal efficiency or is lower than the acceptable minimum efficiency.
The embodiment of the invention develops a simulation convoy decision task based on a virtual reality situation, provides a strong-adaptability, reliable and effective monitoring system for military decision efficiency, dynamically monitors the numerical value of an evaluation index of military personnel when the military personnel make a decision according to an evaluation index system of the military decision efficiency through feedback information in an individual decision process, and provides a feedback signal to the military personnel when intervention training is needed, so that the military decision efficiency is improved, and the adaptability of the system is further ensured.
The following describes embodiments of the present invention in more detail:
in the embodiment of the invention, the wearable device presents a virtual military decision task scene (based on a net c # development language and platform and a UNITY3D game engine) for a virtual reality wearable helmet (VR device) with a model number of HTC vivpro eye, and two wireless controllers are adopted for recording and transmitting behavior information data in real time. And presenting the preset scene in a computer running a Windows 10 operating system. The simulation decision system, the evaluation system and the monitoring and early warning system run on a computer of a Windows 10 operating system, and can realize information interaction with wearable equipment.
The simulation navigational aid decision task is divided into two stages. Stage one: is tested to select one of a pair of stimuli, this stimulus (S)A1) 70% of the cases will lead to the second pair of stimuli (S)B1And SB2). And a second stage: in 30% of cases, the stimulation will be directed to the third pair (S)C1And SC2) And stage one selects another stimulus (S)A2) 30% of the cases lead to a second pair of stimuli (S)B1And SB2) 70% of the cases will lead to the third pair of stimuli (S)C1And SC2). The probability of a change from phase one stimulation to phase two stimulation by individual key selection is called transition probability. The case where the transition probability is large is called a common transition, and the case where the transition probability is small is called a rare transition. The simulated convoy decision task consisted of 201 trials, 3 segments, each segment lasting 502.5s, and 20s of rest after 67 th and 134 th trials.
In the specific implementation process, military personnel receiving monitoring receive the following guidance words through a wearable device (a virtual reality wearable helmet with model number HTC vivipeye): in the following combat decision task, you (supposing that a land convoy commander) can select any one of two roads by pressing keys, the road can bring you into a branch intersection, and at the moment, you need to press keys again to select any one of the two roads, and as a result, the road can safely pass through and cause a certain number of enemy casualties through fire attack, and the road can also encounter enemy vogue so as to cause a certain number of enemy casualties. Please note that there is a certain relationship between the path you choose and the number of casualties, you need to avoid the casualties of our army as much as possible by decision, and cause more casualties of the enemy army. The casualties of both parties can be recorded below the screen. Please enter the drill portion first and then enter the official task. The flow of the simulated convoy decision task is shown in fig. 2.
The method comprises the steps that military personnel receiving monitoring make feedback according to guide words of tasks, the wearable equipment transmits feedback information into an evaluation system, in the specific implementation process, the wearable equipment records behavior information data in the feedback information in real time through two wireless controllers and transmits the behavior information data into the evaluation system, the wearable equipment further comprises an EEG head band, a built-in eye tracker and other equipment used for acquiring physiological indexes, the physiological indexes in the feedback information are acquired through the two equipment, and the physiological indexes are transmitted into the evaluation system in real time.
The evaluation system comprises a military decision efficiency evaluation index system, and the monitoring and early warning system is a military decision efficiency monitoring and early warning system. In a specific implementation Process, the military Decision performance evaluation index system is a POMDP (partially observable Markov Decision Process) based military Decision performance evaluation index system, and the monitoring and early warning system is a military Decision performance monitoring and early warning system based on a biological feedback principle.
The POMDP-based military decision performance evaluation index system is described in detail below:
a series of continuous decisions of military personnel are regarded as a Markov chain with limited state, probability distribution on a system state space is iteratively modified based on observation and action of an environment state and an environment state transition matrix, and behavior data of a two-stage Markov decision task are modeled by using a reinforcement learning algorithm. The decision flow is as follows: (1) firstly, after a decision task (in the embodiment of the invention, the decision task is a simulated convoy decision task) is started and the perceived value of the military environment state is preliminarily estimated, a decision maker (in the embodiment of the invention, the decision maker is a military person under monitoring) modifies a belief vector (defined as beliefs of the military environment state), (2) further searches information or takes action to complete an iterative process of the task, and (3) when the belief that the process reaches a certain degree considers that the task is completed, the decision maker terminates the action by declaring the task to be successful or failed and completes the decision process. The process is shown in figure 3.
The markov decision process is a decision process consisting of a five-tuple consisting of < S, a, P, R, γ >.
S is a limited state set; a is a limited set of actions; p is a state transition matrix and is,
Figure BDA0002412137190000092
r is a reward function, gamma is a discount factor, gamma ∈ [0,1 ]]. Where strategy pi refers to the action probability distribution at a given state, namely: pi (a | s) ═ P (a)t＝a|St＝a)。
Given a policy π, state s, take the action cost function of action a:
Qπ(s)＝Eπ(Gt|St＝s，At＝a)
taking the optimal action cost function of action a at state s:
Figure BDA0002412137190000091
for the markov decision process, there is always a certain optimal strategy, which can be found by maximizing the optimal action cost function.
In the embodiment of the invention, the military decision performance evaluation index system comprises three types of military decision performance evaluation indexes: (1) the flexibility of the decision is evaluated with weights based on model-based reinforcement learning and based on model-free reinforcement learning, wherein the greater the weight of the model-based decision, the more flexible the decision is the ability of the individual to form a cognitive map of the environment and take action accordingly. (2) And evaluating the accuracy of the decision in real time by using the difference between the actual decision and the optimal decision of military personnel. (3) Cognitive status of decisions is assessed in real time using the variability of military personnel in reaction times when performing simulated convoy decision tasks.
Wherein:
(1) the flexibility of the decision is evaluated with weights based on model reinforcement learning and based on model-free reinforcement learning. The method specifically comprises the following steps:
① model independent action value for trial t, using S1,tTo represent the first stage state (i.e., S)A) By S2,tTo identify the second stage state, the first stage and second stage actions are respectively a1,tAnd a2,tTo indicate, second stage and second stage prizes, respectively r1,tAnd r2,tTo indicate. At each stage, the individual associates an action with an expected value through a state-action cost function Q (s, a). Wherein the relationship between model independent action value and reward estimation error is as follows:
QMF(S1，t，a1，t)＝QMF（S1，t，a1，t)+α1λ2，t
wherein α represents learning rate (α)1、α2The learning rate of the first and second phases, new information representing the action result is used for the learning degree. The eligibility parameter λ represents the relative weight (0 ≦ λ ≦ 1) for updating the model-independent action value of the first phase with the state estimate and the final reward result of the second phase. If lambda is 1, the result represents that only the last reward result is used for updating the independent action value of the first-stage model; if λ is 0, the state estimate representing the second stage alone is used to update the first stage model independent action value. The decision task has only two stages, and the influence of the state of the next trial on the independent action value of the first-stage model is not considered.
i,tRepresenting the reward estimation error, i.e. the difference between the actual and expected values of the state-action value:
i，t＝ri，t+QMF(Si+1,t,ai+1，t)-QMF(Si，t，ai，t)
for the State-action value of the first phase, ri，t＝0，i,tDetermined by the second stage state-action value QMF(S2,t，a2,t). For the State-action value of the second phase, QMF(S3,t，a3,t) 0. Since the trial has no further state-action value at this point, there is only an immediate reward r2，t。
② the value of the model-based action. By learning the state transition function (the probability distribution that maps a state-action pair to a subsequent state) and the immediate reward value for each state, then by calculating the cumulative state-action value through iterative expectation of these values. That is, the first stage first decides which action corresponds to which state of the second stage (telling the decision maker to follow a probabilistic structure here), and then learns the instant prize value for each action of the second stage (the instant prize for the first stage is always zero). Assuming that the recalculation is based on the estimates of transition probability and immediate reward in each trial, the model-based action value is defined using the bellman equation as:
Figure BDA0002412137190000111
③ action rules in the first and second phases, the action probability can be considered as a net state-action value QnetInverse temperature parameter β1、β2Adherence to the parameter p and the softmax function indicating the function rep (a), namely:
Figure BDA0002412137190000112
β1、β2representing the certainty of the first and second phase action selection. rep (a) and p represent the repeatability of the action selection, i.e. the same tendency of the key selection as the previous trial, regardless of the actual value of the option. Wherein rep (a) 1 and p>0 represents the same trial as the previous trial, rep (a) ═ e1 and p<0 represents an action selection different from the previous trial.
④ policy weight, value of action Q based on modelMBModel independent action value QMFAnd a policy weight ω (between 0 and 1, ω ═ 0 represents model-independent reinforcement learning only, ω ═ 1 represents model-based reinforcement learning only) for making decisions based on model or model-independent action value, the state-action value of the first stage can be defined as:
Qnet(SA，aj)＝ωQMB(SA，aj)+(1-ω)QTD(SA，aj)
ω can be calculated according to the above formula.
(2) The method for evaluating the accuracy of the decision in real time by using the difference between the actual decision and the optimal decision of military personnel specifically comprises the following steps:
the decision accuracy depends on how far the actual decision of the decision maker deviates from the optimal decision at any point in time. And R is used for representing the difference between the actual decision result and the result generated by executing the optimal decision in each test of n times, wherein the optimal decision represents no harm to the friend army and causes great harm to the enemy army.
Specifically, given K ≧ 2 paths (i ═ 1, … K) and an unknown result (r) associated with each pathi,1，ri,2…，ri,n) For each trial (t 1, … n), the individual selects a path i and results in a correlation result (r)It,t) Will be
Figure BDA0002412137190000122
Defined as the best possible result for path i in trial t. The difference between the actual and optimal decisions of the military personnel after the nth trial can be defined as:
Figure BDA0002412137190000121
in each trial, the subject gets immediate feedback information including reward (enemy casualties), punishment (friend casualties) and final combat (relative casualties). In the Nth test, the number of enemy casualties ranges from 50 to 100, and the number of friend military casualties ranges from 0 to-1250. The cumulative relative casualties is defined as the cumulative relative casualties after the previous march, the casualties of the enemy during the previous march minus the casualties of the friend march, and the initial value is set to 2000 points. The primary outcome variable was the cumulative relative casualties at the end of 200 trials (marchs), the beginning of the route selected by the subject, and not known when the end will eventually occur.
(3) The method comprises the following steps of evaluating the cognitive state of a decision in real time by using the variability of reaction time of military personnel when the military personnel perform a simulated convoy decision task, wherein the method specifically comprises the following steps:
the cognitive state of a decision is the intermediate process from situational awareness to a correct decision by a decision maker for a military environment. Situational awareness, meaning that a decision maker does not fully understand the environmental situation and task requirements, and therefore actively seeks and responds to information in the environment, is called "exploration" (exploration); correct decision, which means that the decision maker has complete knowledge of the environment situation and task requirements and takes action accordingly, is called "development". The decision Problem is modeled as a multiple-arm gambler Problem (MABP), with the ultimate goal of maximizing the sum of the returns at all times. When different paths (namely the rocker arm in the MABP) are selected, the existing path with the maximum return value is utilized, and other paths with higher return values are searched as far as possible, so that the balance between searching and utilization is achieved as far as possible. Exploration and development of cognitive states may be operationally defined as variability in reaction times, where high variability in reaction times represents an exploration state, and low variability in reaction times represents a development state.
The change in reaction time is repeatedly estimated from the moving window of data. Specifically let xiWhen the reaction is performed at time i, i is 2,3, …, 200. Then for a window of data of size w +1, starting from time i-w +2, the variability in the reaction is calculated in turn as follows:
Figure BDA0002412137190000131
wherein
Figure BDA0002412137190000132
Variability in response to reaction
Figure BDA0002412137190000133
Monitoring is carried out when
Figure BDA0002412137190000134
The larger the size, the more likely the cognitive state of the decision maker will be to explore when prompted
Figure BDA0002412137190000141
The smaller, the more development-prone the cognitive state that suggests the decision maker. When in use
Figure BDA0002412137190000142
And when the cognitive state is less than a certain threshold h, prompting the decision maker to switch from exploration to development, and sending a visual feedback signal to the decision maker.
The threshold h is actually an interval h1,h2]Wherein h is2>h1. For a person in an exploration state, when Si2<h1 (at time i) switching to development state; for the person in development, when Si2>A threshold was determined by adding and subtracting twice the standard deviation of all responses from the subject's baseline response time (i.e., when 0 or 50. sup. friend lesions were produced), where ① exploratory status, defined as two Standard Deviations (SD) above baseline response time in response, and ② developmental status, defined as two Standard Deviations (SD) below baseline response time, as shown in figure 3.
The military decision efficiency monitoring and early warning system based on the biofeedback principle is explained in detail as follows:
the military decision efficiency monitoring and early warning system based on the biofeedback principle comprises: (1) a decision performance evaluation system based on the behavior index, (2) a decision performance evaluation system based on the physiological index, and (3) a monitoring and early warning system of military decision performance calculated based on the behavior index of the performance evaluation system and the cognitive state of the physiological index.
Wherein:
(1) the decision efficiency evaluation system based on the behavior index specifically comprises:
and (4) prompting the dynamic change process of the efficiency from the non-optimal decision to the optimal decision according to the consistency of the cognitive state and the decision result. The initial state of the decision is started from the yellow unit, which represents the exploration mode, and the decision accuracy is low, and the decision efficiency is not optimal. Ideally, at a certain moment in the task process, the trainee transitions to the green unit, at this moment, the cognitive state is changed to the development mode, the decision accuracy is highest, and the decision efficiency is prompted to reach the optimum.
Under normal conditions, the cognitive state and the decision result are consistent, when inconsistency occurs, the inconsistency is possibly caused by physiological states such as drowsiness, distraction, high cognitive load, low cognitive participation and the like, and real-time monitoring and early warning can be carried out by combining with neurophysiological indexes. When the trainee's cognitive status is not consistent with the decision result, it is suggested that intervention training should be performed (orange and red units), as shown in fig. 5.
(2) The decision performance evaluation system based on the physiological indexes specifically comprises:
and simulating a convoy decision task through VR, collecting physiological index data such as electroencephalogram and eye movement, acquiring a cognitive state in a user decision process, and finding out a cognitive state change in an optimal decision.
And (4) electroencephalogram indexes. EEG headbands were used to record electroencephalograms synchronously during the decision-making task. The electrical signals were collected by 6 dry electrodes with the headband on the forehead, AF3, AF4, AF7, AF8, FP1 and FP2, reporting 512 raw signals per second and several derived measurement signals by bluetooth. In the case where EEG data is received and the signal quality assessment of the sensor is below 50, the military personnel being monitored are allowed to perform decision-making tasks (on a scale of 0-100, the lower the better). EEG responses with a signal standard deviation over time greater than 5 times the median standard deviation (0.3% of the response) were considered noisy and were not further analyzed. The 4 cycle long Hanning window for 5 frequency bands of the EEG signal (table 1) was time-frequency analyzed using the fieldtip toolbox multi-cone method. The electroencephalogram signals recorded from 500 milliseconds before the reward result to 1500 milliseconds after the reward result serve as electroencephalogram responses to the reward result.
TABLE 1 brain wave frequency bands and associated brain states
Figure BDA0002412137190000151
Figure BDA0002412137190000161
The index of eye movement. The data of the tested eye movement is collected by adopting a built-in eye movement instrument, the data mainly comprises information such as pupil diameter, saccade frequency and the like, and the sampling frequency of the equipment is 30 Hz. Analyzing the eye movement indexes one by one: the fixed times and average fixed duration represent the degree of importance of the decision maker to the information. The more flexible we predict decisions, the more likely the decision maker is to focus on task structure visual information and use it to update the cognitive model of the environment. Extent of pupil dilation refers to the difference between task processing (i.e., decision task) and rest time (i.e., interval time/baseline), representing arousal and cognitive load.
(3) The monitoring and early warning system for military decision efficiency based on cognitive state calculation of behavior indexes and physiological indexes of the efficiency evaluation system specifically comprises:
(1) and a monitoring module. The behavior index and the physiological index are converted into psychological signals in real time through the evaluation system, and key features of the psychological signals are extracted, so that military decision efficiency is monitored in real time. (2) And an early warning module. Analyzing the difference between the baseline condition and the key characteristics of the physiological data in the optimal decision making process, and providing an early warning signal when the decision does not reach the optimal efficiency or is lower than the acceptable minimum efficiency, so that military personnel can update the environmental information in time and adjust the decision making action; and positive feedback is provided when the optimal decision is reached, so that military personnel can adjust the input degree of cognitive resources and maintain the optimal decision.
In summary, compared with the prior art, the method has the following beneficial effects:
1. the embodiment of the invention develops a simulated convoy decision task based on a virtual reality situation, and provides a strong-adaptability, reliable and effective monitoring system for military decision efficiency.
2. The embodiment of the invention dynamically monitors the numerical value of the evaluation index of military personnel in decision making according to the behavior index and the physiological index in the individual decision making process and the military decision efficiency evaluation index system, and provides a feedback signal to the military personnel in real time when intervention training is needed so as to improve the military decision efficiency and further ensure the adaptability of the system.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (9)

1. A simulation military decision-making efficiency monitoring and early warning system based on VR is characterized in that the system comprises a simulation decision-making system, an evaluation system, a monitoring and early warning system and a wearable device, wherein the wearable device can be communicated and interacted with the simulation decision-making system, the evaluation system and the monitoring and early warning system; the wearable device comprises a VR device for presenting simulated convoy decision-making tasks of virtual reality situations to military personnel receiving monitoring;
compiling a simulated convoy decision task based on a virtual reality situation through the simulated decision system, presenting the simulated convoy decision task to military personnel receiving monitoring through VR equipment, and acquiring feedback information made by the military personnel based on the simulated convoy decision task through wearable equipment and transmitting the feedback information to the evaluation system;
the evaluation system obtains the numerical value of the evaluation index of the military personnel decision-making efficiency according to the military decision-making efficiency evaluation index system and the feedback information;
and the monitoring and early warning system acquires decision efficiency in the decision process of military personnel according to the numerical value of the evaluation index, and provides an early warning signal when the decision does not reach the optimal efficiency or is lower than the acceptable minimum efficiency.
2. The VR-based simulated military decision performance monitoring and forewarning system of claim 1, wherein the evaluation index comprises: flexibility of decision, accuracy of decision, and cognitive status of decision.
3. The VR-based simulated military decision performance monitoring and forewarning system of claim 2, wherein the method of assessing flexibility of decisions includes:
the flexibility of the decision is evaluated with weights based on model reinforcement learning and model-free reinforcement learning.
4. The VR-based simulated military decision performance monitoring and forewarning system of claim 2, wherein the method of assessing the accuracy of the decision comprises:
and evaluating the accuracy of the decision in real time by using the difference between the actual decision and the optimal decision of military personnel.
5. The VR-based simulated military decision performance monitoring and forewarning system of claim 2, wherein the decision cognitive state assessment method comprises:
cognitive status of decisions is assessed in real time using the variability of military personnel in reaction times when performing simulated convoy decision tasks.
6. The VR-based simulated military decision performance monitoring and forewarning system of claim 1, wherein the feedback information includes behavioral and physiological indicators.
7. The VR-based simulated military decision-making performance monitoring and forewarning system of claim 6, wherein the monitoring and forewarning system is a military decision-making performance monitoring and forewarning system based on biofeedback principles, comprising:
a decision performance evaluation system based on the behavior index;
a decision performance evaluation system based on the physiological index;
and the monitoring and early warning system is used for monitoring and early warning the military decision efficiency based on the cognitive state calculation of the behavior index and the physiological index of the efficiency evaluation system.
8. The VR-based simulated military decision performance monitoring and forewarning system of claim 1, wherein the physiological indicators comprise: eye movement index and EEG index.
9. The VR-based simulated military decision performance monitoring and forewarning system of claim 8, wherein the wearable device further comprises:
a head band for collecting EEG index and an eye movement instrument for collecting eye movement index.
CN202010179892.1A 2020-03-16 2020-03-16 VR-based military decision efficiency simulation monitoring and early warning system Active CN111580641B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010179892.1A CN111580641B (en) 2020-03-16 2020-03-16 VR-based military decision efficiency simulation monitoring and early warning system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010179892.1A CN111580641B (en) 2020-03-16 2020-03-16 VR-based military decision efficiency simulation monitoring and early warning system

Publications (2)

Publication Number Publication Date
CN111580641A true CN111580641A (en) 2020-08-25
CN111580641B CN111580641B (en) 2022-04-22

Family

ID=72114953

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010179892.1A Active CN111580641B (en) 2020-03-16 2020-03-16 VR-based military decision efficiency simulation monitoring and early warning system

Country Status (1)

Country Link
CN (1) CN111580641B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113435598A (en) * 2021-07-08 2021-09-24 中国人民解放军国防科技大学 Knowledge-driven intelligent strategy deduction decision method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107423527A (en) * 2017-08-23 2017-12-01 太仓苏易信息科技有限公司 A kind of Virtual Modeling System for military exercises
CN207236795U (en) * 2017-02-06 2018-04-17 浙江凡聚科技有限公司 Biological feedback system applied to military training
CN109582140A (en) * 2018-11-23 2019-04-05 哈尔滨工业大学 A kind of architecture indoor pathfinding element vision significance assessment system and method based on virtual reality and eye movement tracking

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN207236795U (en) * 2017-02-06 2018-04-17 浙江凡聚科技有限公司 Biological feedback system applied to military training
CN107423527A (en) * 2017-08-23 2017-12-01 太仓苏易信息科技有限公司 A kind of Virtual Modeling System for military exercises
CN109582140A (en) * 2018-11-23 2019-04-05 哈尔滨工业大学 A kind of architecture indoor pathfinding element vision significance assessment system and method based on virtual reality and eye movement tracking

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113435598A (en) * 2021-07-08 2021-09-24 中国人民解放军国防科技大学 Knowledge-driven intelligent strategy deduction decision method
CN113435598B (en) * 2021-07-08 2022-06-21 中国人民解放军国防科技大学 Knowledge-driven intelligent strategy deduction decision method

Also Published As

Publication number Publication date
CN111580641B (en) 2022-04-22

Similar Documents

Publication Publication Date Title
Dey et al. Exploration of an EEG-based cognitively adaptive training system in virtual reality
CN107224291B (en) Dispatcher capability test system
CN106956271B (en) Predict the method and robot of affective state
KR102166011B1 (en) System and method for determining cognitive impairment using touch input
CN109620185A (en) Self-closing disease assistant diagnosis system, equipment and medium based on multi-modal information
CN106292705A (en) Many rotor wing unmanned aerial vehicles idea remote control system based on Bluetooth brain wave earphone and operational approach
US10772527B2 (en) Brain matching
CN111553618B (en) Operation and control work efficiency analysis method, device and system
CN111553617B (en) Control work efficiency analysis method, device and system based on cognitive power in virtual scene
CN108922629A (en) The screening and its application of brain function corelation behaviour normal form index
WO2017221082A1 (en) Method and system for detection and analysis of cognitive flow
CN113974589B (en) Multi-modal behavior paradigm evaluation optimization system and cognitive ability evaluation method
CN113521723A (en) Cognitive function evaluation system and method based on virtual reality
CA3170152A1 (en) Evaluation of a person or system through measurement of physiological data
CN111580641B (en) VR-based military decision efficiency simulation monitoring and early warning system
WO2022057840A1 (en) Brain cognitive function detection system
CN117547270A (en) Pilot cognitive load feedback system with multi-source data fusion
Chen et al. A situation awareness assessment method based on fuzzy cognitive maps
US11995527B2 (en) Apparatus, method and computer program for developing a behaviour tool to estimate a behaviour of a person
JP6910919B2 (en) How to evaluate the system and actions to be taken to communicate
CN115462755A (en) Pilot cognitive ability assessment method, system and storage medium
CN112257190B (en) Method for monitoring abnormal people in ring flight simulation environment
Matteoni et al. Continual Learning for Human State Monitoring.
CN110693509B (en) Case correlation determination method and device, computer equipment and storage medium
WO2022231589A1 (en) Predicting mental state characteristics of users of wearable devices

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant