KR20190138449A - Apparatus and method for quantifying pilot awakening - Google Patents

Abstract

The present invention provides a method and apparatus for quantifying a pilot awakening status. An apparatus for quantifying pilot awakening according to an embodiment receives the biosignal of a pilot in flight, classifies a pilot awakening status into any one of a plurality of status categories based on the biosignal of the pilot, and analyzes the biosignal of the pilot. So, it is possible to provide a method for quantifying a pilot awakening status according to the classified status category.

Description

Apparatus and method for quantifying a pilot's state of awakening {APPARATUS AND METHOD FOR QUANTIFYING PILOT AWAKENING}

The present disclosure provides an apparatus and method for quantifying awake state of a pilot.

Recently, the demand of pilots is increasing due to the development of manned and unmanned aerial vehicle technology and the continuous increase of overseas travelers using airplanes, but the supply is insufficient. Flight missions require a high level of attention, but with increased mission duration, pilots are exposed to long periods of noise and vibration, causing increased fatigue and reduced attention. Awakening conditions that are inadequate for a pilot's flight are likely to lead to a plane crash due to the pilot's cognitive negligence. Therefore, the pilot's change in alertness must be objectively assessed for the pilot's safe mission and accident prevention.

 Conventionally, a technology for detecting awake state during a flight and providing feedback for continuous awake state using various biosignals of a pilot has been invented, but in the case of the technology, the degree of cognitive deterioration due to an unsuitable state for a pilot's flight is invented. The lack of feedback regarding the quantification method limits the pilot's ability to maintain awakening. In a real flight environment, noise is noisy and complex tasks are required, so pilot's awakening feedback is applied to sensory organs such as visual and auditory organs for artificial expression of alarms and arousal induced brain waves, which have been proposed in other inventions. Randomly providing without taking into account the level of state can result in distractions such as pilot mental workload and concentration distraction. Therefore, in order for the technology to continuously maintain the awakening state of the pilot through real-time analysis of the pilot's biological signal, the pilot's precisely quantified awakening state information must be reflected in the actual flight environment.

Accordingly, there is a need for a technology that quantifies the awakening state of the pilot.

To provide an apparatus and method for quantifying the awake state of the pilot. According to the present invention, it is possible to prevent the safety accident by determining the suitability of performing the flight operation mission on the basis of the awake state of the pilot detected in real time. In addition, it is possible to quantify the type and extent of the non-conformity state that is not considered in the conventional invention when detecting the pilot non-compliance state.

The technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and further technical problems can be inferred from the following embodiments.

As a technical means for achieving the above-described technical problem, a first aspect of the present disclosure, a method for quantifying the awake state of the pilot, comprising: receiving a biosignal of the pilot in flight; Classifying the pilot's awakening status into any one of a plurality of status categories by analyzing the pilot's biosignal; And quantifying the awake state of the pilot according to the classified state category by analyzing the brain waves included in the biological signal.

And receiving, as the biosignal, a signal for at least some of the pilot's brain waves, heart rate, respiration, and skin electrical activity during flight from at least one biosignal sensor attached to the pilot. Can be provided.

The method may further include classifying the pilot awakening status into any one of the plurality of status categories by inputting the biosignal into a classifier.

In addition, the classifier may provide a method, including linear discriminant analysis (LDA), Support Vector Machines (SVM) and neural network model.

In addition, the plurality of status categories can provide a method, including fit status, reduced concentration, fatigue, workload, and unconsciousness.

In addition, calculating a phase lag index (Phase Lag Index) between the pilot EEG signal pairs during the flight; Generating a functional connectivity matrix using the phase delay index; And quantifying the awakening status of the pilot according to the classified state category by analyzing the functional connectivity matrix using a graph theory.

The method may further include acquiring the biosignal of the pilot when the pilot is in a flight suitable state, and calculating a global efficiency value or a local efficiency value of the functional connectivity matrix by using a graph theory. ; Adjusting the global efficiency value or the local efficiency value based on the pilot's biosignal when in a flight suitable state; And quantifying the pilot's arousal state according to the classified state category to the adjusted global efficiency value or the adjusted local efficiency value.

The method may further include providing classification result values for each of the state categories and the quantified awakening state information according to the classified state categories.

The method may further include providing appropriate feedback to the pilot based on the pilot's quantified awake state according to the classified status category.

A second aspect of the present disclosure, the communication unit for receiving a biological signal of the pilot during the flight; And

By analyzing the pilot's biological signal, the pilot's arousal status is classified into any one of a plurality of status categories, and by analyzing the brain waves included in the biological signal, the pilot's awake status according to the classified status category is quantified. It may provide a device, including a control unit.

The communication unit may receive, as the biosignal, signals from at least one biosignal sensor attached to the pilot for at least some of the pilot's brain wave, heart rate, respiration, and skin electrical activity during flight. A device can be provided.

The controller may provide the apparatus to classify the awakening state of the pilot into any one of the plurality of state categories by inputting the biosignal into a classifier.

In addition, the classifier may provide a device that includes a linear discriminant analysis (LDA), support vector machines (SVM) and neural network model.

In addition, the plurality of state categories may provide a device, including a fit state, reduced concentration, fatigue, workload, and unconsciousness.

The control unit may calculate a phase lag index between the pilot EEG signal pairs during the flight, generate a functional connectivity matrix using the phase delay index, and graph a theory. By analyzing the functional connectivity matrix using the < RTI ID = 0.0 >), < / RTI > to quantify the pilot's arousal state according to the classified state category.

The communication unit may acquire the biosignal of the pilot when the pilot is in a flight suitable state, and the controller calculates a global efficiency value or a local efficiency value of the functional connectivity matrix using a graph theory. And adjust the global efficiency value or the local efficiency value based on the pilot's biosignal when the flight is in a suitable state, and adjust the awakening state of the pilot according to the classified state category. A device can be provided that quantifies the adjusted Local efficiency value.

The apparatus may further include a display configured to provide a classification result value for each of the plurality of state categories and the quantified awakening state information according to the classified state category.

The apparatus may further include a feedback providing unit configured to provide appropriate feedback to the pilot based on the pilot's quantified awake state according to the classified state category.

A third aspect of the present disclosure can provide a computer readable recording medium having recorded thereon a program for executing the method of the first aspect on a computer.

According to the present invention, the pilot's ability to perform a mission in flight can be monitored in real time based on the biosignal analysis of the pilot. Specifically, by performing a quantitative assessment of the pilot's mental awakening status (mental workload, attention, fatigue, etc.), it is possible to monitor the pilot's performance in real time and provide feedback to maintain a certain level of awakeness suitable for flight. By providing a safe flight mission.

1 is a flowchart of a method of quantifying awake state of a pilot according to an embodiment.
2 is a view for explaining an example of classifying the awakening state of the pilot according to an embodiment.
3 is a diagram illustrating a classification result value for each of a plurality of state categories according to an exemplary embodiment.
4 is a diagram for describing a functional brain neural network according to an embodiment.
FIG. 5 is a diagram for describing a functional connectivity matrix according to an embodiment.
6 is a diagram for describing an example of quantifying awake state of a pilot according to a classified state category, according to an exemplary embodiment.
7 is a block diagram illustrating a hardware configuration of an awakening state quantification device according to an embodiment.

The phrases “in some embodiments” or “in one embodiment” appearing in various places in the specification are not necessarily all referring to the same embodiment.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and / or software configurations that perform particular functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. In addition, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented in algorithms running on one or more processors. In addition, the present disclosure may employ the prior art for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism", "element", "means" and "configuration" may be used widely and are not limited to mechanical and physical configurations. In addition, the terms "?", "?" Module, etc. described in the specification mean a unit for processing at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

In addition, the connecting lines or connecting members between the components shown in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In an actual device, the connections between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, with reference to the drawings will be described embodiments of the present invention;

1 is a flowchart of a method of quantifying awake state of a pilot according to an embodiment.

Referring to FIG. 1, in step 110, the awakening state quantification device may receive a biosignal of a pilot during a flight.

The alert state quantification device may receive the pilot's biosignal while the pilot is piloting the plane. As the pilot controls the airplane for a long time, fatigue or declining attention may be increased, and the awakening state quantification device may analyze the awakening state of the pilot by receiving a biosignal that changes according to the awakening state of the pilot.

The awakening state quantification device may receive various types of biosignals of a pilot in flight. For example, the biosignal may include, but is not limited to, signals relating to EEG, heart rate, respiration, and skin electrical activity.

The awakening state quantification device may extract a feature that is highly related to the awakening state of the pilot in various kinds of time and frequency domains for each biosignal.

Specifically, EEG refers to the electrical signal according to the electrical activity of the synapse. The characteristics of each band according to the frequency of the EEG and the characteristics in the time domain vary according to the pilot's intention and condition. The awakening state quantifier can apply bandwidth filtering from 1Hz to 30Hz for delta, theta, alpha and beta waves of EEG. In order to extract features that are highly related to the pilot's awakening state, the awakening state quantifier can analyze the degree of change in activation energy in the frequency domain of the delta, theta, alpha, and beta waves of the brain waves.

In addition, electrocardiogram refers to recording the potential associated with the heartbeat in the form of a figure on the body surface. In order to extract a feature highly related to the pilot's alert state, the alert state quantification device analyzes the heart rate per minute in the time domain and the frequency domain. It is possible to analyze the change of size by band by dividing the frequency into 0.04Hz ~ 0.15Hz frequency and 0.15Hz ~ 0.4Hz frequency.

In addition, in relation to breathing, the awakening quantification device analyzes the respiratory rate per minute in the time domain and extracts the Dominant Respiration Frequency (DRF) in the frequency domain, in order to extract features that are highly related to the pilot's awakening status. The change in energy magnitude in the Hz region can be analyzed.

In addition, the skin electrical activity is used by the amount of sweat secreted by the control action of the sympathetic and parasympathetic nerves of the autonomic nervous system according to the pilot's arousal state. The awakening state quantification device may extract a feature that is highly related to the awakening state of the pilot based on the change in the humidity of the skin according to the amount of sweat secreted in the time domain and the change in the electrical conductivity accordingly.

Meanwhile, a biosignal sensor for sensing various biosignals of the pilot may be attached to the body of the pilot. Specifically, the pilot's brain waves may be measured at the pilot's scalp in a non-invasive manner. For example, according to the 10-20 international electrode placement method, 64 channels such as Cz, C3, C4, Oz, O3, O4 are generally used, including the parietal lobe containing the motor area, the occipital lobe including the visual area, and the concentration-related area. Signals in the area of the overall brain, including the frontal lobe, can be measured. In addition, electrocardiogram, heart rate and the like can generally be measured using a three electrode system of RA, LA, LL. Breathing, blood pressure, etc. can be measured using a wearable device (eg, a smart band). In addition, skin electrical activity can be measured by attaching sensors to the pilot's arms or fingers.

In operation 120, the awakening state quantification apparatus may classify the awakening state of the pilot into any one of a plurality of state categories by analyzing the pilot's biosignal.

The awakening state quantifier may classify the awakening state of the pilot into any of a plurality of state categories by inputting the pilot's biosignal into the classifier. In step 110, the awakening state quantification device extracts a feature that is highly related to the awakening state of the pilot in the time domain and the frequency domain for various types of biosignals, and thus the awakening state quantification device is a Feature data highly related to the arousal state can be entered into the classifier. In one embodiment, the status category may include, but is not limited to, "suitable status", "low concentration", "fatigue", "work load" and "unconscious".

In one embodiment, the classifier that classifies the awakening status of the pilot into any one of the plurality of status categories may be Linear Discriminant Analysis (LDA) or Support Vector Machines (SVM). Linear Discriminant Analysis (LDA) is easy for linear classification, and Support Vector Machines (SVM) using the Radial Basis Function (RBF) kernel are easy for nonlinear classification. The classifier may also be an artificial neural network model such as Recurrent Convolutional Neural Network (RCNN). However, the type of classifier is not limited thereto.

In an embodiment, when data having a high correlation with the arousal state extracted from the pilot's biosignal is input to the classifier, a classification result value for each of the plurality of state categories may be calculated. For example, when the status category includes "fit status", "decent concentration", "fatigue", "work load" and "unconscious", the classification result value for each of the five status categories can be calculated. In this case, the awakening state quantification apparatus may compare the classification result values for each of the five state categories, and then classify the pilot's awakening states into the category having the largest classification result value.

The awakening state quantification apparatus may apply the Softmax function to the classification result values for each of the plurality of state categories output from the classifier. By applying the Softmax Function, the sum of the classification result values for each of the plurality of state categories may be 1. That is, the classification result value can be normalized.

In operation 130, the awakening state quantification apparatus may quantify the awakening state of the pilot according to the classified state category by analyzing the brain waves included in the biological signal.

The waking state quantification device may calculate a phase lag index (PLI) between pilot EEG signal pairs during the flight. PLI is a method of measuring asymmetry of the phase difference distribution between EEG signal pairs. In one embodiment, the PLI may be calculated using an EEG signal pair obtained from an electroencephalogram (EGE) electrode.

PLI is not sensitive to volume conduction, which is a disadvantage of EEG signals, and its computation is relatively complex, making it suitable for use in pilot awakening quantification methods that require accurate analysis in real time.

PLI may be expressed as Equation 1 below.

In one embodiment, the awakening state quantification device may apply band filtering to theta, alpha, and beta waves of the pilot's brain waves during the flight. The waking state quantification device may calculate the PLI between the band-filtered EEG signal pairs.

The awakening state quantifier may generate a functional connectivity matrix using the phase delay index. In addition, the awakening state quantification apparatus may quantify the awakening state of the pilot according to the classified state category by analyzing the functional connectivity matrix using a graph theory. In one embodiment, the awakening state quantification apparatus may quantify the awakening state of the pilot by a global / local efficiency value by calculating a global efficiency value or a local efficiency value of the functional connectivity matrix using a graph theory.

Details of generating a functional connectivity matrix and a detailed description of quantifying the awakening state of the pilot using a graph theory will be described later with reference to FIG. 5.

The awakening state quantifier can provide the pilot with a global / local efficiency value that quantifies the awakening state of the pilot. In addition, the awake state quantification device may provide appropriate feedback to the pilot based on the Global / Local efficiency value.

In a real flight environment, pilots can be affected by a variety of factors and external influences. According to one embodiment of the present invention, it is possible to accurately measure the level of cognitive decline due to the awakening state unsuitable for flight. In addition, by providing feedback to the pilot based on the precisely measured pilot's alert state, the inappropriate alert state can be moderately moderated.

On the other hand, in Figure 1, but limited to the method of quantifying the awake state of the pilot's flight, the awakening state quantification device in addition to the pilot maneuver the maneuver / drone, in addition to the pilot of the driver operating a bus, train, car, etc. It is apparent to those skilled in the art that quantification is possible. That is, the awakening state quantification device according to the present invention can be used for preventing accidents of public transportation drivers such as buses and trains as well as aircraft.

2 is a view for explaining an example of classifying the awakening state of the pilot according to an embodiment.

The awakening state quantifier may classify the awakening state of the pilot into any of a plurality of state categories by inputting the pilot's biosignal into the classifier. That is, feature data highly related to the awakening state of the pilot extracted from various kinds of biosignals of the pilot may be input to the classifier.

In one embodiment, the classifier that classifies the awakening status of the pilot into any one of the plurality of status categories may be Linear Discriminant Analysis (LDA) or Support Vector Machines (SVM). Linear Discriminant Analysis (LDA) is easy for linear classification, and Support Vector Machines (SVM) using the Radial Basis Function (RBF) kernel are easy for nonlinear classification. The classifier may also be an artificial neural network model such as Recurrent Convolutional Neural Network (RCNN). However, the type of classifier is not limited thereto.

In one embodiment, the status category may include reduced concentration, fatigue, workload, and fitness status. When the feature data related to the awakening state extracted from the pilot's biosignal is input to the classifier, the classification result value for each of the four categories may be calculated through the result value output from the classifier.

For example, if characteristic data 250 that is highly related to the pilot's arousal state extracted from the pilot's various types of biosignals is displayed on the graph 200, a determination corresponding to the characteristic data 250 and the four state categories is determined. The distances 251, 252, 253, and 254 between the decision boundaries 210, 220, 230, and 240 may be calculated. In this case, the reciprocal of the distances 251, 252, 253, and 254 may be a classification result value for each of four state categories corresponding to the decision boundaries 210, 220, 230, and 240.

The awakening state quantifier compares the feature data 250 and the distances 251, 252, 253, and 254 between the crystal boundaries 210, 220, 230, and 240, and then determines the shortest distance 252 of the pilot's awakening state. It may be classified into a category corresponding to the decision boundary 220 having. That is, the awakening state of the pilot may be determined as 'fatigue'.

Meanwhile, the arousal state quantification apparatus may apply the Softmax function to the classification result values for each of the plurality of state categories output from the classifier. By applying the Softmax Function, the sum of the classification result values for each of the plurality of state categories may be 1. That is, the awakening state quantification apparatus calculates the distances 251, 252, 253, and 254 between the feature data 250 and the decision boundaries 210, 220, 230, and 240, and then calculates the calculated distances 251, 252, 253. And 254) Softmax Function can be applied. The Softmax function can be normalized to be the sum of the distances 251, 252, 253 and 254 through application.

3 is a diagram illustrating a classification result value for each of a plurality of state categories according to an exemplary embodiment.

The awakening state quantifier may classify the awakening state of the pilot into any of a plurality of state categories by inputting the pilot's biosignal into the classifier.

In one embodiment, the status category may include fit status, fatigue, workload, and reduced concentration. When the feature data related to the awakening state extracted from the pilot's biosignal is input to the classifier, the classification result value for each of the four categories may be calculated through the result value output from the classifier.

Referring to FIG. 3, it can be seen that the classification result values of the fit state, the fatigue, the workload, and the concentration deterioration category are 0.6, 0.1, 0.2, and 0.1, respectively. The sum of the classification result values for the four categories is 1, and the awakening state quantification device applies the Softmax function to the classification result values for each of the plurality of state categories output from the classifier, thereby classifying the classification result for each of the plurality of state categories. The sum of the values can be one. When the classification result value for each of the plurality of state categories is derived as shown in FIG. 3, the awakening state quantification device may classify the awakening state of the pilot as a "fit state".

4 is a diagram for describing a functional brain neural network according to an embodiment.

Referring to FIG. 4, the functional brain neural network is a neural network characterized by connectivity between regions of the brain. The functional brain neural network consists of a node (or vertex) 410 and a link (or edge) 420 connecting node 410. Node 410 represents each region of the brain, and link 420 may represent an anatomical, functional, or effective connection between nodes 410. In one embodiment, node 410 constituting the functional brain neural network represents an EEG electrode, and link 420 may represent a correlation between signals received from each EEG electrode.

FIG. 5 is a diagram for describing a functional connectivity matrix according to an embodiment.

Functional connectivity matrix 500 illustrates the functional brain neural network described above in FIG. 4 as connectivity (or adjacency) between nodes. That is, the functional connectivity matrix 500 has inter-node correlations of functional datasets (eg, functional MRI, MEG, and EEG) as matrix elements.

In one embodiment, the awakening state quantification apparatus may generate the functional connectivity matrix 500 by calculating the matrix elements of the functional connectivity matrix 500, that is, the connectivity between nodes, using the PLI. PLI can be calculated using EEG signal pairs obtained at the EEG electrodes. For example, when a signal obtained from 64 EEG electrodes is input, a functional connectivity matrix 500 having a 64x64 size may be generated. For example, in the functional connectivity matrix 500, the matrix elements indicated in red may indicate high connectivity between nodes, and the matrix elements in blue may indicate low connectivity between nodes.

The awakening state quantification apparatus may quantify the awakening state of the pilot according to the classified state category by analyzing the functional connectivity matrix 500 using the graph theory.

In detail, the awakening state quantification apparatus may numerically calculate the degree of functional integration of the functional connectivity matrix 500 using a graph theory. Functional integration is a concept that indicates how efficiently the integrated and processed information distributed between brain areas. To numerically calculate the degree of functional integration of the brain, we measure the ease of mutual information exchange between different areas of the brain and apply the concept of pathways. Here, the path refers to a link between nodes (eg, EEG electrodes), and the shorter the path, the higher the degree of functional integration between regions of the brain.

In one embodiment, as a method of calculating the degree of functional integration for the functional connectivity matrix 500, the awakening state quantification apparatus may calculate a global efficiency value or a local efficiency value of the functional connectivity matrix 500. The global efficiency value represents the overall efficiency of mutual information exchange in functional brain neural networks, and the local efficiency value represents the efficiency of information transfer within a local cluster.

The global efficiency value of the functional connectivity matrix 500 may be expressed as Equation 2 below, and the local efficiency value may be expressed as Equation 3 below.

In one embodiment, the global / local efficiency value may have a large value when the pilot is in a flight-suitable state, and the global / local efficiency value may have a small value when the pilot is in a non-compliance state such as a lack of attention, such as workload or concentration dispersion. .

On the other hand, the degree of awakening of the pilot has a different reference value for each pilot. Accordingly, global / local efficiency values need to be adjusted for each pilot. To this end, the awakening state quantification device may acquire the pilot's biosignal when the pilot is in a flight-compatible state. For example, before the pilot controls the plane, the pilot's biosignal may be acquired without the pilot's fatigue accumulating. The pilot's biosignal can be databased by being periodically updated when in flight compliance.

The awakening state quantification device may adjust the global / local efficiency value based on the pilot's biosignal when the pilot obtains the biosignal when the pilot is in a flight-suitable state. In one embodiment, the awakening state quantifier sets the largest efficiency value for each pilot to 1 and the smallest efficiency value to 0, so that the global / local efficiency value of the pilot is normalized between 0 and 1. Can be adjusted.

FIG. 6 is a diagram illustrating an example of quantifying awake state of a pilot according to a classified state category, according to an exemplary embodiment.

The awakening state quantifier may classify the awakening state of the pilot into any of a plurality of state categories by inputting the pilot's biosignal into the classifier.

In one embodiment, the status category may include fit status, fatigue, workload, and reduced concentration. When the feature data related to the awakening state extracted from the pilot's biosignal is input to the classifier, the classification result value for each of the four categories may be calculated through the result value output from the classifier.

Referring to FIG. 6, it can be seen that the classification result values of the fit state, fatigue, workload, and concentration decay categories are 0.2, 0.6, 0.2, and 0.1, respectively. The sum of the classification result values for the four categories is 1, and the awakening state quantification device applies the Softmax function to the classification result values for each of the plurality of state categories output from the classifier, thereby classifying the classification result for each of the plurality of state categories. The sum of the values can be one. When the classification result value for each of the plurality of state categories is derived as shown in FIG. 6, the awakening state quantification device may classify the awakening state of the pilot as “fatigue”.

The awakening state quantifier may generate a functional connectivity matrix by calculating matrix elements of the functional connectivity matrix using the PLI. In addition, the awakening state quantification apparatus may quantify the awakening state of the pilot according to the classified state category by analyzing the functional connectivity matrix using a graph theory.

In one embodiment, the pilot's arousal state can be quantified by a Global / Local efficiency value calculated by applying graph theory to the functional connectivity matrix. The global / local efficiency value may be large when the pilot is in flight compliance and the global / local efficiency value may be small when the pilot is inadequate, such as lack of attention, such as workload or concentration dissipation.

On the other hand, the degree of awakening of the pilot has a different reference value for each pilot. Accordingly, global / local efficiency values need to be adjusted for each pilot. To this end, the awakening state quantification device may acquire the pilot's biosignal when the pilot is in a flight-compatible state. The awakening state quantification device may adjust the global / local efficiency value based on the pilot's biosignal when the pilot obtains the biosignal when the pilot is in a flight-suitable state. In one embodiment, the awakening state quantifier sets the largest efficiency value for each pilot to 1 and the smallest efficiency value to 0, so that the global / local efficiency value of the pilot is normalized between 0 and 1. Can be adjusted.

Referring to FIG. 6, it can be seen that the global efficiency value quantifying the awake state of the pilot during the flight has a relatively high value of 0.75. That is, in this case the pilot indicates a slight accumulation of fatigue. Meanwhile, the global efficiency value shown in FIG. 6 may be a value adjusted based on a pilot's biosignal when in a flight suitable state.

The awakening state quantification device may provide the pilot with quantified awakening state information of the pilot according to the classified state category. According to an embodiment, the awakening state quantification apparatus may provide a pilot with a classification result value for each state category and an awakening state value according to the classified state category, as shown in FIG. 6. For example, the classification result value for each state category and the awakening state value according to the classified state category are pilots through a head up display (HUD), a head mount display (HMD), or an instrument panel. It may be provided to. Alternatively, the awakening state quantification device may transmit the classification result value for each state category and the awakening state value according to the classified state category to the control tower.

In addition, the awakening state quantification device may provide appropriate feedback to the pilot based on the pilot's quantified awakening state according to the classified state category. The wakeful state quantification device may provide the pilot with direct or indirect feedback based on the pilot's quantitative wakeful state results.

For example, if the status category is classified as "fatigue," but the global / local efficiency value is relatively high, that is, if the fatigue is not high, the awakening state quantifier can increase the oxygen concentration or humidity in the cockpit, taking into account current flight environment information. And provide indirect feedback, such as controlling the cockpit environment. On the other hand, if it is determined that the pilot's fatigue is high because the value of the Global / Local efficiency is close to zero, the awakening state quantification device may provide feedback such as vibration to directly induce the awake state suitable for the pilot's flight.

7 is a block diagram illustrating a hardware configuration of an awakening state quantification device according to an embodiment.

Referring to FIG. 7, the awakening state quantification apparatus 700 may include a controller 710, a communication unit 720, a memory 730, and a display 740. In the awakening state quantification apparatus 700 of FIG. 7, only components related to the embodiment are illustrated. Thus, it will be understood by those skilled in the art that other general purpose components may be further included in addition to the components shown in FIG.

The awakening state quantification apparatus 700 may be implemented with various types of devices such as a personal computer (PC), a server device, a mobile device, and an embedded device.

The controller 710 may control a series of processes for quantifying the awakening state of the pilot described above with reference to FIGS. 1 to 6. The controller 710 controls the overall functions for controlling the awakening state quantification apparatus 700. For example, the controller 710 controls the awakening state quantification apparatus 700 by executing programs stored in the memory 730 in the awakening state quantification apparatus 700. The controller 710 may be implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), etc. provided in the awakening state quantification apparatus 700, but is not limited thereto.

The communicator 720 may include a short range communicator, a mobile communicator, and a broadcast receiver. The communicator 720 may receive a pilot's biosignal during flight. In detail, the communicator 720 may receive, as a biosignal, a signal for at least some of the pilot's brain wave, heart rate, respiration, and skin electrical activity from the at least one biosignal sensor attached to the pilot. In addition, the communicator 720 may receive the pilot's biosignal when the pilot is in a flight suitable state from an external server or the like.

The memory 730 is hardware for storing various types of data processed in the awakening state quantification apparatus 700. For example, the memory 730 is a pilot's biosignal and flight suitability state received by the communication unit 720. In this case, the biosignal related data of the pilot, the data processed by the awakening state quantification apparatus 700, and the data to be processed may be stored. In addition, the memory 730 may store applications, drivers, and the like to be driven by the awakening state quantification apparatus 700. The memory 730 may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

The display 740 displays information processed by the awakening state quantification apparatus 700. For example, the display 740 may display the classification result value for each state category and the arousal state value according to the classified state category.

Meanwhile, when the display 740 and the touch pad form a layer structure and constitute a touch screen, the display 740 may be used as an input device in addition to the output device. The display 740 is a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display (3D). display, an electrophoretic display. In addition, the display 740 may include a head up display (HUD), a head mount display (HMD), or an instrument panel.

In one embodiment, the awakening state quantification apparatus 700 may further include a feedback providing unit. The feedback provider may include a display or a lamp that outputs visual information, a motor that outputs tactile information, and a speaker that outputs sound information. In addition, the feedback providing unit may include an oxygen and humidity control device.

The embodiments may be implemented in the form of an application stored in a recording medium readable by an electronic device that stores instructions and data executable by the electronic device. The instruction may be stored in the form of program code, and when executed by a processor, may generate a predetermined program module to perform a predetermined operation. In addition, the instructions may, when executed by a processor, perform certain operations of the disclosed embodiments.

The embodiments may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, other data in modulated data signals such as program modules, or other transmission mechanisms, and includes any information delivery media.

Further, in this specification, “unit” may be a hardware component such as a processor or a circuit, and / or a software component executed by a hardware component such as a processor.

The foregoing description of the specification is intended to be illustrative, and it is understood that those skilled in the art can easily modify the present invention into other specific forms without changing the technical spirit or essential features of the present invention. Could be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present embodiment is indicated by the following claims rather than the above description, and should be construed as including all changes or modifications derived from the meaning and scope of the claims and their equivalents.

Claims (19)

In the method of quantifying the awake state of the pilot,
Receiving a biosignal of the pilot during the flight;
Classifying the pilot's arousal status into any one of a plurality of status categories by analyzing the pilot's biosignal; And
Quantifying awake state of the pilot according to the classified state category by analyzing the brain waves included in the biosignal;
Including, the method. The method of claim 1,
Receiving the bio-signal,
Receiving, as the biosignal, signals from at least one biosignal sensor attached to the pilot for at least some of the pilot's brain waves, heart rate, respiration and skin electrical activity during flight;
Including, the method. The method of claim 1,
The classifying step,
Classifying the pilot's arousal status into any of the plurality of status categories by inputting the biosignal into a classifier;
Including, the method. The method of claim 3, wherein
Wherein the classifier comprises linear discriminant analysis (LDA), support vector machines (SVM), and neural network models. The method of claim 3, wherein
Wherein the plurality of state categories include fit state, reduced concentration, fatigue, workload, and unconsciousness. The method of claim 1,
The quantification step is
Calculating a phase lag index between the pilot's EEG signal pairs during flight;
Generating a functional connectivity matrix using the phase delay index; And
Quantifying the pilot's arousal state according to the classified state category by analyzing the functional connectivity matrix using a Graph Theory;
Including, the method. The method of claim 6,
The method,
Acquiring a biosignal of the pilot when the pilot is in a flight-suitable state;
More,
The quantification step is
Calculating a global efficiency value or a local efficiency value of the functional connectivity matrix using a graph theory;
Adjusting the global efficiency value or the local efficiency value based on the pilot's biosignal when in a flight suitable state; And
Quantifying the awakening status of the pilot according to the classified state category to the adjusted Global efficiency value or the adjusted Local efficiency value;
Including, the method. The method of claim 1,
The method,
Providing classification result values for each of the plurality of state categories and the quantified awakening state information according to the classified state categories;
Further comprising, the method. The method of claim 1,
The method,
Providing appropriate feedback to the pilot based on the pilot's quantified arousal status according to the classified status category;
Further comprising, the method. In the device for quantifying the awake state of the pilot,
Communication unit for receiving a biological signal of the pilot during the flight; And
A controller configured to classify the awakening state of the pilot into any one of a plurality of state categories based on the biosignal of the pilot, and to quantify the awakening state of the pilot according to the classified state category by analyzing the biosignal of the pilot; ;
Including, the device. The method of claim 10,
And the communication unit receives, as the biosignal, a signal for at least some of the pilot's brain wave, heart rate, respiration, and skin electrical activity during flight from at least one biosignal sensor attached to the pilot. The method of claim 10,
And the controller classifies the pilot's awakening status into any one of the plurality of status categories by inputting the biosignal into a classifier. The method of claim 12,
Wherein the classifier comprises linear discriminant analysis (LDA), support vector machines (SVM), and neural network models. The method of claim 12,
Wherein the plurality of state categories include fit state, reduced concentration, fatigue, workload, and unconsciousness. The method of claim 10,
The controller calculates a phase lag index between the pilot EEG signal pairs during the flight, generates a functional connectivity matrix using the phase delay index, and generates a graph theory. And analyzing the functional connectivity matrix to quantify the alert state of the pilot according to the classified state category. The method of claim 15,
The communication unit, when the pilot is in a flight suitable state acquires the biological signal of the pilot,
The controller calculates a global efficiency value or a local efficiency value of the functional connectivity matrix by using a graph theory, and based on the biosignal of the pilot when the flight is in a suitable state, the global efficiency value or the local adjusting an efficiency value, and quantifying the pilot's arousal state according to the classified state category to the adjusted Global efficiency value or the adjusted Local efficiency value. The method of claim 10,
The device,
A display configured to provide classification result values for each of the plurality of state categories and the quantified awakening state information according to the classified state categories;
Further comprising a device. The method of claim 10,
The device,
A feedback provider for providing appropriate feedback to the pilot based on the pilot's quantified arousal status according to the classified status category;
Further comprising a device. A computer-readable recording medium having recorded thereon a program for executing the method of claim 1 on a computer.

Images