CN115097064A - Gas detection method and device, computer equipment and storage medium - Google Patents
Gas detection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115097064A CN115097064A CN202111119011.8A CN202111119011A CN115097064A CN 115097064 A CN115097064 A CN 115097064A CN 202111119011 A CN202111119011 A CN 202111119011A CN 115097064 A CN115097064 A CN 115097064A
- Authority
- CN
- China
- Prior art keywords
- attention
- self
- multitask
- network
- gas
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N33/00—Investigating or analysing materials by specific methods not covered by groups G01N1/00 - G01N31/00
- G01N33/0004—Gaseous mixtures, e.g. polluted air
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A50/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE in human health protection, e.g. against extreme weather
- Y02A50/20—Air quality improvement or preservation, e.g. vehicle emission control or emission reduction by using catalytic converters
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Combustion & Propulsion (AREA)
- Food Science & Technology (AREA)
- Medicinal Chemistry (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application belongs to the technical field of medical treatment in artificial intelligence, and relates to a gas detection method, a gas detection device, computer equipment and a storage medium based on a multitask self-attention network, wherein the method comprises the following steps: receiving gas component data of gas to be detected sent by an electronic nose system; inputting gas component data into a multitask self-attention network to detect a biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module; and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the concentration information of the biomarker. According to the method, an end-to-end deep learning network model is adopted, the sample composition can be detected by inputting original data, the corresponding marker concentration is obtained, the interference caused by human factors can be effectively avoided, and the problem of information loss caused by manual feature extraction is reduced.
Description
Technical Field
The application relates to the technical field of medical treatment in artificial intelligence, in particular to a gas detection method and device based on a multitask self-attention network, computer equipment and a storage medium.
Background
Lung cancer is the most common cancer worldwide and is also the highest mortality cancer worldwide. Early cancer screening is one of the most effective ways to reduce cancer mortality.
There is a cancer detection method that processes sensor signals according to a machine learning algorithm and finally outputs a cancer detection result.
However, the applicant finds that the traditional cancer detection method based on the machine learning algorithm generally needs to manually extract features when processing sensor signals, human interference factors are brought in the process of manually extracting the features, and the uncertain interference factors influence the algorithm to extract the features from different aspects, so that the results of the final recognition algorithm have human-induced differences. Secondly, the traditional machine learning algorithm cannot process high-dimensional data, and the following gas identification function can be performed only after data dimension reduction. The problem of loss of sensor signal data is necessarily brought about in the process of data dimension reduction, and the final identification accuracy of the algorithm is also influenced by the loss of information.
Disclosure of Invention
The embodiment of the application aims to provide a gas detection method, a gas detection device, computer equipment and a storage medium based on a multitask self-attention network, so as to solve the problem that the traditional cancer detection method based on a machine learning algorithm can cause the loss of sensor signal data, and further influence the final recognition accuracy of the algorithm.
In order to solve the above technical problem, an embodiment of the present application provides a gas detection method based on a multitask self-attention network, which adopts the following technical solutions:
receiving gas component data of gas to be detected sent by an electronic nose system;
inputting the gas composition data into a multitask self-attention network to detect a biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module;
and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the marker concentration information.
Further, the multitask self-attention module comprises:
the multi-head self-attention layer, the normalization layer, the feedforward network layer and the residual error structure.
Further, the multi-head self-attention of the multi-head self-attention layer is represented as:
head i ＝Attention(Q i ，K i ，V i )
Attention MHA ＝(concat(head 1 ，head 2 ，…，head h ))W o
wherein, W o Representing learnable linear transformation parameters; i 1, 2, h denotes the number of heads.
Further, the feed-forward network layer is represented as:
FNN＝max(0，XW 1 +b 1 )W 2 +b 2 。
in order to solve the above technical problem, an embodiment of the present application further provides a gas detection apparatus based on a multitask self-attention network, which adopts the following technical solutions:
the data receiving unit is used for receiving gas component data of the gas to be detected sent by the electronic nose system;
the gas detection unit is used for inputting the gas component data into a multitask self-attention network to detect the biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, and the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module;
and the result confirmation unit is used for confirming a gas detection result corresponding to the gas to be detected according to the biological marker and the marker concentration information.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:
comprising a memory having computer readable instructions stored therein which when executed by the processor implement the steps of the multitasking, self-attention network based gas detection method as described above.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:
the computer readable storage medium has stored thereon computer readable instructions which, when executed by a processor, implement the steps of the multitasking self-attention network based gas detection method as described above.
Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:
the application provides a gas detection method based on a multitask self-attention network, which comprises the following steps: receiving gas component data of gas to be detected sent by an electronic nose system; inputting the gas composition data into a multitask self-attention network to detect a biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position coding module and a multitask self-attention module; and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the marker concentration information. According to the method, an end-to-end deep learning network model is adopted, the sample composition can be detected by inputting original data, the corresponding marker concentration is obtained, the interference caused by human factors can be effectively avoided, and the problem of information loss caused by manual feature extraction is reduced.
Drawings
In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flowchart illustrating an implementation of a gas detection method based on a multitasking self-attention network according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a multitasking self-attention network provided in an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a multi-head self-attention device according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a residual structure and layer normalization structure provided in an embodiment of the present application;
FIG. 6 is a schematic structural diagram of a gas detection apparatus based on a multitask self-attention network according to a second embodiment of the present application;
FIG. 7 is a block diagram of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein in the description of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 101, 102, 103 to interact with a server 105 over a network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the gas detection method based on the multitask self-attention network provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the gas detection apparatus based on the multitask self-attention network is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for an implementation.
With continuing reference to fig. 2, a flowchart of an implementation of a multitask self-attention network-based gas detection method provided in an embodiment of the present application is shown, and for convenience of explanation, only the relevant portions of the present application are shown.
The gas detection method based on the multitask self-attention network comprises the following steps:
step S201: receiving gas component data of gas to be detected sent by an electronic nose system;
step S202: inputting gas component data into a multitask self-attention network to detect a biomarker of a gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module;
step S203: and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the concentration information of the biomarker.
In the embodiment of the present application, referring to the schematic structural diagram of the multitask self-attention network shown in fig. 3, the multitask self-attention network provided by the present application includes two parts, namely an encoder and a decoder, wherein the encoder is composed of a position code and a two-layer stacked self-attention module. The self-attention module includes: the system comprises a multi-head self-attention layer, a normalization layer, a feedforward network layer and a residual error structure. First, sensor data containing position information provided by a position encoding layer is streamed in parallel to a multi-headed self-attention layer to obtain an output named an intermediate vector. In addition, the multi-head self-attention mechanism can perform different linear transformations on input data, and is helpful for capturing various characteristics of signals by a model. In addition, the multi-head self-attention structure of the application has remarkable parallel processing performance because no recursive structure is used, and therefore, better effect can be achieved only by needing shorter training time. In addition, the ability to process time series data in parallel makes a multi-headed self-attention model more suitable for deployment on edge devices. And the intermediate vector flows into a feedforward neural network layer after passing through a normalization layer to obtain a low-dimensional feature vector. The feedforward neural network provides nonlinear transformation for the model, the expression capability of the model is improved, and layer normalization can accelerate the convergence of the model. The residual structure is connected with the multi-head self-attention module and the feedforward neural network, so that the problem of gradient loss in the training process can be prevented, and the layer normalization can accelerate the convergence of the model. The low-dimensional feature vector is streamed into the second stacked self-attention module to obtain the output of the coding layer named high-dimensional feature vector. In the decoder portion, the full link layer decodes the information extracted by the coding layer. Then, a gas classification and concentration prediction result is obtained through an activation function layer and a gas composition and concentration matching mechanism.
In the embodiment of the present application, since each element in the time series data flows through the encoder stack at the same time, the position information of the time series is lost. Therefore, there remains a need for a method to incorporate the order of elements into our model. We add a layer called position coding layer in front of the self-attention module to mark the position information. Assume that the original input data is X-R t×d T is the length of input data, d is the data dimension, and the position matrix generated by the position coding layer has the same shape as the input data, namely P belongs to R t×d The original input data is added to the encoding result to obtain sequence data containing position information. The position matrix may be obtained by:
where i is the index of the data in the sequence, d is the sequence data dimension size, and k is the kth dimension of the data. The above equation indicates that sin variables are added in the even dimension and cos variables are added in the odd dimension of each sequence position vector of the position matrix P. The information for each position is specific and unique and adding this information to the original data results in a sequence signal with position information.
In the embodiment of the present application, a multi-head self-attention structure is shown in fig. 4, and an attention mechanism focuses on information more critical to a current task among a plurality of information, so that attention to other information is reduced, and thus efficiency of processing the task is improved. Whereas the self-attention mechanism is the one that relates different positions within the sequence to calculate the sequence representation. The attention mechanism can be described as a function that maps a query vector Q and a series of key-value pairs (K-V), which are all linear variations of the input vector, to an output value representing the sequence element correlation. Self-attention can be achieved by a method of scaled dot-product attention. The attention is calculated by three steps:
firstly, calculating the similarity between similarity sequence elements by using a query vector Q and a key K in each key value pair (K-V) through dot product, wherein a similarity calculation expression is as follows:
similarly(Q，{K i ，v i } M )＝[similarly(Q，K 1 )，…，similarly(Q，K M )]
secondly, normalizing the M weights by using a softmax function;
finally, weighting and summing the normalized weight and the corresponding value V to obtain the attention. The implementation is shown in the following formula:
wherein d is k For querying the dimension of the vector, when d k When large, the dot product result becomes very discrete, and the dot product result is divided by
Has the function of adjusting so as to prevent the result of the inner product from being too large.
Multi-head self-attention is to split Q, K and V into multiple different parts under the condition that the total amount of parameters is kept unchanged, and the multi-head mechanism does not calculate attention once, but runs scaled dot product attention in different subspaces in parallel. And splicing the self-attention information of different subspaces, and performing linear transformation again to obtain a value serving as an output result of the multi-head self-attention. Since the self-attention is distributed in different subspaces, the multi-head self-attention mechanism can search the association information of different angles between the sequences, and the multi-head self-attention mechanism is also helpful for learning the dependency relationship of long-distance information. The multi-head self-attention calculation formula is as follows:
head i ＝Attention(Q i ，K i ，V i )
Attention MHA ＝(concat(head 1 ，head 2 ，…，head h ))W o
wherein, W o Representing learnable linear transformation parameters; i 1, 2, h denotes the number of heads.
In an embodiment of the present application, the multi-headed self-attention module is followed by a feed-forward neural network module. Since the multi-headed self-attention module involves only linear transformations, the feed-forward neural layer provides the model with nonlinear transformations that can increase the performance of the model. This layer consists of two fully-connected layers, the first fully-connected layer having an activation function Relu, the second fully-connected layer not using an activation function. The expression is as follows:
FNN＝max(0，XW 1 +b 1 )W 2 +b 2
in the embodiment of the application, a residual connection and layer normalization are arranged between the multi-head self-attention layer and the feedforward neural network layer and behind the feedforward neural network layer. Residual concatenation is typically used to solve the multi-layer network training problem, helping to avoid gradient disappearance or gradient explosion problems. Layer normalization is to normalize data, so as to accelerate the convergence speed of the model and reduce the training time of the network, and it can also improve the stability of the model. The residual structure and layer normalization is expressed by the following formula:
LayerNorm(X+MultiherdAttention(X))
LayerNorm(X+FeedForward(X))
where X represents the input of Multi-Head Attenttion or Feed Forward, MultiHeadAttenttion (X) and Feed Forward (X) represent the output, and LayerNorm represents the layer normalization, as shown in FIG. 5.
In the embodiment of the present application, the classification and regression network is composed of a fully connected layer and an activation layer, and the function of the classification and regression network is to map the extracted features of the multi-head self-attention network into a sample space. The transcription layer has two full-connection layers, the first full-connection layer carries out feature transformation and information induction, the activation function is a Relu function, nonlinear change is provided for the classification regression layer, the network convergence speed is accelerated, and the Relu function expression is as follows:
the second full-connection layer is used for data prediction output and comprises 5 neurons, the first two neurons are used for gas concentration prediction output, and the rest neurons are used for gas type prediction output. The output is processed by a Sigmoid activation function, so that the final output result is within the range of 0-1. The Sigmoid function is expressed as follows:
in the embodiment of the present application, the model is divided into two parts, one is an encoder part and one is a decoder part. The main function of the encoder part is feature extraction, and the feature extraction part can have various options. The encoder may select a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a long short term memory network (LSTM). These networks all have better feature extraction capabilities. But the multi-headed self-attentive network of the present application would be a better choice relative to gas data. CNN is suitable for processing image signals, not for time series signals. RNNs easily extract the characteristics of time signals, but due to their recursive network structure, gradient disappearance or a gradient explosion phenomenon easily occurs. The LSTM network's ability to extract features decreases as the length of the sequence signal increases, and thus is not suitable for feature extraction of long sequence signals. The multi-head self-attention network extracts the global feature relation of signals in parallel, the feature extraction capability cannot be weakened along with the increase of the sequence length, and the structure of the multi-head self-attention network can be trained in parallel, so that the training speed and the recognition speed of a model can be effectively improved.
In the embodiment of the application, the self-attention module of the application is connected in a manner that two self-attention modules are stacked in series. Other different connection means may be used. When the sequence length of the sample data set is increased, the stacking of modules can be properly increased, and the deep features of the signal can be effectively extracted.
In summary, an embodiment of the present application provides a gas detection method based on a multitask self-attention network, including: receiving gas component data of gas to be detected sent by an electronic nose system; inputting gas component data into a multitask self-attention network to detect a biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module; and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the concentration information of the biomarker. According to the method, an end-to-end deep learning network model is adopted, the sample composition can be detected by inputting original data, the corresponding marker concentration is obtained, the interference caused by human factors can be effectively avoided, and the problem of information loss caused by manual feature extraction is reduced. Meanwhile, the application can also be applied to other disease screening methods. For example, a diabetic patient is screened by detecting the concentration of acetone in exhaled air, and ammonia gas exhaled by a human body can be used as a biomarker gas for a patient with renal failure. These marker gases can also be detected using electronic nose technology and the multitask self-attention network algorithm of the present application as an auxiliary detection method for disease screening. As a novel gas detection method, the method not only can play a role in disease screening in the medical field, but also can be applied to mines. The method and the device can quickly and accurately detect out inflammable, explosive and toxic gas in a mine, and can effectively reduce the occurrence of explosion or poisoning events.
The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
Those skilled in the art will appreciate that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to computer readable instructions, which can be stored in a computer readable storage medium, and when executed, the computer readable instructions can include the processes of the embodiments of the methods described above. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a part of the steps in the flowcharts of the figures may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of execution is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the sub-steps or stages of other steps.
Example two
With further reference to fig. 6, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a gas detection apparatus based on a multitask self-attention network, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be applied to various electronic devices.
As shown in fig. 6, the gas detection apparatus 600 based on the multitask self-attention network of the present embodiment includes: a data receiving unit 610, a gas detecting unit 620, and a result confirming unit 630. Wherein:
the data receiving unit 610 is used for receiving gas component data of the gas to be detected sent by the electronic nose system;
a gas detection unit 620, configured to input gas composition data into a multitask self-attention network, where the multitask self-attention network includes an encoder and a decoder, and the encoder and the decoder are composed of a position encoding module and a multitask self-attention module;
a result confirmation unit 630, configured to confirm a gas detection result corresponding to the gas to be detected according to the biomarker and the marker concentration information.
In the embodiment of the present application, referring to the schematic structural diagram of the multitask self-attention network shown in fig. 3, the multitask self-attention network provided by the present application includes two parts, namely an encoder and a decoder, wherein the encoder is composed of a position code and a two-layer stacked self-attention module. The self-attention module includes: the system comprises a multi-head self-attention layer, a normalization layer, a feedforward network layer and a residual structure. First, sensor data containing position information provided by a position encoding layer is streamed in parallel to a multi-headed self-attention layer to obtain an output named an intermediate vector. In addition, the multi-head self-attention mechanism can perform different linear transformations on input data, and is helpful for capturing various characteristics of signals by a model. In addition, the multi-head self-attention structure of the application has remarkable parallel processing performance because no recursive structure is used, and therefore, better effect can be achieved only by needing shorter training time. In addition, the ability to process time series data in parallel makes a multi-headed self-attention model more suitable for deployment on edge devices. And the intermediate vector flows into a feedforward neural network layer after passing through a normalization layer to obtain a low-dimensional feature vector. The feedforward neural network provides nonlinear transformation for the model, the model expression capability is improved, and layer normalization can accelerate the convergence of the model. The residual structure is connected with the multi-head self-attention module and the feedforward neural network, so that the problem of gradient loss in the training process can be prevented, and the layer normalization can accelerate the convergence of the model. The low-dimensional feature vector is streamed into the second stacked self-attention module to obtain the output of the coding layer named high-dimensional feature vector. In the decoder portion, the full connection layer decodes the information extracted by the coding layer. Then, a gas classification and concentration prediction result is obtained through an activation function layer and a gas composition and concentration matching mechanism.
In the embodiment of the present application, since each element in the time series data flows through the encoder stack at the same time, the position information of the time series is lost. Therefore, there remains a need for a method to incorporate the order of elements into our model. We add a layer called position coding layer in front of the self-attention module to mark the position information. Assume that the original input data is X-R t×d T is the length of input data, d is the data dimension, and the position matrix generated by the position coding layer has the same shape as the input data, namely P belongs to R t×d The original input data is added to the encoding result to obtain sequence data containing position information. The position matrix may be obtained by:
where i is the index of the data in the sequence, d is the sequence data dimension size, and k is the kth dimension of the data. The above equation indicates that sin variables are added in the even dimension and cos variables are added in the odd dimension of each sequence position vector of the position matrix P. The information for each position is specific and unique and adding this information to the original data results in a sequence signal with position information.
In the embodiment of the present application, a multi-head self-attention structure is shown in fig. 4, and an attention mechanism focuses on information more critical to a current task among a plurality of information, so that attention to other information is reduced, and thus efficiency of processing the task is improved. Whereas the self-attention mechanism is the one that relates different positions within the sequence to calculate the sequence representation. The attention mechanism can be described as a function that maps a query vector Q and a series of key-value pairs (K-V), which are all linear variations of the input vector, to an output value representing the sequence element correlation. Self-attention can be achieved by a method of scaled dot-product attention. The attention is calculated by three steps:
firstly, calculating the similarity between similarity sequence elements by using a query vector Q and a key K in each key value pair (K-V) through a dot product, wherein a similarity calculation expression is as follows:
similarly(Q，{K i ，v i } M )＝[similarly(Q，K 1 )，…，similarly(Q，K M )]
secondly, normalizing the M weights by using a softmax function;
and thirdly, weighting and summing the normalized weight and the corresponding value V to obtain the attention. The implementation is shown in the following formula:
wherein d is k For querying the dimension of the vector, when d k When large, the dot product result becomes very discrete, and the dot product result is divided by
The regulation function is realized, so that the inner product result is not too large.
Multi-head self-attention is to split Q, K and V into multiple different parts under the condition that the total amount of parameters is kept unchanged, and the multi-head mechanism does not calculate attention once, but runs scaled dot product attention in different subspaces in parallel. And splicing the self-attention information of different subspaces, and performing linear transformation once again to obtain a value serving as an output result of the multi-head self-attention. Since the self-attention is distributed in different subspaces, the multi-head self-attention mechanism can search the association information of different angles between the sequences, and the multi-head self-attention mechanism is also helpful for learning the dependency relationship of long-distance information. The multi-head self-attention calculation formula is as follows:
head i ＝Attention(Q i ，K i ，V i )
Attention MHA ＝(concat(head 1 ，head 2 ，…，head h ))W o
wherein, W o Representing a learnable linear transformation parameter; i 1, 2, h denotes the number of heads.
In the embodiment of the application, the multi-head self-attention module is followed by the feedforward neural network module. Since the multi-headed self-attention module involves only linear transformations, the feed-forward neural layer provides a nonlinear transformation to the model that can increase the model's performance. This layer consists of two fully-connected layers, the first fully-connected layer having an activation function Relu, the second fully-connected layer not using an activation function. The expression is as follows:
FNN＝max(0，XW 1 +b 1 )W 2 +b 2
in the embodiment of the application, a residual connection and layer standardization exists between the multi-head self-attention layer and the feedforward neural network layer and after the feedforward neural network layer. Residual concatenation is typically used to solve the multi-layer network training problem, helping to avoid gradient disappearance or gradient explosion problems. Layer normalization is to normalize data, so as to accelerate the convergence speed of the model and reduce the training time of the network, and it can also improve the stability of the model. The residual structure and layer normalization is expressed by the following formula:
LayerNorm(X+MultiherdAttention(X))
LayerNorm(X+FeedForward(X))
where X represents the input of Multi-Head Attention or Feed Forward, Multi HeadAttention (X) and Feed Forward (X) represent the output, and LayerNorm represents layer normalization, as shown in FIG. 5.
In the embodiment of the application, the classification and regression network is composed of a full connection layer and an activation layer, and the function of the classification and regression network is to map the extracted features of the multi-head self-attention network into a sample space. The transcription layer has two full connection layers, the first full connection layer performs characteristic transformation and information induction, the activation function is Relu function, provides nonlinear change for the classification regression layer and accelerates the network convergence speed, the Relu function expression is as follows:
the second full-connection layer is used for data prediction output and comprises 5 neurons, the first two neurons are used for gas concentration prediction output, and the rest neurons are used for gas type prediction output. And the output is subjected to a Sigmoid activation function, so that the final output result is in the range of 0-1. The Sigmoid function is expressed as follows:
in the embodiment of the present application, the model is divided into two parts, one is an encoder part and one is a decoder part. The main function of the encoder part is feature extraction, and the feature extraction part can have various options. The encoder may select a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a long short term memory network (LSTM). These networks all have better feature extraction capabilities. But the multi-headed self-attentive network of the present application would be a better choice than gas data. CNN is suitable for processing image signals, not for time series signals. RNNs easily extract the features of time signals, but due to their recursive network structure, gradient disappearance or gradient explosion phenomena easily occur. The LSTM network's ability to extract features decreases as the length of the sequence signal increases, and thus is not suitable for feature extraction of long sequence signals. The multi-head self-attention network extracts the global feature relation of signals in parallel, the feature extraction capability cannot be weakened along with the increase of the sequence length, and the structure can be trained in parallel, so that the training speed and the recognition speed of a model can be effectively improved.
In the embodiment of the application, the self-attention module of the application is connected in a manner that two self-attention modules are stacked in series. Other different connection means may be used. When the sequence length of the sample data set is increased, the stacking of modules can be properly increased, and the deep features of the signal can be effectively extracted.
In summary, the second embodiment of the present application provides a gas detection apparatus 600 based on a multitasking self-attention network, including: the data receiving unit 610 is used for receiving gas component data of the gas to be detected sent by the electronic nose system; a gas detection unit 620, configured to input gas composition data into a multitask self-attention network, where the multitask self-attention network includes an encoder and a decoder, and the encoder and the decoder are composed of a position encoding module and a multitask self-attention module; a result confirmation unit 630, configured to confirm a gas detection result corresponding to the gas to be detected according to the biomarker and the marker concentration information. According to the method, an end-to-end deep learning network model is adopted, the sample composition can be detected by inputting original data, the corresponding marker concentration is obtained, the interference caused by human factors can be effectively avoided, and the problem of information loss caused by manual feature extraction is reduced. Meanwhile, the application can also be applied to other disease screening methods. For example, a diabetic patient is screened by detecting the concentration of acetone in exhaled air, and ammonia gas exhaled by a human body can be used as a biomarker gas for a patient with renal failure. These marker gases can also be detected using electronic nose technology and the multitask self-attention network algorithm of the present application as an auxiliary detection method for disease screening. As a novel gas detection method, the method not only can play a role in disease screening in the medical field, but also can be applied to mines. The method and the device can quickly and accurately detect out inflammable, explosive and toxic gas in a mine, and can effectively reduce the occurrence of explosion or poisoning events.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 7, fig. 7 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 200 includes a memory 210, a processor 220, and a network interface 230 communicatively coupled to each other via a system bus. It is noted that only computer device 200 having components 210 and 230 is shown, but it is understood that not all of the illustrated components are required and that more or fewer components may alternatively be implemented. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 210 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 210 may be an internal storage unit of the computer device 200, such as a hard disk or a memory of the computer device 200. In other embodiments, the memory 210 may also be an external storage device of the computer device 200, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the computer device 200. Of course, the memory 210 may also include both internal and external storage devices of the computer device 200. In this embodiment, the memory 210 is generally used for storing an operating system and various types of application software installed on the computer device 200, such as computer readable instructions of a multitasking self-attention network-based gas detection method. In addition, the memory 210 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 220 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 220 generally functions to control the overall operation of the computer device 200. In this embodiment, the processor 220 is configured to execute computer readable instructions stored in the memory 210 or to process data, such as executing computer readable instructions of the multitask self-attention network based gas detection method.
The network interface 230 may include a wireless network interface or a wired network interface, and the network interface 230 is generally used to establish communication connections between the computer device 200 and other electronic devices.
The computer equipment provided by the application adopts an end-to-end deep learning network model, can detect the composition of the sample by inputting the original data and obtain the corresponding marker concentration, can effectively avoid the interference caused by human factors and reduce the problem of information loss caused by manual feature extraction.
The present application provides yet another embodiment, which is a computer-readable storage medium having computer-readable instructions stored thereon which are executable by at least one processor to cause the at least one processor to perform the steps of the multitask self-attention network based gas detection method as described above.
According to the computer-readable storage medium, an end-to-end deep learning network model is adopted, the sample composition can be detected by inputting original data, the corresponding marker concentration can be obtained, the interference caused by human factors can be effectively avoided, and the problem of information loss caused by manual feature extraction is solved.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It should be understood that the above-described embodiments are merely illustrative of some, but not all, embodiments of the present application, and that the present invention is not limited by the scope of the appended claims. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the attached drawings of the present application are directly or indirectly applied to other related technical fields, and are within the protection scope of the present application patent.
Claims (10)
1. A gas detection method based on a multitask self-attention network is characterized by comprising the following steps:
receiving gas component data of gas to be detected sent by an electronic nose system;
inputting the gas composition data into a multitask self-attention network to detect a biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, wherein the multitask self-attention network comprises an encoder and a decoder which are composed of a position coding module and a multitask self-attention module;
and confirming a gas detection result corresponding to the gas to be detected according to the biomarker and the marker concentration information.
2. The multitask, self-attention network-based gas detection method according to claim 1, wherein the multitask, self-attention module comprises:
the multi-head self-attention layer, the normalization layer, the feedforward network layer and the residual error structure.
3. The multitask self-attention network-based gas detection method according to claim 2, wherein the multi-head self-attention of the multi-head self-attention layer is represented as:
head i ＝Attention(Q i ，K i ，V i )
Attention MHA ＝(concat(head 1 ，head 2 ，…，head h ))W o
wherein, W o Representing learnable linear transformation parameters; i 1, 2, h denotes the number of heads.
4. The multitask, self-attention network based gas detection method of claim 2, wherein the feed forward network layer is represented as:
FNN＝max(0，XW 1 +b 1 )W 2 +b 2 。
5. a gas detection device based on a multitasking self-attention network, comprising:
the data receiving unit is used for receiving gas component data of the gas to be detected sent by the electronic nose system;
the gas detection unit is used for inputting the gas component data into a multitask self-attention network to detect the biomarker of the gas to be detected and marker concentration information corresponding to the biomarker, and the multitask self-attention network comprises an encoder and a decoder which are composed of a position encoding module and a multitask self-attention module;
and the result confirmation unit is used for confirming a gas detection result corresponding to the gas to be detected according to the biological marker and the marker concentration information.
6. The multitask, self-attention network-based gas detection device of claim 5, wherein the multitask, self-attention module comprises:
the multi-head self-attention layer, the normalization layer, the feedforward network layer and the residual error structure.
7. The multitask, self-attention network-based gas detection method according to claim 6, wherein the multi-head self-attention of the multi-head self-attention layer is represented as:
head i ＝Attention(Q i ，K i ，V i )
Attention MHA ＝(concat(head 1 ，head 2 ，…，head h ))W o
wherein, W o Representing a learnable linear transformation parameter; i 1, 2, h denotes the number of heads.
8. The multitask, self-attention network-based gas detection method of claim 6, wherein the feed-forward network layer is represented as:
FNN＝max(0，XW 1 +b 1 )W 2 +b 2 。
9. a computer device comprising a memory having computer readable instructions stored therein and a processor that when executed performs the steps of the multitasking self-attention network based gas detection method according to any one of claims 1-4.
10. A computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, implement the steps of the multitasking self-attention network based gas detection method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111119011.8A CN115097064B (en) | 2021-09-24 | 2021-09-24 | Gas detection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111119011.8A CN115097064B (en) | 2021-09-24 | 2021-09-24 | Gas detection method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115097064A true CN115097064A (en) | 2022-09-23 |
| CN115097064B CN115097064B (en) | 2023-06-20 |
Family
ID=83287627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111119011.8A Active CN115097064B (en) | 2021-09-24 | 2021-09-24 | Gas detection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115097064B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116189800A (en) * | 2023-02-23 | 2023-05-30 | 深圳大学 | Pattern recognition method, device, equipment and storage medium based on gas detection |
| CN116502158A (en) * | 2023-02-07 | 2023-07-28 | 北京纳通医用机器人科技有限公司 | Method, device, equipment and storage medium for identifying lung cancer stage |
| CN117091799A (en) * | 2023-10-17 | 2023-11-21 | 湖南一特医疗股份有限公司 | Intelligent three-dimensional monitoring method and system for oxygen supply safety of medical center |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103018282A (en) * | 2012-12-21 | 2013-04-03 | 上海交通大学 | Electronic nose system for early detection of lung cancer |
| US20140096590A1 (en) * | 2012-05-07 | 2014-04-10 | Alexander Himanshu Amin | Electronic nose system and method |
| CN104751004A (en) * | 2015-04-15 | 2015-07-01 | 苏州大学 | Disease pre-warning method, device and system |
| US20160106935A1 (en) * | 2014-10-17 | 2016-04-21 | Qualcomm Incorporated | Breathprint sensor systems, smart inhalers and methods for personal identification |
| CN105738434A (en) * | 2016-02-01 | 2016-07-06 | 清华大学深圳研究生院 | Diabetes diagnostic system for detecting respiratory gases based on electronic nose |
| CN107463766A (en) * | 2017-06-23 | 2017-12-12 | 深圳市中识创新科技有限公司 | Generation method, device and the computer-readable recording medium of blood glucose prediction model |
| CN108693353A (en) * | 2018-05-08 | 2018-10-23 | 重庆大学 | A kind of long-range diabetes intelligent diagnosis system detecting breathing gas based on electronic nose |
| CN110146642A (en) * | 2019-05-14 | 2019-08-20 | 上海大学 | A kind of smell analysis method and device |
| CN111540463A (en) * | 2019-12-18 | 2020-08-14 | 中国科学院上海微系统与信息技术研究所 | Exhaled gas detection method and system based on machine learning |
-
2021
- 2021-09-24 CN CN202111119011.8A patent/CN115097064B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140096590A1 (en) * | 2012-05-07 | 2014-04-10 | Alexander Himanshu Amin | Electronic nose system and method |
| CN103018282A (en) * | 2012-12-21 | 2013-04-03 | 上海交通大学 | Electronic nose system for early detection of lung cancer |
| US20160106935A1 (en) * | 2014-10-17 | 2016-04-21 | Qualcomm Incorporated | Breathprint sensor systems, smart inhalers and methods for personal identification |
| CN104751004A (en) * | 2015-04-15 | 2015-07-01 | 苏州大学 | Disease pre-warning method, device and system |
| CN105738434A (en) * | 2016-02-01 | 2016-07-06 | 清华大学深圳研究生院 | Diabetes diagnostic system for detecting respiratory gases based on electronic nose |
| CN107463766A (en) * | 2017-06-23 | 2017-12-12 | 深圳市中识创新科技有限公司 | Generation method, device and the computer-readable recording medium of blood glucose prediction model |
| CN108693353A (en) * | 2018-05-08 | 2018-10-23 | 重庆大学 | A kind of long-range diabetes intelligent diagnosis system detecting breathing gas based on electronic nose |
| CN110146642A (en) * | 2019-05-14 | 2019-08-20 | 上海大学 | A kind of smell analysis method and device |
| CN111540463A (en) * | 2019-12-18 | 2020-08-14 | 中国科学院上海微系统与信息技术研究所 | Exhaled gas detection method and system based on machine learning |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502158A (en) * | 2023-02-07 | 2023-07-28 | 北京纳通医用机器人科技有限公司 | Method, device, equipment and storage medium for identifying lung cancer stage |
| CN116502158B (en) * | 2023-02-07 | 2023-10-27 | 北京纳通医用机器人科技有限公司 | Method, device, equipment and storage medium for identifying lung cancer stage |
| CN116189800A (en) * | 2023-02-23 | 2023-05-30 | 深圳大学 | Pattern recognition method, device, equipment and storage medium based on gas detection |
| CN116189800B (en) * | 2023-02-23 | 2023-08-18 | 深圳大学 | Pattern recognition method, device, equipment and storage medium based on gas detection |
| CN117091799A (en) * | 2023-10-17 | 2023-11-21 | 湖南一特医疗股份有限公司 | Intelligent three-dimensional monitoring method and system for oxygen supply safety of medical center |
| CN117091799B (en) * | 2023-10-17 | 2024-01-02 | 湖南一特医疗股份有限公司 | Intelligent three-dimensional monitoring method and system for oxygen supply safety of medical center |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115097064B (en) | 2023-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115097064B (en) | Gas detection method and device, computer equipment and storage medium | |
| Wu et al. | Vulnerability detection with deep learning | |
| WO2020232861A1 (en) | Named entity recognition method, electronic device and storage medium | |
| CN111428044A (en) | Method, device, equipment and storage medium for obtaining supervision identification result in multiple modes | |
| CN111241304A (en) | Answer generation method based on deep learning, electronic device and readable storage medium | |
| CN109743311A (en) | A kind of WebShell detection method, device and storage medium | |
| CN112084779B (en) | Entity acquisition method, device, equipment and storage medium for semantic recognition | |
| CN112560504B (en) | Method, electronic equipment and computer readable medium for extracting information in form document | |
| CN111324738A (en) | Method and system for determining text label | |
| CN111783471A (en) | Semantic recognition method, device, equipment and storage medium of natural language | |
| CN108229680B (en) | Neural network system, remote sensing image recognition method, device, equipment and medium | |
| CN117807482B (en) | Method, device, equipment and storage medium for classifying customs clearance notes | |
| CN113569998A (en) | Automatic bill identification method and device, computer equipment and storage medium | |
| CN112232052A (en) | Text splicing method and device, computer equipment and storage medium | |
| CN110222144B (en) | Text content extraction method and device, electronic equipment and storage medium | |
| CN115438149A (en) | End-to-end model training method and device, computer equipment and storage medium | |
| CN114398466A (en) | Complaint analysis method and device based on semantic recognition, computer equipment and medium | |
| CN113901836A (en) | Word sense disambiguation method and device based on context semantics and related equipment | |
| CN113807089A (en) | Text data processing method, neural network training method and related equipment | |
| CN112364649B (en) | Named entity identification method and device, computer equipment and storage medium | |
| CN114781358A (en) | Text error correction method, device and equipment based on reinforcement learning and storage medium | |
| CN112733645A (en) | Handwritten signature verification method and device, computer equipment and storage medium | |
| CN113657104A (en) | Text extraction method and device, computer equipment and storage medium | |
| CN113159315A (en) | Neural network training method, data processing method and related equipment | |
| CN113420869A (en) | Translation method based on omnidirectional attention and related equipment thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |