CN113344119A - Small sample smoke monitoring method under complex environment of industrial Internet of things - Google Patents
Small sample smoke monitoring method under complex environment of industrial Internet of things Download PDFInfo
- Publication number
- CN113344119A CN113344119A CN202110718610.5A CN202110718610A CN113344119A CN 113344119 A CN113344119 A CN 113344119A CN 202110718610 A CN202110718610 A CN 202110718610A CN 113344119 A CN113344119 A CN 113344119A
- Authority
- CN
- China
- Prior art keywords
- smoke
- discriminator
- environment
- training
- small
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000779 smoke Substances 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012544 monitoring process Methods 0.000 title claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 17
- 238000013526 transfer learning Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to a small sample smoke monitoring method under a complex environment of an industrial Internet of things, which realizes smoke detection by utilizing two parallel branches and specifically comprises the following steps: s1, generating a confrontation network generation data set by the first branch by using conditions, inputting the generated data set into a convolutional neural network for training, and fixing parameters; and S2, the second branch adopts a transfer learning method to transmit the source domain picture and the target domain picture into a convolutional neural network for training, S3 obtains new probabilities by weighting the probabilities obtained in the steps S1 and S2, and the label with the highest probability is the category, so that smoke detection under a small sample is realized. The invention combines the generation countermeasure network and the transfer learning method, the former solves the problem of small samples by expanding a data set, the latter solves the problem by the transfer learning, and the two are combined, so that the model can well monitor smoke in the snow environment even under the condition of only a small amount of samples.
Description
Technical Field
The invention belongs to the technical field of intelligent image recognition, particularly relates to a small sample smoke monitoring method in an industrial Internet of things complex environment, and particularly relates to a small sample smoke monitoring method in an industrial Internet of things complex environment based on generation of a countermeasure network and transfer learning.
Background
In industrial environments, where fires pose a great economic and social hazard, it is important to monitor them, and fires produce smoke and move up faster than fires so that smoke can be seen from a distance, and early detection of smoke can help in detecting fires, which is helpful to disaster management systems. Many methods of monitoring smoke have been proposed over the years, such as: combining color information with motion for smoke detection, computing motion features using optical flow, and then classifying them into smoke and non-smoke using a back propagation neural network; color features are combined with the energy of the image to perform smoke detection, etc. These methods, while performing well in ordinary video surveillance, do not perform satisfactorily in snow surveillance environments. For example, CN112101473A discloses a smoke detection algorithm based on small sample learning, which is a method for generating a large amount of sample data through an improved generation countermeasure network, inputting the data into a convolutional neural network for learning and training, and adjusting parameters of the neural network to achieve the purpose of accurately detecting a fire, but this method requires a large amount of sample data, and has a poor detection effect in a complex environment, especially in a snow environment and in a small number of samples.
Smoke detection in a snow environment is a challenging task and plays a key role in disaster management of an industrial system, most of smoke detection methods in the prior art have the problems of disorder and unclear content, so that the result of a video stream captured from the snow environment is unsatisfactory, but due to the nature of disaster management, the accuracy of smoke detection must be high enough, the number of false alarms needs to be greatly reduced, otherwise, a large amount of resources are wasted, and the safety cannot be guaranteed, so that the methods are difficult to apply to smoke detection in the snow environment. Although there are also many smoke monitoring methods based on deep learning in the snow environment recently, the monitoring effect is greatly influenced because of the few picture samples that can be acquired.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method for smoke detection based on generation countermeasure network and transfer learning, the present invention uses two parallel branches, the first branch uses an improved conditional generation countermeasure network to expand the data set, and inputting the images into a convolutional neural network such as mobilenet v2 for training, wherein the second image adopts a transfer learning method, the source domain uses four types of images including smoke under the snowy environment, smoke under the snowless environment, smoke under the snowy environment and smoke under the snowy environment under various environments, the target domain uses the same four types of images under the industrial environment, the neural networks of the two branches can output probability values of certain types of images, the probabilities obtained by the two branches are weighted to obtain new probabilities, and the maximum probability is the type to which the probability belongs, so that smoke detection under a small sample is realized.
Specifically, the small sample smoke monitoring method comprises the following steps:
step a: the first branch is trained and its parameters are fixed. The method adopts improved conditions to generate a pseudo image for the countermeasure network to generate real smoke, and is particularly divided into four types: smog in snowy environment, smog in snowless environment, and snowy environmentNo smoke exists under the environment without the smoke and the snow. Unlike the conventional conditional generation countermeasure network which uses one generator G and two discriminators, the picture generated by the generator and the real picture are simultaneously transferred into the two discriminators, but the picture is transferred into the discriminator D1Previously, the data set is segmented into small pieces, thereby improving the authenticity of the network on local features when generating the data set. The loss of the network as a whole is represented by:
μ、σ、
is a parameter that is a function of,
shown is discriminator D1The loss of (a) is reduced to (b),
shown is discriminator D2Loss of (L)GShown is the loss of generator D, shown as;
LG＝E[log(D1(G(z|y))+D2(G(z|y)))]
where z is the input noise vector, E is the desired operator, y is the class label, x represents the target real image, and x represents the real picture cut into small blocks.
The specific training comprises the following processes:
first, the generator generates a labeled pseudo image from the input noise and the label. Then, the image is transferred to a discriminator D2Simultaneously, the image is cut into small blocks and then transmitted to a discriminator D1In (1). After the front feed-through, the discriminator sends the error gradient to the generator, and the discriminator updates itself. Finally, the discriminator is updated by a loss function.
And then inputting the data generated by the generation countermeasure network into a convolutional neural network for training, and fixing the parameters.
B, training a second branch by a transfer learning method, training the second branch after training the first branch, wherein the source domain uses four types of pictures of smoke under the snow environment, smoke under the snow-free environment, smoke under the snow environment and smoke under the snow environment under various environments, namely the non-industrial environment, and the target domain uses the same four types of pictures under the industrial environment, and the training comprises the following specific steps:
s2.1, transmitting the picture of the source domain into a convolutional neural network for training;
s2.2, transferring the knowledge of the trained convolutional layer and the fully-connected layer of the convolutional neural network to a target domain image data set, and adding a fully-connected layer to form a new model;
and S2.3, carrying out fine tuning training on the new model in the step S2.2 by using the data set of the target domain.
Step c, the probability P of the first branch outputiAnd probability of second branch output
Weighting to obtain a new probability, as shown in the following formula:
where α is a hyperparameter. The label with the highest probability is the category.
The invention has the beneficial effects that: the small sample smoke monitoring method under the complex environment of the industrial Internet of things uses two parallel branches, the first branch uses an improved condition to generate a confrontation network to generate a data set, the confrontation network is input into a convolutional neural network for training, the cost of manually adding labels can be saved due to the fact that the output data are provided with labels, the second branch adopts a transfer learning method, the probability output by the first branch and the probability output by the second branch are weighted to obtain a new probability, and the label with the highest probability is the category.
The scheme combines a generation countermeasure network and a transfer learning method, wherein the former solves the problem of small samples by expanding a data set, the latter solves the problem of small samples by transfer learning, and the two are combined, so that the model can well monitor smoke in a snow environment even under the condition of only a small number of samples.
Drawings
FIG. 1 is a schematic diagram of the training process for generating a countermeasure network according to the present invention.
FIG. 2 is a schematic diagram of the inventive transfer learning process.
FIG. 3 is a schematic diagram of the overall structure of the model of the present invention.
Detailed Description
In the following description, for purposes of explanation, numerous implementation details are set forth in order to provide a thorough understanding of the embodiments of the invention. It should be understood, however, that these implementation details are not to be interpreted as limiting the invention. That is, in some embodiments of the invention, such implementation details are not necessary. In addition, some conventional structures and components are shown in simplified schematic form in the drawings.
As shown in fig. 3, the overall model structure diagram of the present invention is that two parallel branches are used to implement smoke monitoring in the environment of the internet of things when snow exists, the first branch is to generate a data set by using a condition to generate an antagonistic network, and then input the data set into the mobilenet v2 for training, the other branch is to adopt the migration learning, the source domain uses four types of pictures of smoke in the snow environment, smoke in the snow-free environment, smoke in the snow environment, and smoke in the snow environment, the pictures in the source domain are easy to obtain, a larger data set can be obtained relatively easily, the target domain uses the same four types of pictures in the industrial environment, and there are fewer pictures in the industrial environment where there are few pictures in the snow condition, so that the migration learning is needed to migrate the experience in other environments into the industrial environment, and finally, weighting the probabilities obtained by the two branches to obtain a new probability, wherein the probability with the maximum probability is the category, so that the smoke detection under a small sample is realized.
Specifically, the small-sample smoke monitoring method under the complex environment of the industrial Internet of things comprises the following steps:
s1, generating a data set by the first branch using the condition to generate the countermeasure network, inputting the generated data set into the convolutional neural network for training, and fixing the parameters, wherein the specific training process includes the following steps as shown in fig. 1:
s1.1: the generator generates a labeled pseudo image according to the input noise and the label: smog in a snowy environment, and smog in a snowy environment. Unlike the existing method which performs binary classification on smoke and no smoke, the method classifies each picture into one of the following four categories: "smoke exists", "smoke-free", "smoke exists when snow exists" and "smoke-free when snow exists". The pictures with the four types of labels are input into a convolutional neural network for training, and the network can effectively identify the pictures during testing.
S1.2: in the conventional conditional generation countermeasure network model, only one generator and one discriminator are provided, the discriminator focuses on the authenticity of the overall characteristic, but cannot consider the local characteristic, which affects the efficiency of the network and the authenticity of the generated image, therefore, the method additionally adds one discriminator in the conventional conditional generation countermeasure network, and transmits the pseudo image of the step S1.1 to the discriminator D2Meanwhile, the pseudo image of step S1.1 is cut into small blocks and then transmitted to a discriminator D1In this way, the local reality of the image can be improved, and the overall reality of the image can be improved.
S1.3: after the front feed-through, the discriminator sends the error gradient to the generator, and the discriminator updates itself.
S1.4: updating discriminator D by a loss function1And discriminator D2。
The loss of the condition generating countermeasure network as a whole is represented by the following formula:
LG＝E[log(D1(G(z|y))+D2(G(z|y)))]
Where z is the input noise vector, E is the desired operator, y is the class label, x represents the target real image, and x' represents the real picture cut into small blocks.
And S2, the second branch adopts a transfer learning method to transmit the source domain picture and the target domain picture into a convolutional neural network for training to obtain class probability.
As shown in fig. 2, in the migration learning process, four types of pictures, namely, smoke in a snow environment, smoke in a non-industrial environment, smoke in a snow-free environment, and smoke in a non-industrial environment are used in a source domain, four types of pictures in an industrial environment are used in a target domain, and the pictures in the source domain and the target domain are different in environment, basically similar in other conditions and similar in distribution, so that a better effect can be obtained by using the migration learning.
The specific training steps are as follows:
s2.1, transmitting the picture of the source domain into a convolutional neural network for training;
s2.2, transferring the knowledge of the trained convolutional layer and the fully-connected layer of the convolutional neural network to a target domain image data set, and adding a fully-connected layer to form a new model;
and S2.3, carrying out fine tuning training on the new model in the step S2.2 by using the data set of the target domain.
S3, weighting the probabilities obtained in the steps S1 and S2 to obtain a new probability, wherein the label with the highest probability is the category, and therefore smoke detection under a small sample is achieved.
The parallel structure used by the invention can accurately identify the smoke under the complex environment even if the samples are less.
The above description is only an embodiment of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (6)
1. A small sample smoke monitoring method in a complex environment of an industrial Internet of things is characterized by comprising the following steps: the monitoring method realizes smoke detection by utilizing two parallel branches, and specifically comprises the following steps:
s1, generating a confrontation network generation data set by the first branch by using conditions, inputting the generated data set into a convolutional neural network for training, and fixing parameters to obtain class probability;
s2, the second branch adopts a transfer learning method to transmit the source domain picture and the target domain picture into a convolutional neural network for training to obtain class probability;
s3, weighting the probabilities obtained in the steps S1 and S2 to obtain a new probability, wherein the label with the highest probability is the category, and therefore smoke detection under a small sample is achieved.
2. The small-sample smoke monitoring method in the complex environment of the industrial internet of things according to claim 1, characterized in that: the conditional generation countermeasure network employs a generator and a discriminator D1And discriminator D2The picture is rubbed into the discriminator D1Previously divided into small blocks.
3. The small-sample smoke monitoring method in the complex environment of the industrial internet of things according to claim 1, characterized in that: in step S1, the generated data set is input into a convolutional neural network for training, and the specific training step includes:
s1.1: the generator generates a pseudo image with a label according to the input noise and the label;
s1.2: the pseudo-image of step S1.1 is transferred to discriminator D2Meanwhile, the pseudo image of step S1.1 is cut into small blocks and then transmitted to a discriminator D1Performing the following steps;
s1.3: after the front feed-through is finished, the discriminator sends the error gradient to the generator, and the discriminator updates the discriminator;
s1.4: updating discriminator D by a loss function1And discriminator D2。
4. The small-sample smoke monitoring method in the complex environment of the industrial internet of things according to claim 3, characterized in that: in said step S1.4, the overall loss of said conditional generation countermeasure network is represented by:
wherein: mu, sigma,
Is a parameter that is a function of,
shown is discriminator D1The loss of (a) is reduced to (b),
shown is discriminator D2Loss of (L)GDenoted is the loss of generator D:
LG＝E[log(D1(G(z|y))+D2(G(z|y)))]
where z is the input noise vector, E is the desired operator, y is the class label, x represents the target real image, and x' represents the real picture cut into small blocks.
5. The small-sample smoke monitoring method in the complex environment of the industrial internet of things according to claim 1, characterized in that: the source domain pictures which are transmitted into the convolutional neural network for training use four types of pictures, namely smoke in a snow environment, smoke in a non-snow environment, smoke in a snow environment and smoke in a non-snow environment in various environments, and the target domain pictures which are transmitted into the convolutional neural network for training use four types of pictures, namely smoke in a snow environment, smoke in a non-snow environment, smoke in a snow environment and smoke in a non-snow environment in an industrial environment.
6. The small-sample smoke monitoring method in the complex environment of the industrial Internet of things according to claim 1 or 5, characterized by comprising the following steps: the step S2 of importing the source domain picture and the target domain picture into the convolutional neural network training specifically includes the following steps:
s2.1, transmitting the picture of the source domain into a convolutional neural network for training;
s2.2, transferring the knowledge of the trained convolutional layer and the fully-connected layer of the convolutional neural network to a target domain image data set, and adding a fully-connected layer to form a new model;
and S2.3, carrying out fine tuning training on the new model in the step S2.2 by using the data set of the target domain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110718610.5A CN113344119A (en) | 2021-06-28 | 2021-06-28 | Small sample smoke monitoring method under complex environment of industrial Internet of things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110718610.5A CN113344119A (en) | 2021-06-28 | 2021-06-28 | Small sample smoke monitoring method under complex environment of industrial Internet of things |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113344119A true CN113344119A (en) | 2021-09-03 |
Family
ID=77479116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110718610.5A Pending CN113344119A (en) | 2021-06-28 | 2021-06-28 | Small sample smoke monitoring method under complex environment of industrial Internet of things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113344119A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117095257A (en) * | 2023-10-16 | 2023-11-21 | 珠高智能科技(深圳)有限公司 | Multi-mode large model fine tuning method, device, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109272023A (en) * | 2018-08-27 | 2019-01-25 | 中国科学院计算技术研究所 | A kind of Internet of Things transfer learning method and system |
| CN109522965A (en) * | 2018-11-27 | 2019-03-26 | 天津工业大学 | A kind of smog image classification method of the binary channels convolutional neural networks based on transfer learning |
| CN109977790A (en) * | 2019-03-04 | 2019-07-05 | 浙江工业大学 | A kind of video smoke detection and recognition methods based on transfer learning |
| WO2020142461A1 (en) * | 2018-12-31 | 2020-07-09 | Oregon Health & Science University | Translation of images of stained biological material |
| CN111767800A (en) * | 2020-06-02 | 2020-10-13 | 华南师范大学 | Remote sensing image scene classification score fusion method, system, equipment and storage medium |
| CN112101473A (en) * | 2020-09-22 | 2020-12-18 | 南京邮电大学 | Smoke detection algorithm based on small sample learning |
| US20210012198A1 (en) * | 2018-05-31 | 2021-01-14 | Huawei Technologies Co., Ltd. | Method for training deep neural network and apparatus |
-
2021
- 2021-06-28 CN CN202110718610.5A patent/CN113344119A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210012198A1 (en) * | 2018-05-31 | 2021-01-14 | Huawei Technologies Co., Ltd. | Method for training deep neural network and apparatus |
| CN109272023A (en) * | 2018-08-27 | 2019-01-25 | 中国科学院计算技术研究所 | A kind of Internet of Things transfer learning method and system |
| CN109522965A (en) * | 2018-11-27 | 2019-03-26 | 天津工业大学 | A kind of smog image classification method of the binary channels convolutional neural networks based on transfer learning |
| WO2020142461A1 (en) * | 2018-12-31 | 2020-07-09 | Oregon Health & Science University | Translation of images of stained biological material |
| CN109977790A (en) * | 2019-03-04 | 2019-07-05 | 浙江工业大学 | A kind of video smoke detection and recognition methods based on transfer learning |
| CN111767800A (en) * | 2020-06-02 | 2020-10-13 | 华南师范大学 | Remote sensing image scene classification score fusion method, system, equipment and storage medium |
| CN112101473A (en) * | 2020-09-22 | 2020-12-18 | 南京邮电大学 | Smoke detection algorithm based on small sample learning |
Non-Patent Citations (1)
| Title |
|---|
| 朱洪波;杨龙祥;: "物联网技术体系创新与智慧服务产业发展", 信息通信技术, no. 05, pages 4 - 5 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117095257A (en) * | 2023-10-16 | 2023-11-21 | 珠高智能科技(深圳)有限公司 | Multi-mode large model fine tuning method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Cyber security in smart cities: a review of deep learning-based applications and case studies | |
| Wu et al. | Forest fire recognition based on feature extraction from multi-view images. | |
| Abeshu et al. | Deep learning: The frontier for distributed attack detection in fog-to-things computing | |
| Bansal et al. | A comparative analysis of machine learning techniques for botnet detection | |
| CN108229550B (en) | Cloud picture classification method based on multi-granularity cascade forest network | |
| Han et al. | Meta-learning adversarial domain adaptation network for few-shot text classification | |
| CN113159329B (en) | Model training method, device, equipment and storage medium | |
| Idrissi et al. | An unsupervised generative adversarial network based-host intrusion detection system for internet of things devices | |
| Ortet Lopes et al. | Towards effective detection of recent DDoS attacks: A deep learning approach | |
| CN110334105B (en) | Stream data abnormity detection method based on Storm | |
| CN113037783B (en) | Abnormal behavior detection method and system | |
| CN111431819A (en) | Network traffic classification method and device based on serialized protocol flow characteristics | |
| Guo et al. | An ensemble broad learning scheme for semisupervised vehicle type classification | |
| US11934536B2 (en) | Dynamic network risk predicting method based on a graph neural network | |
| CN113591674A (en) | Real-time video stream-oriented edge environment behavior recognition system | |
| CN113344119A (en) | Small sample smoke monitoring method under complex environment of industrial Internet of things | |
| Kamran et al. | Semi-supervised conditional GAN for simultaneous generation and detection of phishing URLs: A game theoretic perspective | |
| Wang et al. | Robust unsupervised network intrusion detection with self-supervised masked context reconstruction | |
| CN112990357B (en) | Black box video countermeasure sample generation method based on sparse disturbance | |
| Zhang et al. | On streaming disaster damage assessment in social sensing: A crowd-driven dynamic neural architecture searching approach | |
| Anwer et al. | Intrusion detection using deep learning | |
| Mahabub et al. | A voting approach of modulation classification for wireless network | |
| CN115952343A (en) | Social robot detection method based on multi-relation graph convolutional network | |
| CN114760104A (en) | Distributed abnormal flow detection method in Internet of things environment | |
| Zhou et al. | IoT unbalanced traffic classification system based on Focal_Attention_LSTM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |