CN112465923A - Underwater bubble image generation method based on condition generation type countermeasure network - Google Patents

Underwater bubble image generation method based on condition generation type countermeasure network Download PDF

Info

Publication number
CN112465923A
CN112465923A CN202011393857.6A CN202011393857A CN112465923A CN 112465923 A CN112465923 A CN 112465923A CN 202011393857 A CN202011393857 A CN 202011393857A CN 112465923 A CN112465923 A CN 112465923A
Authority
CN
China
Prior art keywords
generator
discriminator
layer
condition
activation function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011393857.6A
Other languages
Chinese (zh)
Inventor
杨雪
刘静
郭铁铮
温秀平
陈巍
杨刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Institute of Technology
Original Assignee
Nanjing Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Institute of Technology filed Critical Nanjing Institute of Technology
Priority to CN202011393857.6A priority Critical patent/CN112465923A/en
Publication of CN112465923A publication Critical patent/CN112465923A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to an underwater bubble image generation method based on a condition generation type countermeasure network, which comprises the following steps: constructing a condition generating type confrontation network, wherein the condition generating type confrontation network comprises a generator and a discriminator; using a randomly generated bubble state type label b as a condition factor of a condition generation type countermeasure network, generating random noise z through Gaussian distribution, and simultaneously acquiring real data x; generating synthetic data by taking the type label b and the random noise z as the input of a generator; taking the synthetic data, the real data x and the corresponding type label b as the input of the discriminator; training the generator and the discriminator at the same time, and obtaining the target condition generating type confrontation network when the confrontation between the generator and the discriminator reaches balance. The method can generate a large number of vivid underwater bubble images, and can effectively solve the problems of insufficient underwater target number, small training set scale and influence on state judgment precision.

Description

Underwater bubble image generation method based on condition generation type countermeasure network
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an underwater bubble image generation method based on a condition generation type countermeasure network.
Background
The bubble plume is used as one of gas-liquid two-phase flow and has been widely applied in engineering. In a homogeneous environment, bubble plume formation is subject to three stages: the plume generation area, the plume main body area, the surface flow area and the bubble plume form identification can provide important information for the work of engine underwater exhaust, ship wake tracking, natural gas pipeline fault point investigation and the like.
At present, deep learning has made great progress in the field of target identification, but due to the difficulty of underwater image acquisition, the training set data amount is small, and the deep learning method needs a large amount of sensor data to automatically extract data features, and under the condition that the data set scale is small, an overfitting condition is easy to occur in training, so that the network generalization capability is poor, and the identification accuracy is reduced.
Disclosure of Invention
The invention aims to provide an underwater bubble image generation method based on a condition generation type countermeasure network, which aims to generate sensor data of underwater bubble flow approximate reality in different states, expand the scale of a data set of classification problems of the underwater bubble flow and improve the recognition accuracy of a deep learning algorithm.
The technical scheme adopted by the invention is as follows:
an underwater bubble image generation method based on a condition generation type countermeasure network comprises the following steps:
step one, constructing a condition generating type confrontation network, wherein the condition generating type confrontation network comprises a generator and a discriminator;
step two, taking the randomly generated bubble state type label b as a condition factor of the condition generation type countermeasure network, generating random noise z through Gaussian distribution, and simultaneously acquiring real data x;
thirdly, generating synthetic data by taking the type label b and the random noise z as the input of the generator; taking the synthetic data, the real data x and the corresponding type label b as the input of the discriminator; and simultaneously training the generator and the discriminator, and obtaining a target condition generating type confrontation network when the confrontation between the generator and the discriminator reaches balance.
Further, in step one, the generator includes four two-dimensional convolutional layers, two LSTM layers, and two fully-connected layers.
Further, in the generator, the number of filters in the convolutional layer 1 is 16, the kernel size is 7 × 7, and a ReLU activation function is adopted; the number of filters in convolutional layer 2 is 32, the kernel size is 5 x 5, and a ReLU activation function is adopted; the number of filters in convolutional layer 3 is 64, the kernel size is 3 x 3, and a ReLU activation function is adopted; the number of filters in convolutional layer 4 is 128, the kernel size is 1 × 1, and the ReLU activation function is still used; the dropower of each convolution layer is set to 0.4;
extracting feature vectors from the four convolution layers as input of the LSTM layers, setting units parameters of the two LSTM layers to be 200, and adopting a Hanh activation function; setting the unit parameter of the first full connection layer connected with the LSTM layer as 150, and adopting a ReLU activation function; the second full connection layer units parameter is set to 3, and a Linear activation function is adopted.
Further, in the first step, the discriminator includes two-dimensional convolution layers, a pooling layer, a Flatten layer, and two fully-connected layers.
Further, in the discriminator, the number of filters of the convolution layer 1 is 64, the kernel size is 5 × 5, and a LeakyReLU activation function is adopted; volume base layer 2 has a filter count of 128 and kernel size of 3 x 3, and also employs the LeakyReLU activation function; the dropower of each convolution layer is set to 0.4;
the pooling layer adopts a maximum pooling method, and the poolsize value is set to be 2 x 2; flattening the pooled data by a Flatten layer, and inputting the data into a full connection layer; setting the units parameter of the first full connection layer as 100, and adopting a ReLU activation function; the second full connection layer units parameter is set to 1, and a Sigmoid activation function is adopted.
Further, in step three, the objective function of the generator and the arbiter is:
Figure BDA0002813822430000021
wherein G is a generator and D is a discriminator; minGmaxDV (D, G) represents that the training target of the discriminator is that the value of the function V is maximum, and the training target of the generator G is that the value of the function V is minimum; z is random noise, b is a type label as a condition factor, and x is real data; ex～pdata(x) Represents a slave distribution pdataMiddle sampling x, EZ～pz(z) denotes the distribution pzA middle sample z; in the training process, following the training target of the minimization of the generator objective function and the maximization of the discriminator objective function; minimizing log (1-D (G (z | b))) by adjusting the parameters of G), and minimizing logD (x | b) by adjusting the parameters of D; the process of fighting between the generator and the arbiter continues until equilibrium is reached, at which point the generator will no longer be trained and optimized.
The invention has the beneficial effects that:
when the sensor data are generated for the underwater bubble flow in different states, the same condition generation type countermeasure network can be used, and the time-consuming training process is not required to be repeated for different bubble states. The method can generate a large number of vivid underwater bubble images, and can effectively solve the problems of insufficient underwater target number, small training set scale and influence on state judgment precision.
Drawings
FIG. 1 is a schematic diagram of an underwater bubble image generation method of a conditional generation type countermeasure network (CGAN) in the present invention;
FIG. 2 is a schematic diagram of a generator configuration;
FIG. 3 is a schematic diagram of a discriminator.
Detailed Description
The underwater bubble image generation method based on the condition generating type countermeasure network of the present invention is further described in detail with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, a method for generating an underwater bubble image based on a conditional generation type countermeasure network includes the following steps:
step one, constructing a condition generating type confrontation network, wherein the condition generating type confrontation network comprises a generator and a discriminator.
As shown in fig. 2, the generator includes four two-dimensional convolutional layers, two LSTM layers, and two fully-connected layers. A competing network. In the generator, the number of filters in convolutional layer 1 is 16, the kernel size is 7 × 7, and the ReLU activation function is used. The number of filters in convolutional layer 2 was 32, kernel size was 5 x 5, and the ReLU activation function was used. The number of filters in convolutional layer 3 was 64, kernel size was 3 x 3, and the ReLU activation function was used. The number of filters in convolutional layer 4 is 128, the kernel size is 1 x 1, and the ReLU activation function is still used. The droprate of each convolution layer is set to 0.4.
The feature vectors extracted from the four convolutional layers are used as input to the LSTM layers, the units parameters for both LSTM layers are set to 200, and the Hanh activation function is used. The units parameter of the first fully-connected layer connected to the LSTM layer is set to 150, using the ReLU activation function. The second full connection layer units parameter is set to 3, and a Linear activation function is adopted.
In this embodiment, the generator not only uses CNN as a generation model, but also uses LSTM as a feature learning layer. LSTM is a variant form of RNN with which the temporal properties of time series data can be captured, effectively preventing the problems of gradient dilation and gradient disappearance. The generator generates a series of sensor data by providing a pre-prediction of the input to the following LSTM unit. Thus, the generator can generate time series sensor data that is good enough.
Because the game is continuously played between the discriminator and the generator, the discriminator can influence the generator, so that the classification result is inconsistent with the training result, and the training classification precision cannot be achieved. Therefore, when designing the arbiter, appropriate model parameters need to be set to ensure that the arbiter does not overwhelm the generator. The trainable parameters of the arbiter should be substantially the same as the generator to balance the two models. As shown in fig. 3, the discriminator includes two-dimensional convolutional layers, one pooling layer, one Flatten layer, and two fully-connected layers. In the discriminator, the number of filters of convolutional layer 1 is 64, the kernel size is 5 × 5, and the LeakyReLU activation function is used. Volume base layer 2 has a filter count of 128 and a kernel size of 3 x 3, and also employs the LeakyReLU activation function. The droprate of each convolution layer is set to 0.4.
The pooling layer employs a pooling method of maximum pooling, and sets the poolsize value to 2 x 2. And flattening the pooled data by a Flatten layer, and inputting the data into a full connection layer. The units parameter of the first fully-connected layer is set to 100, and the ReLU activation function is used. The second full connection layer units parameter is set to 1, and a Sigmoid activation function is adopted.
And step two, taking the randomly generated bubble state type label b as a condition factor of the condition generation type countermeasure network, generating random noise z through Gaussian distribution, and acquiring real data x.
The stages where the bubble plume form is located (plume generation region, plume body region and surface flow region) contain a lot of information and thus can be used as a bubble state type label.
And step three, generating the synthetic data by taking the type label b and the random noise z as the input of the generator (because the condition factor is the type label b, the generator can generate the synthetic data according to the type label). The synthesized data and the real data x, and the corresponding type label b are used as the input of the discriminator (the same type label is input into the discriminator together with the data generated by the generator, and likewise, the real data x and the type label thereof are also input into the discriminator together). Training the generator and the discriminator at the same time, and obtaining the target condition generating type confrontation network when the confrontation between the generator and the discriminator reaches balance.
In step three, the objective functions of the generator and the discriminator are standard GAN objective functions:
Figure BDA0002813822430000041
where G is the generator and D is the discriminator. minGmaxDV (D, G) denotes that the training objective of the discriminator is that the function V is maximally valued, while the training objective of the generator G is that the function V is minimally valued. z is random noise, b is a type label as a conditional factor, and x is true data. Ex～pdata(x) Represents a slave distribution pdataMiddle sampling x, EZ～pz(z) denotes the distribution pzAnd z is sampled.
During the training process, the type labels b enable the generator and the arbiter to learn from different activities. The input of the generator is random noise z and type label b, and the input of the discriminator is the generated data and type label b thereof, and may also be real data x and type label b thereof. The generator and the arbiter will be trained simultaneously and follow the training objectives of generator objective function minimization and arbiter objective function maximization. Minimizing log (1-D (G (z | b))) by adjusting the parameters of D, and minimizing logD (x | b) by adjusting the parameters of D. The process of fighting between the generator and the arbiter continues until equilibrium is reached, at which point the generator will no longer be trained and optimized.
The condition generating type countermeasure network constructed by the invention has three input signals: random noise z, type label b and real sensor data x, and the magnitude of two parameters of epoch and batch in the training process is defined by self. The working process of the condition generating type countermeasure network is as follows: the discriminator is first trained by inputting a set of random noise z and type labels b to the generator. The generator will produce composite data which will be input to the arbiter along with the truth data x and type label b. The discriminator makes a judgment on the authenticity of the input data based on the probability of calculating whether the input data is synthetic data or real data x, and the discriminator performs self-adjustment according to its loss function to complete training.
The generator is then trained, and only the synthetic data participates in the training of the generator. In the training process, the generator strives to generate very real synthetic data and updates the generator by using the precision of the trained discriminator, so that the generator can continuously learn and adjust model parameters to minimize a loss function and generate more vivid synthetic data. The entire countermeasure process will be performed according to preset epoch parameters. The condition generating countermeasure network will be trained within a given time until the generator is able to produce high quality sensor data.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any alternative or alternative method that can be easily conceived by those skilled in the art within the technical scope of the present invention should be covered within the scope of the present invention.

Claims (6)

1. An underwater bubble image generation method based on a condition generation type countermeasure network is characterized by comprising the following steps:
step one, constructing a condition generating type confrontation network, wherein the condition generating type confrontation network comprises a generator and a discriminator;
step two, taking the randomly generated bubble state type label b as a condition factor of the condition generation type countermeasure network, generating random noise z through Gaussian distribution, and simultaneously acquiring real data x;
thirdly, generating synthetic data by taking the type label b and the random noise z as the input of the generator; taking the synthetic data, the real data x and the corresponding type label b as the input of the discriminator; and simultaneously training the generator and the discriminator, and obtaining a target condition generating type confrontation network when the confrontation between the generator and the discriminator reaches balance.
2. The underwater bubble image generation method based on the condition generating countermeasure network of claim 1, wherein in step one, the generator comprises four two-dimensional convolution layers, two LSTM layers and two full-connected layers.
3. The method for generating underwater bubble image based on conditional generation countermeasure network according to claim 2, wherein in the generator, the number of filters in the convolution layer 1 is 16, the kernel size is 7 × 7, and the ReLU activation function is adopted; the number of filters in convolutional layer 2 is 32, the kernel size is 5 x 5, and a ReLU activation function is adopted; the number of filters in convolutional layer 3 is 64, the kernel size is 3 x 3, and a ReLU activation function is adopted; the number of filters in convolutional layer 4 is 128, the kernel size is 1 × 1, and the ReLU activation function is still used; the dropower of each convolution layer is set to 0.4;
extracting feature vectors from the four convolution layers as input of the LSTM layers, setting units parameters of the two LSTM layers to be 200, and adopting a Hanh activation function; setting the unit parameter of the first full connection layer connected with the LSTM layer as 150, and adopting a ReLU activation function; the second full connection layer units parameter is set to 3, and a Linear activation function is adopted.
4. The underwater bubble image generation method based on the conditional generation countermeasure network of claim 1, wherein in the first step, the discriminator comprises two-dimensional convolution layers, a pooling layer, a Flatten layer and two fully connected layers.
5. The underwater bubble image generation method based on the conditional generation countermeasure network of claim 4, wherein in the discriminator, the number of filters of convolution layer 1 is 64, the kernel size is 5 × 5, and a LeakyReLU activation function is used; volume base layer 2 has a filter count of 128 and kernel size of 3 x 3, and also employs the LeakyReLU activation function; the dropower of each convolution layer is set to 0.4;
the pooling layer adopts a maximum pooling method, and the poolsize value is set to be 2 x 2; flattening the pooled data by a Flatten layer, and inputting the data into a full connection layer; setting the units parameter of the first full connection layer as 100, and adopting a ReLU activation function; the second full connection layer units parameter is set to 1, and a Sigmoid activation function is adopted.
6. The underwater bubble image generation method based on the conditional generation countermeasure network of any one of claims 1 to 5, wherein in step three, the objective functions of the generator and the discriminator are as follows:
Figure FDA0002813822420000021
wherein G is a generator and D is a discriminator; minGmaxDV (D, G) represents that the training target of the discriminator is that the value of the function V is maximum, and the training target of the generator G is that the value of the function V is minimum; z is random noise, b is a type label as a condition factor, and x is real data; ex～pdata(x) Represents a slave distribution pdataMiddle sampling x, EZ～pz(z) denotes the distribution pzA middle sample z; in the training process, following the training target of the minimization of the generator objective function and the maximization of the discriminator objective function; minimizing log (1-D (G (z | b))) by adjusting the parameters of G), and minimizing logD (x | b) by adjusting the parameters of D; the process of fighting between the generator and the arbiter continues until equilibrium is reached, at which point the generator will no longer be trained and optimized.
CN202011393857.6A 2020-12-03 2020-12-03 Underwater bubble image generation method based on condition generation type countermeasure network Pending CN112465923A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011393857.6A CN112465923A (en) 2020-12-03 2020-12-03 Underwater bubble image generation method based on condition generation type countermeasure network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011393857.6A CN112465923A (en) 2020-12-03 2020-12-03 Underwater bubble image generation method based on condition generation type countermeasure network

Publications (1)

Publication Number Publication Date
CN112465923A true CN112465923A (en) 2021-03-09

Family

ID=74805325

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011393857.6A Pending CN112465923A (en) 2020-12-03 2020-12-03 Underwater bubble image generation method based on condition generation type countermeasure network

Country Status (1)

Country Link
CN (1) CN112465923A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113376516A (en) * 2021-06-07 2021-09-10 科润智能控制股份有限公司 Medium-voltage vacuum circuit breaker operation fault self-diagnosis and early-warning method based on deep learning
CN113298841B (en) * 2021-07-26 2024-01-12 四川大学华西医院 Skin oil parting method, computer equipment, system and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113376516A (en) * 2021-06-07 2021-09-10 科润智能控制股份有限公司 Medium-voltage vacuum circuit breaker operation fault self-diagnosis and early-warning method based on deep learning
CN113298841B (en) * 2021-07-26 2024-01-12 四川大学华西医院 Skin oil parting method, computer equipment, system and storage medium

Similar Documents

Publication Publication Date Title
Yeh et al. Lightweight deep neural network for joint learning of underwater object detection and color conversion
CN110348330B (en) Face pose virtual view generation method based on VAE-ACGAN
Hu et al. A novel image steganography method via deep convolutional generative adversarial networks
CN109543606B (en) Human face recognition method with attention mechanism
CN112052886A (en) Human body action attitude intelligent estimation method and device based on convolutional neural network
CN107529222B (en) WiFi indoor positioning system based on deep learning
CN108615010A (en) Facial expression recognizing method based on the fusion of parallel convolutional neural networks characteristic pattern
Wu et al. Dynamic filtering with large sampling field for convnets
CN111931820B (en) Method for extracting line spectrum of target radiation noise LOFAR spectrogram in water based on convolution residual error network
CN112465923A (en) Underwater bubble image generation method based on condition generation type countermeasure network
CN109325513B (en) Image classification network training method based on massive single-class images
Xu et al. Face expression recognition based on convolutional neural network
EP3671635B1 (en) Curvilinear object segmentation with noise priors
Yu et al. Exemplar-based recursive instance segmentation with application to plant image analysis
Li et al. Findnet: Can you find me? boundary-and-texture enhancement network for camouflaged object detection
CN112214630A (en) Clothing image retrieval system and method based on expansion convolution residual error network
CN111144497B (en) Image significance prediction method under multitasking depth network based on aesthetic analysis
Lai et al. Accelerated guided sampling for multistructure model fitting
CN115294424A (en) Sample data enhancement method based on generation countermeasure network
CN112070009B (en) Convolutional neural network expression recognition method based on improved LBP operator
Xiong et al. Artificial reef detection and recognition based on Faster-RCNN
CN111507185B (en) Tumble detection method based on stack cavity convolution network
Zhao et al. Facial expression recognition based on visual transformers and local attention features network
CN114140495A (en) Single target tracking method based on multi-scale Transformer
CN113255666A (en) Personalized question answering system and method based on computer vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination