CN112884758B - Defect insulator sample generation method and system based on style migration method - Google Patents
Defect insulator sample generation method and system based on style migration method Download PDFInfo
- Publication number
- CN112884758B CN112884758B CN202110270530.8A CN202110270530A CN112884758B CN 112884758 B CN112884758 B CN 112884758B CN 202110270530 A CN202110270530 A CN 202110270530A CN 112884758 B CN112884758 B CN 112884758B
- Authority
- CN
- China
- Prior art keywords
- image
- sample
- style
- insulator
- style migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000012212 insulator Substances 0.000 title claims abstract description 135
- 230000005012 migration Effects 0.000 title claims abstract description 103
- 238000013508 migration Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000007547 defect Effects 0.000 title description 42
- 230000002950 deficient Effects 0.000 claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000000007 visual effect Effects 0.000 claims abstract description 7
- 230000011218 segmentation Effects 0.000 claims description 20
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 15
- 238000010606 normalization Methods 0.000 claims description 15
- 230000000694 effects Effects 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 abstract description 9
- 238000001514 detection method Methods 0.000 abstract description 9
- 238000005457 optimization Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004660 morphological change Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention discloses a defective insulator sample generation method and a defective insulator sample generation system based on a style migration method.A plurality of acquired insulator image samples are divided into a plurality of image domains according to visual difference, each image domain is coded, and then the style migration training is carried out on the image domains through a style migration network to obtain a style migrator between any two image domains; finally, carrying out style migration on the defective insulator sample in the image domain by using the obtained style migration device to generate a new defective insulator sample; according to the method, the style migration device is used for carrying out style migration on the defective insulator sample in the image domain to generate a new more vivid style migration image sample, the generated defective insulator image sample is high in quality, semantic connection information of the insulator sample is reserved, the generated defective insulator sample can effectively provide accuracy and recall rate of a target detection model based on deep learning, and the method has certain practical value.
Description
Technical Field
The invention relates to the technical field, in particular to a defective insulator sample generation method and system based on a style migration method.
Background
In recent years, in the intelligent routing inspection work of an electric power system, a national power grid tries to introduce a novel artificial intelligence technology to solve the labor-intensive problem in the routing inspection work, for example, an intelligent routing inspection target is realized through a target detection algorithm based on deep learning, but because the occurrence frequency of partial fault defects in the electric power system is low, collectable defect samples are few, and the requirement of deep learning on data volume is difficult to meet, a method for generating the defect samples is urgently needed to solve the problem.
For a defective insulator sample, the current available data volume still cannot meet the requirement of neural network training, so that the detection accuracy of the defective insulator is low in the actual verification work of the artificial intelligence technology, and the actual application is difficult. Aiming at the problem of lack of insulator defect samples, the following two data enhanced sample expansion technologies are mainly used, namely a transformation technology and a generation technology.
The transformation type generation method emphasizes that data enhancement is realized by using traditional methods such as mirror image inversion and affine transformation, some methods realize data enhancement by fusing a defective insulator with a new background through ps software, and some methods realize data enhancement by adding noise to a fused image.
The generation method is most representative of generation of a countermeasure network (Gan), the Gan is created according to the idea of zero sum game, the generator and the discriminator form the generation method, and real samples can be generated through game training of the generator and the discriminator to realize sample expansion. The generation method based on Gan is roughly divided into the following two methods, one is a method for generating a real sample through an implicit variable, and the method is characterized in that a sample which is in accordance with expected distribution is generated from an implicit space through the confrontation training of a generator and a discriminator, but the generation method depends on a large amount of original sample training and is not suitable for insulator data with a small sample amount per se; secondly, a method for realizing sample expansion through style migration is characterized in that a defect sample of a current style is used as input to generate sample data of multiple different styles, the method is less in sample dependence amount compared with a hidden space generation method, a Cycle-Gan-based method generates defect insulator samples under multiple environmental styles, although the style migration method can realize data enhancement of the insulator samples to a certain extent, the existing algorithm often has the following problems:
1. the defect sample generation ratio is insufficient;
2. the generation quality of the defect sample is poor;
3. the defect samples cannot be used for deep learning target detection model training.
Disclosure of Invention
In order to solve the technical problems, the invention provides a defective insulator sample generation method and system based on a style migration method, based on how to generate a large number of insulator defect samples which are high in quality and can be used for deep learning target detection model training on the basis of the existing small sample defect insulator data.
The invention is realized by the following technical scheme:
the scheme provides a defective insulator sample generation method based on a style migration method, which comprises the following steps:
s1, collecting a plurality of insulator image samples;
s2, dividing the insulator image sample into a plurality of image domains according to visual difference and coding each image domain;
s3, performing style migration training on the image domains through a style migration network to obtain a style migrator between any two image domains;
and S4, carrying out style migration on the defect insulator sample in the image domain by using the style migrator obtained in the S3 to generate a new defect insulator sample.
The working principle of the scheme is as follows: the method for generating the defect image sample in the prior art needs to rely on a large amount of original sample training and is not suitable for insulator data with less sample amount per se, and a large amount of defect insulator image samples with different styles can be generated based on the acquisition of a small amount of sample defect insulator image samples in a plurality of insulator image samples, so that one-to-many generation can be realized, and the generation conversion ratio is high; the method for realizing sample expansion through style migration is characterized in that a defect sample of a current style is used as input to generate sample data of a plurality of different styles, and the method has less sample dependence on the existing generation method; according to the scheme, the style migration of the defective insulator samples in the image domain is performed through the style migration device, the new vivid style migration image samples are generated, the generated defective insulator image samples are high in quality, semantic connection information of the insulator samples is kept, the generated defective insulator samples can effectively provide accuracy and recall rate of a target detection model based on deep learning, and the method has certain practical value.
The further optimization scheme is that S3 comprises the following substeps:
s31, arbitrarily selecting two image domains, and respectively taking an insulator image sample from each of the two image domains as a source domain image sample and a target domain image sample;
s32, extracting an insulator region segmentation mask image of the source region image sample;
s33, establishing a style migration network Tw _ Cycle Gan based on the insulator region segmentation mask map and the target domain image sample to realize conversion from a source domain to a target domain;
s34, using the style migration network Tw _ Cycle Gan as a style migrator between two image domains.
The further optimization scheme is that S32 comprises the following substeps:
s321, carrying out image annotation on an insulator region in a source domain image sample through labelme software; (including marking of the complete insulator image and marking of the partial defective insulator image)
S322, establishing a U _ net region segmentation network, and realizing insulator image pixel level classification of the image-labeled source region image sample so as to obtain an insulator region segmentation mask image.
The scheme uses a U _ net segmentation network, can realize classification at a pixel level, and is still applicable to small sample amount data. Part of insulator image samples are labeled through labelme, and since the image segmentation network classifies the pixel levels, as for the pixels, the defect samples and the non-defect samples have almost no difference, the trained segmentation network can simultaneously extract segmentation mask images of the defect samples and the non-defect samples.
In a further optimization scheme, S33 includes the following substeps:
establishing a generation network consisting of five downsampling residual blocks, four upsampling residual blocks and five middle residual blocks, and enabling the generation network to realize a style migration effect by using example normalization and self-adaptive example normalization;
establishing a discrimination network, wherein the discrimination network is used for discriminating the truth of the generated image sample and judging the image domain to which the generated image sample belongs; the discrimination network is composed of a multitask convolution network, k layers which are completely connected are used for carrying out true and false classification on each domain, wherein k represents the number of the domains, and the discrimination network can judge the image domain to which the generated image sample belongs while discriminating the true and false of the generated image sample;
establishing a style coding network with the same architecture as the discrimination network, wherein the style coding network is used for extracting style codes of the target domain image samples; the style coding network and the discrimination network use the same architecture and are also provided with k full-connection layers, each domain is provided with the own full-connection layer, and the style coding network receives the insulator image sample and extracts the style coding of the target domain image sample;
establishing a mapping network; the mapping network consists of k output multilayer perceptrons, wherein k represents the number of domains, all the domains share four full-connection layers, and each domain consists of four specific full-connection layers;
style migration loss and local constraint loss are established based on the segmentation mask map.
The style belongs to global information and contains certain content information, and due to the lack of constraints on local insulators and defects, defect characteristics can be damaged due to style migration of a defect sample using a non-defect sample as a reference, and the following loss functions are added to strengthen the constraints on the local characteristics in order to ensure that the local defect characteristics and the insulators are kept unchanged as much as possible in the style migration process: a style migration loss and a local constraint loss are established based on the segmentation mask map.
The further optimization scheme is that the local constraint loss comprises a target cyclic consistency loss and a target mask loss; the style migration loss includes a generation confrontation loss, a style reconstruction loss, a style diversity loss, and a cycle consistency loss.
The further optimization scheme is that the sample normalization is used for realizing the style migration effect on the downsampling residual block of the generated network, and the self-adaptive sample normalization is used for realizing the style migration effect on the upsampling residual block of the generated network.
Target cyclic consistent loss L w_cyc The model is as follows:
given an insulator image X, which belongs to an insulator image domain set X, namely X belongs to X, and X belongs to an original domain Y, and a domain set Y (0, 1, · to) to which Y belongs, namely Y belongs to Y,
for target domain coding, Z is a randomly generated Gaussian distribution hidden variable belonging to a Gaussian distribution hidden variable set Z, namely Z belongs to Z, mask is a segmentation mask graph of an input insulator image x,
is a target domain
Is coded in accordance with one of the styles of,
for a style code of the original field y, the generator G generates x as a target field style code
Output ofInsulator image
The generator G will in turn generate
Is generated as a source domain
Lower cycle uniform output insulator image
Computing input image x and cycle consistent output
The difference between the two and the mask is expected after the product of the corresponding elements of the matrix is made
L1 gap is as small as possible, | | \ | Limu 1 To calculate the L1 norm.
Target mask penalty L M The model is as follows:
generator G generates x as a target domain style code
Lower output insulator image
Computing input image x and output
The difference between the two and the mask is expected after the product of the corresponding elements of the matrix is made
L1 is in fullThe amount is small.
The further optimization scheme is that S4 comprises the following substeps:
s41, selecting a defective insulator image sample as a source image input, and taking a non-defective insulator image sample as a reference style image sample;
and S42, extracting style codes of the non-defective insulator image samples by the style migration device, and performing style migration on the source image to obtain a new defective insulator image sample.
According to the defective insulator sample generation method based on the style migration method, the defective insulator sample generation system based on the style migration method is provided, and comprises the following steps:
the image acquisition module is used for acquiring a plurality of insulator image samples;
the dividing module is used for dividing the insulator image sample into a plurality of image domains according to visual difference and coding each image domain;
the style migration network module is used for carrying out style migration training on the image domains through a style migration network to obtain a style migrator between any two image domains;
and the sample generation module performs style migration on the defective insulator sample in the image domain by using the style migrator obtained by the style migration network module to generate a new defective insulator sample.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. according to the method and the system for generating the defective insulator sample based on the style migration method, a large number of defective insulator image samples can be generated based on the small sample defective insulator image sample, the defective insulator image samples can be generated in a one-to-many mode, and the generation conversion ratio is high.
2. According to the method and the system for generating the defective insulator sample based on the style migration method, the style migration device establishes the style migration loss and the local constraint loss in the style migration process based on the segmentation mask image, and the image generated by the generator is more vivid through game training of the generator and the discriminator and generation of constraint for resisting the loss; enhancing the style extraction capability of the mapping network and the coding network through style diversity loss and style reconstruction loss constraint; reducing changes of global content information through cyclic consistent loss constraints; semantic connection information of the insulator sample is reserved, and the generated defect sample is high in quality.
3. According to the method and the system for generating the defective insulator sample based on the style migration method, the generated defective insulator sample can effectively provide the accuracy and the recall rate of the target detection model based on deep learning, and certain practical value is achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention.
FIG. 1 is a schematic flow chart of a defective insulator sample generation method based on a style migration method;
FIG. 2 is a schematic diagram of an insulator style translator configuration;
fig. 3 is a mask calculation flow diagram.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and the accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not used as limiting the present invention.
Example 1
The embodiment provides a defective insulator sample generation method based on a style migration method, which specifically comprises the following steps:
s1: collecting a plurality of insulator image samples;
s2: the insulator image sample is divided into a plurality of image domains according to the visual difference and each image domain is encoded.
The specific implementation process comprises the following steps of firstly collecting defective insulator samples and non-defective insulator samples on a power line, manually dividing the samples into two different image domains based on visual distinction, wherein the image domains comprise a yellow-style image domain and a green-style image domain, and the styles belong to global information characteristics, so that the defective samples and the non-defective samples only have local defect differences, and the style migration tasks have small difference.
Each image domain is encoded using (0, 1,. Cndot.) with 0,1 representing the first image domain and the second image domain in turn.
S3: and performing style migration training on the divided image domains by a style migration method to obtain a style migrator between any two image domains.
In this embodiment, we perform style migration training on two divided image domains, but the method is still applicable to style migration between multiple domains.
As shown in fig. 2, the style migration network comprises four components: generating a network, judging the network, coding the network and mapping the network.
TABLE 1
Establishing a generating network: as shown in table 1, the generation network is composed of five downsampling residual blocks, four upsampling residual blocks, and five intermediate residual blocks, and for the downsampling residual blocks and the upsampling residual blocks, the example normalization and the adaptive example normalization are respectively used to realize the style migration effect, and an example normalization formula is as follows:
for the input tensor:
the example normalization normalizes the feature map mean value and standard deviation of each channel, wherein gamma and beta are learnable affine parameters, and mu (x) and sigma (x) respectively represent the mean value and the standard deviation and are obtained by calculation on the space dimension and are independent of the channel and the batch.
The adaptive example normalization formula is as follows:
different from example normalization, adaptive example normalization does not need training parameters, and only needs to receive source image input x and style input y and match the channel-level mean value and standard deviation of x to the channel-level mean value and standard deviation of y.
TABLE 2
Establishing a discrimination network: as shown in table 2, the discrimination network is composed of a multi-task convolutional network, and performs true and false classification on each domain by using K fully-connected layers, where K represents the number of domains and D is an output dimension, and the discrimination network can also determine an image domain to which a generated image sample belongs while discriminating whether the generated image sample is true or false.
Establishing a coding network: as shown in table 2, the style coding network and the discrimination network use the same architecture, and also have K fully-connected layers, each domain has its own fully-connected layer, and the style coding network receives the insulator image sample, extracts the style code of the input target insulator image sample, and in this embodiment, 512-dimensional style coding is used.
TABLE 3
Establishing a mapping network: as shown in table 3, the mapping network consists of K output multi-layer perceptrons, where K represents the number of domains, all domains share four fully-connected layers, each domain consists of four specific fully-connected layers, and the output is 512-dimensional style coding.
After the network model is established through the steps, further, in order to realize the style migration target, a loss function needs to be set and constrained to realize the style migration effect, and the style migration loss is initially established as follows:
generating the confrontation loss:
in the following formulas, G, F, E, and D represent a generation network, a discrimination network, an encoding network, and a mapping network, respectively.
Giving an insulator image X belonging to X, an original domain Y belonging to Y, and Y being a domain code, wherein the original domain Y belonging to X represents different domains, a Gaussian distribution hidden variable Z belonging to Z is randomly generated, and a target domain is generated by a mapping network
Down style coding
Inputting the insulator image and the insulator image x to obtain a generated image sample
The authenticity of the generated image sample is judged by judging the network D.
Loss of style reconstruction:
generation of style codes from a mapping network F
And giving an insulator image x to obtain an output insulator image
The insulator image is produced by the stylistic coding network ECode, calculating the difference between the two stylistic encodings, which allows the G-synthesis to reflect the style of the reference image x
Is used to output an insulator image.
Loss of style diversity:
implicit variable z is mapped by mapping network F 1 、z 2 Generating
And calculating the difference between the generated samples, wherein the difference between the generated samples is expected to be as large as possible in order to ensure the diversity, so that when the loss is calculated finally, the loss weight coefficient of the style diversity is set as a negative sign.
Loss of cycle consistency:
the x is transmitted by the generating network G,
input derived styles
Lower insulator image sample
Then extracting input x style codes through style coding network E
Generating through a generating network G to obtain an input x style code
Underlying insulator image sample
Ensuring that this output is as consistent as possible with the original input x.
Through the constraint of the preliminary loss function, the insulator image style migration target can be realized, as shown in fig. 2, the input is source, the reference image and the target domain style code s are extracted through a coding network or a mapping network and input into a generation network to obtain an output, and a more vivid style migration image sample is obtained through game training of the generation network and a discrimination network. However, the style belongs to global information and includes certain content information, and due to lack of constraints on local insulators and defects, defect feature is damaged due to style migration of a defect sample using a non-defect sample as a reference, and in order to ensure that the local defect feature and the insulator themselves remain unchanged as much as possible in the style migration process, as shown in fig. 3, the following loss function is added to strengthen the constraints on the local feature:
target cycle consistent loss:
where mask splits the mask map for input insulator picture x,
still outputting insulator samples for cycle consistency, and calculating the target mask loss after the insulator samples and the mask are respectively subjected to matrix corresponding element product:
where mask splits the mask map for input insulator image x,
is a target style
And calculating the difference between the output result and the mask after the product of the matrix element and the output result is respectively carried out on the output result and the mask.
Network partitioning: the mask image of the insulator target object needs to be extracted before calculation, and the method and the device can realize pixel-level classification by using the U _ net to divide the network, and are still applicable to small sample data. Part of insulator image samples are marked through labelme, and since the image segmentation network classifies the pixel levels, regarding pixels, the defect samples and the non-defect samples have almost no difference, so that the trained segmentation network can simultaneously extract mask images of the defect samples and the non-defect samples.
Training details: the target cycle always loss and target mask loss weights are set to 4, the remaining loss weights are set to 1, the style migration network uses an Adam optimizer, beta 1 、β 2 Setting the learning rate of the generation network, the discrimination network and the coding network as 0.0001, setting the learning rate of the mapping network as 0.000001, operating the image segmentation network and the style migration network on different GPUs, setting the image input/output size as 512 x 512, requiring more than 11G of video memory for the style migration network, and setting the time for 80000 iterations of the style migration network to be about three days.
Style migration workflow: as shown in fig. 2, a source domain image source and a target domain image reference are obtained each time of iteration, a target domain image or style coding s under a target domain can be obtained through a coding network level mapping network, and an image generated by a generator is more vivid through game training of the generator and a discriminator and generation of constraint of resistance loss; enhancing the style extraction capability of a mapping network and a coding network through style diversity loss and style reconstruction loss constraint; reducing changes of global content information through cyclic consistent loss constraints; in order to further enhance the local constraint of the insulator, as shown in fig. 3, a segmentation mask map of the insulator sub-target region is extracted, and the local insulator is constrained by the target cyclic coherence loss and the target mask loss, so that the generated defective insulator can reduce the morphological change.
S4: and performing style migration on the defect insulator sample in one domain according to the reference insulator samples in other domains by using the style migration device to realize sample amplification.
As shown in fig. 2, a certain stylistic defect image is used as an input, a stylistic code of another reference image is extracted by using a coding network, and the two images are input into a generation network to obtain a reference image stylistic defect image. By expanding the generated sample into the data based on the deep learning target detection model, the performance of the deep learning target detection model can be effectively improved by carrying out training experiments, and the defect insulator sample generated by using the method has certain practical value.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (6)
1. A defective insulator sample generation method based on a style migration method is characterized by comprising the following steps:
s1, collecting a plurality of insulator image samples;
s2, dividing the insulator image sample into a plurality of image domains according to visual difference and coding each image domain;
s3, performing style migration training on the image domains through a style migration network to obtain a style migration device between any two image domains;
s4, carrying out style migration on the defective insulator sample in the image domain by using the style migration device obtained in the S3 to generate a new defective insulator sample;
wherein, S3 comprises the following substeps:
s31, arbitrarily selecting two image domains, and respectively taking an insulator image sample from each of the two image domains as a source domain image sample and a target domain image sample;
s32, extracting an insulator region segmentation mask image of the source region image sample;
s33, establishing a style migration network Tw _ Cycle Gan based on the insulator region segmentation mask map and the target domain image sample to realize conversion from a source domain to a target domain;
s34, taking a style migration network Tw _ Cycle Gan as a style migration device between two image domains;
s4 comprises the following substeps:
s41, selecting a defective insulator image sample as a source image input, and taking a non-defective insulator image sample as a reference style image sample;
and S42, extracting style codes of the non-defective insulator image samples by the style migration device, and performing style migration on the source image to obtain a new defective insulator image sample.
2. The defective insulator sample generation method based on the style migration method as claimed in claim 1, wherein S32 comprises the following substeps:
s321, carrying out image annotation on an insulator region in a source region image sample through labelme software;
s322, establishing a U _ net region segmentation network, and realizing insulator image pixel level classification of the image-labeled source region image sample so as to obtain an insulator region segmentation mask image.
3. The defective insulator sample generation method based on the style migration method as claimed in claim 1, wherein S33 comprises the following substeps:
establishing a generation network consisting of five downsampling residual blocks, four upsampling residual blocks and five middle residual blocks, and enabling the generation network to realize a style migration effect by using example normalization and self-adaptive example normalization;
establishing a discrimination network, wherein the discrimination network is used for discriminating the authenticity of the generated image sample and judging the image domain to which the generated image sample belongs;
establishing a style coding network with the same architecture as the discrimination network, wherein the style coding network is used for extracting style codes of the target domain image samples;
establishing a mapping network;
style migration loss and local constraint loss are established based on the segmentation mask map.
4. The defective insulator sample generation method based on the style migration method according to claim 3, wherein the local constraint loss comprises a target cyclic consistency loss and a target mask loss; the style migration loss includes a generation confrontation loss, a style reconstruction loss, a style diversity loss, and a cycle consistency loss.
5. The defective insulator sample generation method based on the style migration method of claim 3, wherein the style migration effect is achieved on the downsampling residual block of the generation network by using instance normalization, and the style migration effect is achieved on the upsampling residual block of the generation network by using adaptive instance normalization.
6. A defective insulator sample generation system based on a style migration method, which is applied to any one of claims 1 to 5, and is characterized by comprising:
the image acquisition module is used for acquiring a plurality of insulator image samples;
the dividing module is used for dividing the insulator image sample into a plurality of image domains according to visual difference and coding each image domain;
the style migration network module is used for carrying out style migration training on the image domains through a style migration network to obtain a style migrator between any two image domains;
the sample generation module performs style migration on the defective insulator sample in the image domain by using a style migration device obtained by the style migration network module to generate a new defective insulator sample;
the style migration network module is used for executing the following processes:
randomly selecting two image domains, and respectively taking an insulator image sample from each of the two image domains as a source domain image sample and a target domain image sample;
extracting an insulator region segmentation mask map of a source domain image sample;
establishing a style migration network Tw _ Cycle Gan based on an insulator region segmentation mask map and a target domain image sample to realize conversion from a source domain to a target domain;
taking a style migration network Tw _ Cycle Gan as a style migrator between two image domains;
the style migration network module is used for executing the following processes:
selecting a defective insulator image sample as a source image input, and taking a non-defective insulator image sample as a reference style image sample; and the style migration device extracts style codes of the non-defective insulator image samples to perform style migration on the source graph to obtain new defective insulator image samples.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110270530.8A CN112884758B (en) | 2021-03-12 | 2021-03-12 | Defect insulator sample generation method and system based on style migration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110270530.8A CN112884758B (en) | 2021-03-12 | 2021-03-12 | Defect insulator sample generation method and system based on style migration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112884758A CN112884758A (en) | 2021-06-01 |
| CN112884758B true CN112884758B (en) | 2023-01-10 |
Family
ID=76041648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110270530.8A Active CN112884758B (en) | 2021-03-12 | 2021-03-12 | Defect insulator sample generation method and system based on style migration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112884758B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114332086B (en) * | 2022-03-14 | 2022-05-13 | 启东市固德防水布有限公司 | Textile defect detection method and system based on style migration and artificial intelligence |
| CN114418919B (en) * | 2022-03-25 | 2022-07-26 | 北京大甜绵白糖科技有限公司 | Image fusion method and device, electronic equipment and storage medium |
| CN116664839B (en) * | 2023-05-31 | 2024-02-13 | 博衍科技(珠海)有限公司 | Weak supervision semi-supervision defect perception segmentation method and device and electronic equipment |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767328B (en) * | 2017-10-13 | 2021-12-17 | 上海媒智科技有限公司 | Migration method and system of any style and content generated based on small amount of samples |
| US11199506B2 (en) * | 2018-02-21 | 2021-12-14 | Applied Materials Israel Ltd. | Generating a training set usable for examination of a semiconductor specimen |
| CN109117863B (en) * | 2018-07-04 | 2021-08-13 | 全球能源互联网研究院有限公司 | Insulator sample expansion method and device based on deep convolution generation countermeasure network |
| CN109961491B (en) * | 2019-04-12 | 2023-05-26 | 上海联影医疗科技股份有限公司 | Multi-mode image truncation compensation method, device, computer equipment and medium |
| CN110222757A (en) * | 2019-05-31 | 2019-09-10 | 华北电力大学（保定） | Based on insulator image pattern extending method, the system for generating confrontation network |
| CN111291885B (en) * | 2020-01-20 | 2023-06-09 | 北京百度网讯科技有限公司 | Near infrared image generation method, training method and device for generation network |
| CN111598842A (en) * | 2020-04-24 | 2020-08-28 | 云南电网有限责任公司电力科学研究院 | Method and system for generating model of insulator defect sample and storage medium |
| CN111783525B (en) * | 2020-05-20 | 2022-10-18 | 中国人民解放军93114部队 | Aerial photographic image target sample generation method based on style migration |
| CN111681162B (en) * | 2020-06-09 | 2023-09-01 | 创新奇智(成都)科技有限公司 | Defect sample generation method and device, electronic equipment and storage medium |
| CN111932438A (en) * | 2020-06-18 | 2020-11-13 | 浙江大华技术股份有限公司 | Image style migration method, equipment and storage device |
-
2021
- 2021-03-12 CN CN202110270530.8A patent/CN112884758B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112884758A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Golts et al. | Unsupervised single image dehazing using dark channel prior loss | |
| CN110310221B (en) | Multi-domain image style migration method based on generation countermeasure network | |
| Chen et al. | The face image super-resolution algorithm based on combined representation learning | |
| CN112884758B (en) | Defect insulator sample generation method and system based on style migration method | |
| CN111292264B (en) | Image high dynamic range reconstruction method based on deep learning | |
| CN108875935B (en) | Natural image target material visual characteristic mapping method based on generation countermeasure network | |
| CN112396607B (en) | Deformable convolution fusion enhanced street view image semantic segmentation method | |
| CN110889449A (en) | Edge-enhanced multi-scale remote sensing image building semantic feature extraction method | |
| CN112036260A (en) | Expression recognition method and system for multi-scale sub-block aggregation in natural environment | |
| CN115240201B (en) | Chinese character generation method for alleviating network mode collapse problem by using Chinese character skeleton information | |
| Zhou et al. | FSAD-Net: Feedback spatial attention dehazing network | |
| CN114820871A (en) | Font generation method, model training method, device, equipment and medium | |
| CN114723950A (en) | Cross-modal medical image segmentation method based on symmetric adaptive network | |
| Zheng et al. | T-net: Deep stacked scale-iteration network for image dehazing | |
| CN112381082A (en) | Table structure reconstruction method based on deep learning | |
| CN114359269A (en) | Virtual food box defect generation method and system based on neural network | |
| Zhou et al. | MSAR‐DefogNet: Lightweight cloud removal network for high resolution remote sensing images based on multi scale convolution | |
| CN112784831A (en) | Character recognition method for enhancing attention mechanism by fusing multilayer features | |
| Yu et al. | MagConv: Mask-guided convolution for image inpainting | |
| CN113962332A (en) | Salient target identification method based on self-optimization fusion feedback | |
| CN114581789A (en) | Hyperspectral image classification method and system | |
| CN114529450A (en) | Face image super-resolution method based on improved depth iterative cooperative network | |
| CN114494387A (en) | Data set network generation model and fog map generation method | |
| Feng et al. | Coal mine image dust and fog clearing algorithm based on deep learning network | |
| Lei et al. | MHANet: A multiscale hierarchical pansharpening method with adaptive optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |