TWI803243B - Method for expanding images, computer device and storage medium - Google Patents

Abstract

The present application provides a method for expanding images, a computer device and a storage medium. The method includes obtaining an image to be expanded and obtaining a plurality of test images, the plurality of test images including a plurality of gas leaked images; constructing a variational learner and a discriminator based on a convolutional neural network, obtaining a plurality of target images by inputting the plurality of gas leaked images into the variational learner; generating a variational autoencoder model based on the variational learner and a result that acquired from the discriminator by processing the plurality of target images; calculating a reconstruction accuracy rate of the variational autoencoder model according to the plurality of test images. In response that the reconstruction accuracy rate is less than a preset threshold, the method obtains an expansion model by adjusting the variational autoencoder model using the plurality of gas leaked images, and obtains expanded images by putting the image to be expanded into the expansion model. This application can improve a reconstruction accuracy and image clarity of the expanded images.

Description

Image amplification method, computer equipment and storage medium

The present application relates to the field of image processing, in particular to an image augmentation method, computer equipment and storage media.

In the current image augmentation method, the size of the input image needs to be considered when using the variational autoencoder, and the reconstructed image is relatively blurred, resulting in low reconstruction accuracy. Therefore, under the premise of not considering the size of the input image, how to construct a clear augmented image that can be accurately reconstructed has become an urgent problem to be solved.

In view of the above, it is necessary to provide an image augmentation method, computer equipment and storage medium, which can accurately reconstruct a clear augmented image.

The present application provides an image augmentation method. The image augmentation method includes: acquiring an image to be augmented and a test image, wherein the test image includes a gas leakage image; Build a variational learner and a discriminator; input the gas leakage image into the variational learner to obtain a target image; train the target image according to the discriminant result of the discriminator A variational learner to obtain a variational autoencoder model; calculate the reconstruction accuracy of the variational autoencoder model based on the test image; If the reconstruction accuracy is less than a preset threshold, adjust the variational autoencoder model based on the gas leakage image to obtain an amplification model; input the image to be amplified into the amplification model In , the enlarged image is obtained.

According to an optional embodiment of the present application, the training of the variational learner according to the discriminant result of the discriminator on the target image to obtain the variational autoencoder model includes: inputting the target image into the In the discriminator, obtain the discriminant probability that the discriminator determines the target image as a false image; re-input the target image corresponding to the discriminant probability greater than or equal to the first preset value into the variation Train in the learner to obtain the first image; calculate the loss value of the variational learner based on the gas leakage image, the target image and the first image, and use gradient backpropagation Updating the weight value of the variational learner until the loss value drops to a minimum, and obtaining the variational autoencoder model.

其中，loss為所述損失值，M是指所述目標圖像中所有畫素點的數量，N是指所述氣體外洩圖像中所有畫素點的數量，K是指所述第一圖像中所有畫素點的數量，i是指所述目標圖像中第i個畫素點，j是指所述氣體外洩圖像中與i對應的畫素點，r是指所述第一圖像中與i對應的畫素點，y _i 是指所述目標圖像中第i個畫素點的畫素值，x _j 是指所述氣體外洩圖像中第j個畫素點的畫素值，z _r 是指所述第一圖像中第r個畫素點的畫素值。 According to an optional embodiment of the present application, the calculation of the loss value of the variational learner based on the gas leakage image, the target image and the first image includes: a calculation method of the loss value for:

Wherein, loss is the loss value, M refers to the number of all pixels in the target image, N refers to the number of all pixels in the gas leakage image, and K refers to the first The number of all pixels in the image, i refers to the i- th pixel in the target image, j refers to the pixel corresponding to i in the gas leakage image, r refers to the The pixel point corresponding to i in the first image, y _i refers to the pixel value of the i- th pixel point in the target image, and x _j refers to the j -th pixel point in the gas leakage image The pixel value of the pixel point, z _r refers to the pixel value of the rth pixel point in the first image.

According to an optional embodiment of the present application, the calculating the reconstruction accuracy rate of the variational autoencoder model based on the test image includes: acquiring an annotation result of the test image; The test image is input into the variational self-encoder model to obtain a feature image; the similarity value between the feature image and the test image is calculated; the similarity value is compared with the second prediction Comparing the set values to obtain the verification result of the test image; comparing the verification result with the labeling result; determining the test image corresponding to the same verification result as the labeling result as the second image image, and determine the feature image corresponding to the second image as a similar image; calculate the ratio of the similar image in the feature image, and determine the ratio as the weight construction accuracy.

c ₁=(K ₁ L)²；c ₂=(K ₂ L)²；其中，SSIM(x,y)為所述相似值，x為所述第三圖像，y為所述第四圖像，μ _x 為所述第三圖像的灰度平均值，μ _y 為所述第四圖像的灰度平均值，σ _x 為所述第三圖像的灰度標準差，σ _y 為所述第四圖像的灰度標準差，σ _xy 為所述第三圖像與所述第四圖像之間的灰度協方差，c ₁及c ₂均為預設參數，L為所述第四圖像中最大的畫素值，K ₁及K ₂是預先設置的常數，且K ₁≪1，K ₂≪1。 According to an optional embodiment of the present application, the calculating the similarity value between the feature image and the test image includes: performing grayscale processing on the feature image to obtain a grayscale image; The grayscaled image is subjected to binarization processing to obtain a third image; the test image corresponding to the feature image is subjected to grayscale processing and binarization processing to obtain a fourth image; the calculation of the The similarity value of the third image and the fourth image, the determination formula of the similarity value is:

c ₁ =( K ₁ L ) ² ; c ₂ =( K ₂ L ) ² ; wherein, SSIM ( x,y ) is the similarity value, x is the third image, and y is the fourth image Like, μ _x is the gray average value of the third image, μ _y is the gray average value of the fourth image, σ _x is the gray standard deviation of the third image, σ _y is The grayscale standard deviation of the fourth image, σxy is the _grayscale covariance between _the third image and the fourth image, c1 _and c2 are preset parameters, and L is the The largest pixel value in the fourth image, K ₁ and K ₂ are preset constants, and K ₁ ≪1, K ₂ ≪1.

According to an optional embodiment of the present application, the adjusting the variational autoencoder model based on the gas leakage image to obtain an augmented model includes: inputting the gas leakage image into the variational autoencoder The machine model is trained until the reconstruction accuracy rate is greater than or equal to the preset threshold to obtain the augmented model.

According to an optional embodiment of the present application, the augmented model includes an encoder and a decoder, a fully convolutional neural network is used in the encoder, and the fully convolutional neural network includes multiple hidden layers. A deconvolution neural network is used in the decoder, and the deconvolution neural network includes multiple operation layers.

According to an optional embodiment of the present application, the inputting the image to be amplified into the augmentation model to obtain the augmented image includes: inputting the image to be amplified into the hidden image of the encoder Feature extraction is performed in the layer to obtain a feature vector, wherein there are 2 n elements in the feature vector; the first n elements in the feature vector are extracted as the mean vector; the last n elements in the feature vector are extracted as standard deviation vector; generate Gaussian random numbers according to the mean vector and the standard deviation vector; randomly sample the Gaussian random numbers to obtain sampling values; compare each element in the mean vector with the sampling values A multiplication operation to obtain multiple multiplication results; adding each multiplication result to the corresponding element in the standard deviation vector to obtain a latent vector; inputting the latent variable to the operation layer of the decoder Perform mapping processing to obtain the amplified image.

The present application provides a computer device, and the computer device includes: a memory storing at least one instruction; and a processor obtaining the instruction stored in the memory to implement the image augmentation method.

The present application provides a computer-readable storage medium, at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in a computer device to implement the image augmentation method.

It can be seen from the above technical solutions that the variational learner constructed by this application adopts the structure of a fully convolutional neural network, which not only can accept input images of any size, but also can better extract the image to be amplified. The characteristics of the image solve the problem of inappropriate size of the input image, and then input the gas leakage image into the variational learner to obtain the target image, and use the discriminator to classify the Train the variational learner with the discrimination result of the target image to obtain the variational autoencoder model, and only when the target image is clear enough, will stop using the discrimination result to learn the variation trainer, which can improve the clarity of the image generated by the variational autoencoder model, and further, by calculating the reconstruction accuracy of the variational autoencoder model in the test image, Comparing the reconstruction accuracy rate with the preset threshold to determine whether to adjust the variational autoencoder model, which improves the reconstruction accuracy of the augmented model, so that the augmented model A clear amplified image can be accurately reconstructed.

1: Computer equipment

12: Storage

13: Processor

S10~S16: Steps

FIG. 1 is a flow chart of a preferred embodiment of the image augmentation method of the present application.

Fig. 2 is a schematic structural diagram of a variational learner of a preferred embodiment of the image augmentation method of the present application.

Fig. 3 is a schematic structural diagram of a discriminator of a preferred embodiment of the image augmentation method of the present application.

Fig. 4 is a schematic structural diagram of a computer device implementing a preferred embodiment of the image augmentation method of the present application.

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

As shown in FIG. 1 , it is a flowchart of a preferred embodiment of an image augmentation method of the present application. According to different requirements, the order of each step in the flowchart can be adjusted according to actual requirements, and some steps can be omitted. The execution body of the method is a computer device, such as the computer device 1 shown in FIG. 4 .

The image augmentation method can be applied to one or more computer devices 1 . The computer device 1 is a device that can automatically perform numerical calculations and/or information processing according to preset or stored instructions, and its hardware includes, but is not limited to: microprocessors, application specific integrated circuits (Application Specific Integrated Circuits) , ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital signal processor (Digital Signal Processor, DSP), embedded devices, etc.

The computer device 1 can be any electronic product capable of man-machine interaction with the user, for example, a personal computer, a tablet computer, a smart phone, a personal digital assistant (Personal Digital Assistant, PDA), a game console, an interactive Internet TV (Internet Protocol Television, IPTV), smart wearable devices, etc.

The computer equipment 1 may also include network equipment and/or user equipment. Wherein, the network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on Cloud Computing.

The network where the computer device 1 is located includes but not limited to Internet, wide area network, metropolitan area network, local area network, virtual private network (Virtual Private Network, VPN) and so on.

Step S10, acquiring an image to be amplified and a test image, wherein the test image includes a gas leakage image.

In at least one embodiment of the present application, the test image can be used to calculate the reconstruction accuracy of the variational autoencoder model, and the computer device can obtain the test image from a preset first database .

In at least one embodiment of the present application, the gas leakage image refers to an image containing leaked gas, and the gas in the gas leakage image may be chlorine gas, sulfur dioxide gas, etc., may be It should be understood that the gas leak image may be a leaked chlorine gas image, a leaked sulfur dioxide gas image, etc., and the gas leak image may be used to train a variational learner.

In at least one embodiment of the present application, the image to be amplified refers to an image that does not contain the leaked gas, and the computer device can obtain the image to be amplified from a preset second database picture.

Step S11, constructing a variational learner and a discriminator based on the fully convolutional neural network.

In at least one embodiment of the present application, the variational learner may be used to generate a reconstructed image.

In at least one embodiment of the present application, the discriminator is configured to determine whether the input image is generated by the variational learner.

In at least one embodiment of the present application, the variational learner includes an encoding network and a decoding network, and the computer device constructing a variational learner based on a fully convolutional neural network includes: The computer device constructs four hidden layers as the encoding network, each hidden layer is composed of a convolutional layer and a first activation function layer, further, the computer device constructs four operation layers as the decoding network, Each operation layer is composed of a deconvolution layer and the first activation function layer.

In at least one embodiment of the present application, the computer device constructing a discriminator includes: The computer equipment constructs four deep convolutional network layers and a second activation function layer as the discriminator, and the first three deep convolutional network layers are composed of a convolutional layer, a batch normalization layer and the first activation function layer , the fourth deep convolutional network layer is composed of a convolutional layer and the second activation function layer.

As shown in FIG. 2 , FIG. 2 is a schematic structural diagram of a variational learner of a preferred embodiment of the image augmentation method of the present application. The parameters of the encoding network (Encoder) in Figure 2 are as follows: set the number of filters (Channel) of the first convolutional layer to 32, and the size of the filter (kernel size) to 4×4 pixels, The stride size (stride) is set to 2 pixels, the number of filters in the second convolutional layer is set to 64, the filter size is set to 4×4 pixels, and the stride size is set to 2 pixels , set the number of filters in the third convolutional layer to 128, the filter size to 4×4 pixels, and the step size to 2 pixels, and set the number of filters in the fourth convolutional layer to Set to 256, the filter is large The minimum is set to 4×4 pixels, the step size is set to 2 pixels, and the activation functions are all ReLu. The parameters of the decoding network (Decoder) in Figure 2 are as follows: set the number of filters in the first deconvolution layer to 256, set the filter size to 4×4 pixels, and set the step size to 2 pixel, the number of filters in the second deconvolution layer is set to 128, the filter size is set to 4×4 pixels, the step size is set to 2 pixels, and the third deconvolution layer is set to The number of filters is set to 64, the filter size is set to 5×5 pixels, the step size is set to 2 pixels, the number of filters in the fourth deconvolution layer is set to 32, and the filter The size is set to 6×6 pixels, the step size is set to 2 pixels, and the activation functions are all ReLu.

As shown in FIG. 3 , FIG. 3 is a schematic structural diagram of a discriminator of a preferred embodiment of the image augmentation method of the present application. The parameters of each layer of the discriminator in Figure 3 are as follows: set the number of filters in the first deep convolutional network layer to 64, set the filter size to 128×128 pixels, add BN layer and ReLu activation function , set the number of filters in the second deep convolutional network layer to 128, set the filter size to 64×64 pixels, add the BN layer and ReLu activation function, and set the third deep convolutional network The number of filters in the layer is set to 256, the filter size is set to 32×32 pixels, the BN layer and the ReLu activation function are added, and the number of filters in the fourth deep convolutional network layer is set to 512 , the filter size is set to 31×31 pixels, the sigmoid activation function is added, and the last layer uses the sigmoid activation function.

Through the above implementation, the variational learner can be constructed based on the fully convolutional neural network, so that the variational learner can extract features from an input image of any size.

Step S12, inputting the gas leakage image into the variational learner to obtain a target image.

In at least one embodiment of the present application, the target image refers to an image including gas features in the gas leakage image.

The specific generation process of the target image is consistent with the generation process of the augmented image below, so the present application will not repeat it here.

Step S13, training the variational learner according to the discrimination result of the discriminator on the target image to obtain a variational autoencoder model.

In at least one embodiment of the present application, the variational autoencoder model refers to a model obtained after training the variational learner using the gas leakage image, and the variational autoencoder model It can be used to generate an image having the characteristics of the gas in the gas leak image.

In at least one embodiment of the present application, the computer device trains the variational learner according to the discrimination result of the discriminator to the target image, and the obtained variational autoencoder model includes: The computer device inputs the target image into the discriminator, and obtains the discriminant probability that the discriminator determines the target image as a fake image, further, the computer device will be greater than or equal to the first The target image corresponding to the discriminant probability of a preset value is re-inputted into the variational learner for training to obtain the first image. Further, the computer device is based on the gas leakage image, the The target image and the first image calculate the loss value of the variational learner, and use gradient backpropagation to update the weight of the variational learner until the loss value drops to the minimum, and the obtained Variational autoencoder model described above.

Wherein, the first preset value can be set by itself, which is not limited in this application.

The fake image refers to the image generated by the variational learner.

其中，loss為所述損失值，M是指所述目標圖像中所有畫素點的數量，N是指所述氣體外洩圖像中所有畫素點的數量，K是指所述第一圖像中所有畫素點的數量，i是指所述目標圖像中第i個畫素點，j是指所述氣體外洩圖像中與i對應的畫素點，r是指所述第一圖像中與i對應的畫素點，y _i 是指所述目標圖 像中第i個畫素點的畫素值，x _j 是指所述氣體外洩圖像中第j個畫素點的畫素值，z _r 是指所述第一圖像中第r個畫素點的畫素值。 Specifically, calculating the loss value of the variational learner based on the gas leakage image, the target image, and the first image by the computer device includes: the calculation method of the loss value is:

Wherein, loss is the loss value, M refers to the number of all pixels in the target image, N refers to the number of all pixels in the gas leakage image, and K refers to the first The number of all pixels in the image, i refers to the i- th pixel in the target image, j refers to the pixel corresponding to i in the gas leakage image, r refers to the The pixel point corresponding to i in the first image, y _i refers to the pixel value of the i- th pixel point in the target image, and x _j refers to the j -th pixel point in the gas leakage image The pixel value of the pixel point, z _r refers to the pixel value of the rth pixel point in the first image.

Through the above-mentioned embodiment, the discriminator is used to discriminate the target image to obtain the discrimination probability, and the target image corresponding to the discrimination probability greater than or equal to the first preset value is determined as the first An image, and inputting the first image into the variational learner for retraining increases the training data of the variational learner so that the variational autoencoder model can learn more accurately The feature of the gas leakage image is used to improve the reconstruction ability of the variational autoencoder model.

Step S14, calculating the reconstruction accuracy rate of the variational autoencoder model based on the test image.

In at least one embodiment of the present application, the reconstruction accuracy refers to the detection accuracy of the variational autoencoder model on the test image.

In at least one embodiment of the present application, calculating the reconstruction accuracy rate of the variational autoencoder model based on the test image by the computer device includes: the computer device obtains an annotation result of the test image, The test image is input into the variational autoencoder model to obtain a feature image, and the computer device calculates a similarity value between the feature image and the test image, further, the The computer device compares the similarity value with a second preset value to obtain a verification result of the test image, further, the computer device compares the verification result with the labeling result, and compares the verification result with the The test image corresponding to the verification result with the same labeling result is determined as a second image, and the feature image corresponding to the second image is determined as a similar image, and the computer device calculates the similarity map The proportion of the image in the feature image, and determine the proportion as the reconstruction accuracy rate.

Wherein, the labeling result includes that the gas feature exists in any test image, and that the gas feature does not exist in any test image.

The second preset value can be set by itself, which is not limited in this application.

Specifically, calculating the similarity value between the feature image and the test image by the computer device includes: The computer device performs grayscale processing on the characteristic image to obtain a grayscale image, and performs binarization processing on the grayscale image to obtain a third image, and the computer device converts the The test image corresponding to the feature image is grayscaled and binarized to obtain a fourth image, and a similarity value between the third image and the fourth image is calculated.

Specifically, the computer device compares the similarity value with a second preset value, and the verification result of the test image obtained includes: the computer device will be greater than or equal to the similarity value of the second preset value The corresponding verification result is determined to be that the gas feature exists in the test image, and the verification result corresponding to a similarity value smaller than the second preset value is determined to be that the test image does not have the gas feature.

c ₁=(K ₁ L)²；c ₂=(K ₂ L)²；其中，SSIM(x,y)為所述相似值，x為所述第三圖像，y為所述第四圖像，μ _x 為所述第三圖像的灰度平均值，μ _y 為所述第四圖像的灰度平均值，σ _x 為所述第三圖像的灰度標準差，σ _y 為所述第四圖像的灰度標準差，σ _xy 為所述第三圖像與所述第四圖像之間的灰度協方差，c ₁及c ₂均為預設參數，L為所述第四圖像中最大的畫素值，K ₁及K ₂是預先設置的常數，且K ₁≪1，K ₂≪1。 The formula for determining the similarity value is:

c ₁ =( K ₁ L ) ² ; c ₂ =( K ₂ L ) ² ; wherein, SSIM ( x,y ) is the similarity value, x is the third image, and y is the fourth image Like, μ _x is the gray average value of the third image, μ _y is the gray average value of the fourth image, σ _x is the gray standard deviation of the third image, σ _y is The grayscale standard deviation of the fourth image, σxy is the _grayscale covariance between _the third image and the fourth image, c1 _and c2 are preset parameters, and L is the The largest pixel value in the fourth image, K ₁ and K ₂ are preset constants, and K ₁ ≪1, K ₂ ≪1.

Through the above-mentioned implementation manner, grayscale and binarization processing are performed on the test image and the feature image respectively, so that the pixels of different pixel points in the third image and the fourth image The value difference is more obvious, and it is more convenient to calculate the similarity value between the third image and the fourth image. By comparing the similarity value with the second preset value, it is possible to accurately select A feature image that is sufficiently similar to the test image is used as the similar image, and the reconstruction accuracy of the variational autoencoder model can be accurately calculated according to the similar image.

Step S15, if the reconstruction accuracy is less than a preset threshold, adjust the variational autoencoder model based on the gas leakage image to obtain an augmented model.

In at least one embodiment of the present application, the preset threshold may include, but not limited to: 0.8, 0.9.

The augmented model refers to a variational autoencoder model whose reconstruction accuracy rate is greater than or equal to the preset threshold. The augmented model includes an encoder and a decoder, the encoder is generated after training the encoding network according to the gas leakage image, and the decoder is generated according to the gas leakage image generated after training the decoding network.

In at least one embodiment of the present application, the computer device adjusts the variational autoencoder model based on the gas leakage image, and obtaining the augmented model includes: the computer device converts the gas leakage image input to the variational autoencoder model for training until the reconstruction accuracy rate is greater than or equal to the preset threshold to obtain the augmented model.

Wherein, the adjustment method of the variational autoencoder model is to input the gas leakage image into the variational autoencoder model for training until the reconstruction accuracy rate is greater than or equal to the preset threshold , to obtain the amplification model.

Through the above implementation manner, the variational autoencoder corresponding to the reconstruction accuracy less than the preset threshold can be adjusted, so as to improve the reconstruction accuracy of the augmented model.

Step S16, inputting the image to be augmented into the augmentation model to obtain an augmented image.

In at least one embodiment of the present application, the augmented image refers to a reconstructed image including gas leakage features in the gas leakage image.

In at least one embodiment of the present application, the computer device inputs the image to be amplified into the augmentation model, and obtaining the augmented image includes: the computer device inputs the image to be amplified Input to the hidden layer of the encoder for feature extraction to obtain a feature vector, wherein there are 2n elements in the feature vector, extract the first n elements in the feature vector as a mean vector, and extract the feature The last n elements in the vector are used as standard deviation vectors. Further, the computer device generates Gaussian random numbers according to the mean vector and the standard deviation vector, and randomly samples the Gaussian random numbers to obtain sampled values. Each element in the mean value vector is multiplied by the sampling value to obtain multiple multiplication results, and further, the computer device multiplies each multiplication result with the corresponding element in the standard deviation vector performing an addition operation to obtain a latent vector, and inputting the latent variable to the operation layer of the decoder for mapping processing to obtain the augmented image.

Wherein, the Gaussian random number can be generated by a Box-Muller algorithm according to the mean vector and the standard deviation vector.

Through the above implementation, the augmented image can be compressed into the latent vector by using the augmented model, and the noise in the image to be augmented is filtered during the compression process, so that the augmented image The image is clearer, because the amplification model has learned the gas features of the gas leakage image, and the reconstruction accuracy is high, so that the amplification model can accurately reconstruct the clear image containing the gas features. image.

It can be seen from the above technical solutions that the variational learner constructed by this application adopts the structure of a fully convolutional neural network, which not only can accept input images of any size, but also can better extract the image to be amplified. The characteristics of the image solve the problem of inappropriate size of the input image, and then input the gas leakage image into the variational learner to obtain the target image, and use the discriminator to classify the Train the variational learner with the discrimination result of the target image to obtain the variational autoencoder model, and only when the target image is clear enough, will stop using the discrimination result to learn the variation trainer, which can improve the clarity of the image generated by the variational autoencoder model, and further, by calculating the reconstruction accuracy of the variational autoencoder model in the test image, Comparing the reconstruction accuracy rate with the preset threshold to determine whether to adjust the variational autoencoder model, which improves the reconstruction accuracy of the augmented model, so that the augmented model A clear amplified image can be accurately reconstructed.

As shown in FIG. 4 , it is a schematic structural diagram of a computer device implementing a preferred embodiment of the image augmentation method of the present application.

In one embodiment of the present application, the computer device 1 includes, but is not limited to, a storage 12, a processor 13, and a computer program stored in the storage 12 and operable on the processor 13 , such as image augmentation programs.

Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1 and does not constitute a limitation to the computer device 1. It may include more or less components than those shown in the illustration, or combine certain components, or have different Components, for example, the computer device 1 may also include input and output devices, network access devices, bus bars, and the like.

The processor 13 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC) , Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., and the processor 13 is the computing core and control center of the computer device 1, and utilizes various interfaces and lines to connect the entire computer device 1 Each part of the system, and obtain the operating system of the computer device 1 and various installed applications, program codes, etc.

The processor 13 acquires the operating system of the computer device 1 and various installed applications. The processor 13 acquires the application program to implement the steps in the above embodiments of the image augmentation method, such as the steps shown in FIG. 1 .

Exemplarily, the computer program can be divided into one or more modules/units, and the one or more modules/units are stored in the storage 12 and acquired by the processor 13, to complete this application. The one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the acquisition process of the computer program in the computer device 1 .

The storage 12 can be used to store the computer programs and/or modules, and the processor 13 executes or obtains the computer programs and/or modules stored in the storage 12, and calls the computer programs and/or modules stored in the storage 12 to realize various functions of the computer device 1. The memory 12 can host It should include a storage program area and a storage data area, wherein the storage program area can store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the storage data area can be stored according to computer equipment. The use of created materials, etc. In addition, the storage 12 may include non-volatile storage, such as hard disk, memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash memory A memory card (Flash Card), at least one magnetic disk storage device, a flash memory device, or other non-volatile solid state storage devices.

The storage 12 may be an external storage and/or an internal storage of the computer device 1 . Further, the storage 12 may be a physical storage, such as a storage stick, a TF card (Trans-flash Card) and the like.

If the integrated modules/units of the computer device 1 are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, all or part of the processes in the methods of the above embodiments of the present application can also be completed by instructing related hardware through computer programs, and the computer programs can be stored in a computer-readable storage medium. When the computer program is acquired by the processor, it can realize the steps of the above-mentioned various method embodiments.

Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, obtainable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer storage, a read-only memory (ROM, Read- Only Memory).

Referring to FIG. 1 , the memory 12 in the computer device 1 stores a plurality of instructions to implement an image augmentation method, and the processor 13 can acquire the plurality of instructions to realize: acquire an image to be amplified and a test image, wherein the test image includes a gas leakage image; a variational learner and a discriminator are constructed based on a fully convolutional neural network; the gas leakage image is input to the variation learning In the device, the target image is obtained; the variational learner is trained according to the discrimination result of the target image by the discriminator to obtain the variational autoencoder model; the variational autoencoder model is calculated based on the test image. The reconstruction accuracy rate of the encoder model; if the reconstruction accuracy rate is less than the preset threshold, based on the gas leakage image adjustment Adjusting the variational self-encoder model to obtain an augmented model; inputting the image to be augmented into the augmented model to obtain an augmented image.

Specifically, for the specific implementation method of the above instructions by the processor 13, reference may be made to the description of relevant steps in the embodiment corresponding to FIG. 1 , and details are not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple networks on the unit. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented not only in the form of hardware, but also in the form of hardware plus software function modules.

Therefore, no matter from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the application is defined by the appended claims rather than the above description, so it is intended to All changes within the meaning and range of equivalents of the elements are embraced in this application. Any attached reference mark in a claim shall not be deemed to limit the claim to which it relates.

In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or devices stated in this application may also be realized by one unit or device through software or hardware. The terms first, second, etc. are used to denote names and do not imply any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art It should be understood that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

S10~S16: Steps

Claims (10)

An image augmentation method executed on computer equipment, wherein the image augmentation method includes: acquiring an image to be augmented and a test image, wherein the test image includes a gas leakage image; A convolutional neural network constructs a variational learner and a discriminator; the gas leakage image is input into the variational learner to obtain a target image; according to the discrimination of the target image by the discriminator As a result, train the variational learner to obtain a variational autoencoder model; calculate the reconstruction accuracy rate of the variational autoencoder model based on the test image; if the reconstruction accuracy rate is less than a preset threshold, adjusting the variational autoencoder model based on the gas leakage image to obtain an augmented model; inputting the image to be augmented into the augmented model to obtain an augmented image. The image augmentation method according to claim 1, wherein the training of the variational learner according to the discriminant result of the discriminator for the target image, and obtaining the variational autoencoder model includes: The target image is input into the discriminator to obtain the discriminant probability that the discriminator determines the target image as a false image; the target image corresponding to the discriminant probability greater than or equal to the first preset value Re-input into the variational learner for training to obtain a first image; calculate the loss value of the variational learner based on the gas leakage image, the target image and the first image , and use the gradient backpropagation to update the weight value of the variational learner until the loss value drops to the minimum, and obtain the variational autoencoder model. The image augmentation method according to claim 2, wherein the calculating the loss value of the variational learner based on the gas leakage image, the target image and the first image includes: The calculation method of the loss value is:

Wherein, loss is the loss value, M refers to the number of all pixels in the target image, N refers to the number of all pixels in the gas leakage image, and K refers to the first The number of all pixels in the image, i refers to the i- th pixel in the target image, j refers to the pixel corresponding to i in the gas leakage image, r refers to the The pixel point corresponding to i in the first image, y _i refers to the pixel value of the i- th pixel point in the target image, and x _j refers to the j -th pixel point in the gas leakage image The pixel value of the pixel point, z _r refers to the pixel value of the rth pixel point in the first image. The image augmentation method according to claim 1, wherein the calculation of the reconstruction accuracy rate of the variational autoencoder model based on the test image includes: obtaining the labeling result of the test image; The test image is input into the variational autoencoder model to obtain a feature image; the similarity value between the feature image and the test image is calculated; the similarity value is compared with a second preset Values are compared to obtain the verification result of the test image; the verification result is compared with the labeling result; the test image corresponding to the verification result identical to the labeling result is determined as the second image , and determine the feature image corresponding to the second image as a similar image; calculate the ratio of the similar image in the feature image, and determine the ratio as the reconstructed Correct rate. The image augmentation method according to claim 4, wherein the calculating the similarity value between the feature image and the test image includes: performing grayscale processing on the feature image to obtain gray The grayscale image is subjected to binarization processing to obtain a third image; the test image corresponding to the feature image is grayscale processed and binarized to obtain a third image. Four images; calculating the similarity value of the third image and the fourth image, the determination formula of the similarity value is:

c ₁ =( K ₁ L ) ² ; c ₂ =( K ₂ L ) ² ; wherein, SSIM ( x,y ) is the similarity value, x is the third image, and y is the fourth image Like, μ _x is the gray average value of the third image, μ _y is the gray average value of the fourth image, σ _x is the gray standard deviation of the third image, σ _y is The grayscale standard deviation of the fourth image, σxy is the _grayscale covariance between _the third image and the fourth image, c1 _and c2 are preset parameters, and L is the The largest pixel value in the fourth image, K ₁ and K ₂ are preset constants, and K ₁ ≪1, K ₂ ≪1. The image augmentation method according to claim 1, wherein said adjusting the variational autoencoder model based on the gas leak image to obtain an augmented model includes: inputting the gas leak image The variational autoencoder model is trained until the reconstruction accuracy rate is greater than or equal to the preset threshold to obtain the augmented model. The image augmentation method as described in claim 1, wherein the augmented model includes an encoder and a decoder, and a fully convolutional neural network is used in the encoder, and the fully convolutional neural network includes A plurality of hidden layers, a deconvolution neural network is used in the decoder, and a plurality of operation layers are included in the deconvolution neural network. The image augmentation method according to claim 7, wherein the inputting the image to be augmented into the augmentation model to obtain the augmented image includes: inputting the image to be augmented Perform feature extraction in the hidden layer of the encoder to obtain a feature vector, wherein there are 2 n elements in the feature vector; extract the first n elements in the feature vector as a mean vector; extract the feature vector The latter n elements in the vector are used as standard deviation vectors; Gaussian random numbers are generated according to the mean value vector and the standard deviation vector; random sampling is carried out to the Gaussian random numbers to obtain sampling values; each of the mean value vectors is Elements are multiplied by the sampling value to obtain multiple multiplication results; each multiplication result is added to the corresponding element in the standard deviation vector to obtain a potential vector; the potential variable is input to The operation layer of the decoder performs mapping processing to obtain the augmented image. A computer device, wherein the computer device includes: a memory storing at least one instruction; and a processor obtaining the instruction stored in the memory to realize the image as described in any one of claims 1 to 8 Amplification method. A computer-readable storage medium, wherein: at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in a computer device to implement any one of claims 1 to 8. image augmentation method.

Images