CN111612699B - Image processing method, apparatus and computer readable storage medium - Google Patents
Image processing method, apparatus and computer readable storage medium Download PDFInfo
- Publication number
- CN111612699B CN111612699B CN201910130830.9A CN201910130830A CN111612699B CN 111612699 B CN111612699 B CN 111612699B CN 201910130830 A CN201910130830 A CN 201910130830A CN 111612699 B CN111612699 B CN 111612699B
- Authority
- CN
- China
- Prior art keywords
- image
- type
- images
- generator
- optimized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 62
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000005457 optimization Methods 0.000 claims abstract description 21
- 230000008485 antagonism Effects 0.000 claims abstract description 12
- 210000000697 sensory organ Anatomy 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 9
- 238000003708 edge detection Methods 0.000 claims description 6
- 230000001815 facial effect Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000012546 transfer Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 10
- 210000000056 organ Anatomy 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 210000003128 head Anatomy 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000003796 beauty Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The disclosure provides an image processing method, an image processing device and a computer readable storage medium, and relates to the technical field of image processing. An image processing method of the present disclosure includes: acquiring a first image shot by an image acquisition device; a first generator of the antagonism network generates a second image based on the first image, wherein the antagonism network is generated for training with the set of first type images and the set of target type images; the first generator performs total variation regular optimization constraint on the second image to reduce gradient, and obtains an optimized second image. By the method, the countermeasure network can be trained, and the image shot by the image acquisition device is converted into a target type image; the image gradient can be reduced by utilizing the total variation regularization optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
Description
Technical Field
The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, an image processing device, and a computer readable storage medium.
Background
With the development and popularization of social networks, the creation of art and style processing on the basis of photos is popular, for example, a hand-painted style head portrait based on real self-timer can also attract a lot of eyes while protecting personal privacy.
At present, the generation technology for the hand-drawn head portrait mainly comprises two kinds, namely, a method in the field of computer images is utilized, each part is obtained by applying the theoretical segmentation of feature extraction to the face, then the corresponding style is obtained for the parts by adopting the technologies of NPR (Non-Photorealistic Rendering, non-real rendering), image geometric transformation and the like, and finally the images are recombined. The other relies on manual creation, with manual adjustment and refinement by the painter via an image editing tool.
Disclosure of Invention
The inventor finds that although an automatic image processing method can quickly generate an image in an artistic form, the aspects of the effect, quality and the like of the image often cannot meet the requirements; however, the manual method can obtain the effect of satisfying the user, but takes huge resources such as time, money, manpower and the like.
It is an object of the present disclosure to improve the quality of automated image processing.
According to one aspect of the present disclosure, there is provided an image processing method including: acquiring a first image shot by an image acquisition device; a first generator of the antagonism network generates a second image based on the first image, wherein the antagonism network is generated for training with the set of first type images and the set of target type images; the first generator performs total variation regular optimization constraint on the second image to reduce gradient, and obtains an optimized second image.
In some embodiments, the first type of image is an image captured by an image capture device; and/or the object type image is a hand-drawn image.
In some embodiments, the first image is a facial image captured by an image capture device.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images includes: a first generator generating a second type of image from images in the set of first type of images; the first generator performs total variation regular optimization constraint on the second type image to reduce gradient, and obtains an optimized second type image; judging whether the optimized second type image belongs to the target type image or not by the discriminator of the countermeasure network until the judgment result is that all the optimized second type images belong to the target type image.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images further comprises: performing edge detection and Gaussian blur processing on images in a set of target type images by an countermeasure network, adding noise, and obtaining a blurred target type image; and judging whether the fuzzy target type image belongs to the target type image or not by the judging device until the judging result is that all the fuzzy target type images belong to the target type image.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images further comprises: a second generator generates a third type of image from the optimized second type of image, wherein the second generator is configured to reconstruct the target type of image into the first type of image; the difference value of the first type image generating the optimized second type image and the third type image generated based on the optimized second type image is acquired and reversely transferred to reduce the difference value.
In some embodiments, the image processing method further comprises: performing five sense organ detection on the image acquired by the first generator to determine the position of the five sense organs; acquiring a difference value of a first type image generating an optimized second type image and a third type image generated based on the optimized second type image includes: and determining a difference value according to the difference of the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, wherein the weight of the five sense organs is greater than that of other positions.
By the method, the countermeasure network can be trained, and the image shot by the image acquisition device is converted into a target type image; the image gradient can be reduced by utilizing the total variation regularization optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
According to an aspect of other embodiments of the present disclosure, there is provided an image processing apparatus including: an image acquisition unit configured to acquire a first image captured by an image capturing device; a first generator configured to generate a second image based on the first image, wherein a countermeasure network to which the first generator belongs is generated for training with the set of first type images and the set of target type images; and performing total variation regular optimization constraint on the second image to reduce gradient, and obtaining an optimized second image.
In some embodiments, the first type of image is an image captured by an image capture device; and/or the object type image is a hand-drawn image.
In some embodiments, the first image is a facial image captured by an image capture device.
In some embodiments, the first generator is further configured to generate a second type of image from the images in the first type of image set; performing total variation regular optimization constraint on the second type image to reduce gradient and obtain an optimized second type image; the image processing apparatus further includes: and the discriminator is configured to judge whether the optimized second type image generated by the first generator belongs to the target type image in the countermeasure network training until all judgment results are that the optimized second type image belongs to the target type image.
In some embodiments, the image processing apparatus further includes: the blurring processing unit is configured to perform edge detection and Gaussian blurring processing on images in the set of the target type images, add noise and acquire blurred target type images; the discriminator is further configured to judge whether the blurred object type image belongs to the object type image until the judgment result is that all blurred object type images belong to the object type image.
In some embodiments, the image processing apparatus further includes: a second generator configured to generate a third type of image from the optimized second type of image, wherein the second generator is configured to reconstruct the target type of image into the first type of image; and a difference acquisition unit configured to acquire and reversely transfer a difference value of the first type image generating the optimized second type image and the third type image generated based on the optimized second type image to reduce the difference value.
In some embodiments, the image processing apparatus further includes: the system comprises an five-sense organ position extraction unit, a five-sense organ position detection unit and a control unit, wherein the five-sense organ position extraction unit is configured to perform five-sense organ detection on the first image and determine the five-sense organ position; the difference acquisition unit is configured to determine a difference value according to a difference between the first type image and the third type image at each relative pixel position and a weight of the corresponding pixel position and reversely transfer the difference value so as to reduce the difference value, wherein the weight of the five-element position is larger than that of other positions.
According to an aspect of further embodiments of the present disclosure, there is provided an image processing apparatus including: a memory; and a processor coupled to the memory, the processor configured to perform any of the image processing methods described above based on instructions stored in the memory.
The image processing device can convert the image shot by the image acquisition device into a target type image by utilizing a trained countermeasure network; the image gradient can be reduced by utilizing the total variation regularization optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
According to an aspect of further embodiments of the present disclosure, a computer-readable storage medium is presented, on which computer program instructions are stored, which instructions, when executed by a processor, implement the steps of any one of the image processing methods above.
By executing instructions on such a computer-readable storage medium, the image captured by the image capture device can be converted into a target type image using a trained countermeasure network; the image gradient can be reduced by utilizing the total variation regularization optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate and explain the present disclosure, and together with the description serve to explain the present disclosure. In the drawings:
fig. 1 is a flow chart of some embodiments of an image processing method of the present disclosure.
Fig. 2 is a flow chart of other embodiments of the image processing method of the present disclosure.
Fig. 3 is a flow chart of still further embodiments of the image processing method of the present disclosure.
Fig. 4 is a schematic diagram of some embodiments of an image processing apparatus of the present disclosure.
Fig. 5 is a schematic diagram of further embodiments of an image processing apparatus of the present disclosure.
Fig. 6 is a schematic diagram of still other embodiments of an image processing apparatus of the present disclosure.
Fig. 7 is a schematic diagram of still other embodiments of an image processing apparatus of the present disclosure.
Detailed Description
The technical scheme of the present disclosure is described in further detail below through the accompanying drawings and examples.
A flowchart of some embodiments of the image processing method of the present disclosure is shown in fig. 1.
In step 101, a first image captured by an image capturing device is acquired. In some embodiments, the first image may be taken by a camera or may be taken from a photograph. In some embodiments, the first image may be a captured image of the face of the user, such as a user's own photograph or the like.
In step 102, the first generator generates a second image based on the first image, the first generator belongs to a antagonism network, the antagonism network is generated by training by using the set of the first type image and the set of the target type image, and the whole process is optimized through continuous training, so that a stable generation model is finally obtained.
In step 103, the first generator of the countermeasure network performs a full variation canonical optimization constraint on the second image to reduce the gradient, and obtains an optimized second image.
By the method, the countermeasure network can be trained, the image shot by the image acquisition device is converted into the target type image, and the image processing quality is improved. In addition, since the first image is captured, under the influence of angles and environments, interference factors such as uneven illumination may cause color patches to be generated in the image. Through total variation regular optimization constraint, gradient distribution can be more uniform, noise is reduced, the influence of interference factors is avoided, colors are smoother and more uniform, and therefore overall attractive results are obtained. Particularly, when the face image is processed, the face lines can be clean and attractive, the definition and the beauty are realized, and the image quality is improved.
In some embodiments, a total variation constraint may be added to the first generator generated image G AB (x), namely:
Representing the total variation canonical optimization constraint processing of the image x belonging to the set a of the first type images. Since the total variation of the image contaminated with noise is significantly larger than that of the noise-free image, noise such as black blocks or the like is formed on the generated image by the disturbance factor. By introducing total variation constraint and limiting noise, the background of the generated image can be smoother and more uniform in color change, and therefore the overall attractive and clean result is obtained.
In some embodiments, the set of images captured by the image capture device (i.e., the set of first type images) and the set of target type images (e.g., the set of hand-drawn style images) may be configured during the training process. Inputting a set of first type images into a first generator, wherein the first generator generates images, judging whether the generated images belong to a target type by a discriminator of an antagonism network after the images are subjected to total variation constraint, and the training aims at judging that the generated images belong to the target type; a set of target type images may also be input to the arbiter, the goal of the training being that the input images belong to the target type.
The method can enable the capabilities of the first generator and the discriminator to be continuously improved in the process of mutual antagonism and iteration, and when the final discriminator capability is improved to a certain degree and the data source cannot be judged to be the first generator or the target type image, the image processing capability of the first generator is determined to meet the requirement.
A flowchart of further embodiments of the image processing method of the present disclosure is shown in fig. 2.
In step 201, a first generator generates a second type of image according to images in a set of first type of images, and performs total variation regular optimization constraint on the second type of image to obtain an optimized second type of image.
In step 202, an countermeasure network performs edge detection and gaussian blur processing on images in a set of target type images, adds noise, and acquires a blurred target type image.
The optimized second type image in step 201 and the blurred object type image in step 202 are randomly input into a discriminator of the countermeasure network to make a judgment.
In step 203, the discriminator of the countermeasure network determines whether the optimized second type image, the blurred object type image, or not belongs to the object type image. The judging device feeds back the judging result to the first generator until the judging result is that all the fuzzy target type images belong to the target type images, and the second type images are optimized to belong to the target type images.
In some embodiments, the data input to the arbiter may include:
wherein A is a set of first type images, B is a set of target type images, Z is a set of blurred target type images, for images x belonging to A, images y belonging to B, images Z belonging to Z, Representing taking element x in set a, and so on.
By the method, noise can be manufactured for the target type image to be judged in the discriminator, so that the noise identification capability of the discriminator is improved, the noise reduction capability of the generator is enhanced, and the image generated by the first generator is clearer and cleaner.
In some embodiments, two generators may be included in the countermeasure network, the first generator G AB being aimed at converting the first type of image to the target type, the second generator G BA being used to generate the first type of image based on the images generated by G AB. In some embodiments, the generator may employ Unet and the arbiter may employ PatchGAN.
In some embodiments, two discriminators may be included in the countermeasure network, a first discriminator D B for determining whether the image generated by G AB is a target type image and a second discriminator D A for determining whether the image generated by G BA is a first type image. Inputting a set of target type images into a second generator, randomly generating images by the second generator, judging whether the generated images belong to a first type or not by D A, wherein the training aims at judging that the generated images belong to the first type; the set of images of the first type is also entered into D A, the training being aimed at inputting images of the first type. The method can enable the capability of the second generator and the capability of the second discriminator to be continuously improved in the process of mutual antagonism and iteration, and when the capability of the second discriminator is finally improved to a certain degree and the data source cannot be judged to be the second generator or the first type image, the image processing capability of the second generator is determined to meet the requirement.
In some embodiments, a first type of image F1 may be input to the first generator, a second type of image F2 may be output and input to the second generator, and F2 may be input to the second generator to obtain an image F3 of a third type; comparing F1 with F3, obtaining the difference between the images, gradually reducing the difference in the training process, and optimizing the capacities of the first generator and the second generator.
A flowchart of still further embodiments of the image processing method of the present disclosure is shown in fig. 3.
In step 301, a five sense organ detection is performed on the image acquired by the first generator, and a five sense organ position is determined. In some embodiments, five sense organ detection may be performed on the image, such as:
E(x)＝[bbox1,…bboxj,…bbox0]，
Wherein X is an E X, E (X) is a method for detecting facial features, bbox j represents position information of five sense organs, specifically coordinates of an upper left vertex and a lower right vertex of a rectangular frame, and k is the number of detected positions. The more the number k, the more delicate the five sense organs are produced. For example, taking k=3, representing three face parts of eyes, nose, and mouth, these pieces of position information are stored after the detection is completed.
In step 302, a first generator generates a second type of image from images in a set of first type of images. And performing total variation regular optimization constraint on the second type image to reduce gradient, and obtaining an optimized second type image.
In step 303, the second generator generates a third type of image from the optimized second type of image.
In step 304, a difference value is determined according to the difference between the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, wherein the weight of the five sense organs is greater than the weight of the other positions. In some embodiments, the formula may be employed:
the pixel-by-pixel distance of the two images is determined. In the scene of the hand-drawn image of the generated facial image, as the fine granularity generated by the five sense organs directly determines the aesthetic property of the result, besides adding constraint to the whole image, we pay more attention to the generation of single components such as eyes, nose, mouth and the like of the face, so that the network generates more delicate local characteristics. In some embodiments, k=4 may be taken, Representing the four parts of the whole figure, eyes, nose and mouth respectively,/>Is provided by step 301, i being the picture identity. The weight lambda j can be configured for each part according to the requirement, different weights represent the degree of the attention of the network, and the larger the weight is, the more the network focuses on the part, and the finer the part is generated after training is completed.
By the method, the part of the five sense organs can be focused more, the image of the part of the five sense organs is finer, and the definition and the accuracy of the part of the five sense organs in the image generated by the first generator are highlighted.
A schematic diagram of some embodiments of an image processing apparatus of the present disclosure is shown in fig. 4. The image acquisition unit 401 is capable of acquiring a first image captured by an image capturing device. In some embodiments, the first image may be taken by a camera or may be taken from a photograph. In some embodiments, the first image may be a captured image of the face of the user, such as a user's own photograph or the like.
The first generator 402 is capable of generating a second image based on the first image, the first generator being attributed to an antagonism network generated for training with the set of first type images and the set of target type images; and then the first generator of the countermeasure network performs total variation regular optimization constraint on the second image to reduce gradient, and the optimized second image is obtained.
The image processing device can convert the image shot by the image acquisition device into a target type image by utilizing a trained countermeasure network; the image gradient can be reduced by utilizing the total variation regularization optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
A schematic diagram of further embodiments of the image processing apparatus of the present disclosure is shown in fig. 5. In the training process, a set of first type images is input into a first generator 501, a first generator 502 generates second type images, and full-variation regular optimization constraint is carried out on the second type images to obtain optimized second type images. Judging by the discriminator 503 of the countermeasure network whether the generated optimized second type image belongs to the target type image, the object of the training being to judge that the generated image belongs to the target type image; the set of object type images is also input to the arbiter 503, the object of this training being that the input images belong to the object type images.
The image processing device can enable the capabilities of the first generator and the discriminator to be continuously improved in the process of mutual antagonism and iteration in the training process, when the final discriminator capability is improved to a certain degree and the data source is not judged to be the first generator or the target type image, the image processing capability of the first generator is determined to meet the requirement, and when the image acquisition unit 501 inputs the acquired image into the first generator, the image belonging to the target type can be obtained.
In some embodiments, as shown in fig. 5, the image processing apparatus may further include a blurring processing unit 504 capable of blurring the images in the set of target type images prepared in advance, such as edge detection and gaussian blurring, adding noise, and acquiring a blurred target type image. The blurring processing unit 504 inputs the blurred object type image to the discriminator 503, and judges whether or not it belongs to the object type image, and the object is the object type image. The image processing device can improve the identification capability of the discriminator on noise and enhance the capability of the generator for reducing the noise, so that the image generated by the first generator is clearer and cleaner.
In some embodiments, as shown in fig. 5, the image processing apparatus may further include a second generator 505 and a difference acquisition unit 506. The second generator 505 is capable of generating a first type of image from the second type of image generated by the first generator 502. In some embodiments, a first type of image F1 may be input to the first generator 502, an image F2 belonging to the optimized second type of image is output, and F2 is input to the second generator 505, resulting in an image F3 belonging to the third type; the difference obtaining unit 506 compares the F1 and the F3 to obtain a difference between the images, gradually reduces the difference in the training process, and optimizes the capabilities of the first generator and the second generator, so as to improve the fit degree of the image generated by the first generator 502 and the input image, and improve the reduction degree of the head portrait obtained by shooting when being used for generating the head portrait hand-painted image, and optimize the image generation effect.
In some embodiments, to highlight the lines of the five sense organs and blur other parts, the method may further include a five sense organ position extraction unit 507 capable of performing five sense organ detection on the image acquired by the first generator to determine the positions of the five sense organs. The difference obtaining unit 506 determines a difference value according to the difference between the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, where the weight of the five sense organs is greater than the weights of other positions, so as to realize a part focusing on the five sense organs more, make the image of the five sense organs more fine and smooth, and highlight the definition and accuracy of the five sense organs in the image generated by the first generator.
A schematic structural diagram of some embodiments of the image processing apparatus of the present disclosure is shown in fig. 6. The image processing apparatus includes a memory 601 and a processor 602. Wherein: the memory 601 may be a magnetic disk, flash memory or any other non-volatile storage medium. The memory is used to store instructions in the corresponding embodiments of the image processing method above. The processor 602 is coupled to the memory 601 and may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 602 is configured to execute instructions stored in the memory, so that the color block can be prevented from affecting the image processing effect, and the image processing quality can be improved.
In some embodiments, as also shown in fig. 7, the image processing apparatus 700 includes a memory 701 and a processor 702. The processor 702 is coupled to the memory 701 through a BUS 703. The image processing apparatus 700 may also be connected to an external storage apparatus 705 via a storage interface 704 for invoking external data, and may also be connected to a network or another computer system (not shown) via a network interface 706. And will not be described in detail herein.
In this embodiment, the data instruction is stored in the memory, and then the instruction is processed by the processor, so that the color block can be prevented from affecting the image processing effect, and the image processing quality can be improved.
In other embodiments, a computer readable storage medium has stored thereon computer program instructions which, when executed by a processor, implement the steps of the method in the corresponding embodiments of the image processing method. It will be apparent to those skilled in the art that embodiments of the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Thus far, the present disclosure has been described in detail. In order to avoid obscuring the concepts of the present disclosure, some details known in the art are not described. How to implement the solutions disclosed herein will be fully apparent to those skilled in the art from the above description.
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above-described sequence of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the sequence specifically described above unless specifically stated otherwise. Furthermore, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
Finally, it should be noted that: the above embodiments are merely for illustrating the technical solution of the present disclosure and are not limiting thereof; although the present disclosure has been described in detail with reference to preferred embodiments, those of ordinary skill in the art will appreciate that: modifications may be made to the specific embodiments of the disclosure or equivalents may be substituted for part of the technical features; without departing from the spirit of the technical solutions of the present disclosure, it should be covered in the scope of the technical solutions claimed in the present disclosure.
Claims (12)
1. An image processing method, comprising:
Acquiring a first image shot by an image acquisition device;
A first generator of an countermeasure network generating a second image based on the first image, wherein the countermeasure network is generated for training with a set of first type images and a set of target type images;
The first generator performs a total variation canonical optimization constraint on the second image to reduce the gradient, obtain an optimized second image,
Wherein training the countermeasure network with the set of first type images and the set of target type images comprises:
The first generator generates a second type of image from images in a set of first type of images;
the first generator performs total variation regular optimization constraint on the second type image to reduce gradient, and obtains an optimized second type image;
the countermeasure network performs edge detection and Gaussian blur processing on images in the set of target type images, adds noise, and obtains a blurred target type image;
The discriminator judges whether the optimized second type image and the fuzzy target type image belong to the target type image or not, and feeds back the judging result to the first generator until the judging result is that all the fuzzy target type image and the optimized second type image belong to the target type image.
2. The method of claim 1, wherein the first type of image is an image captured by an image capture device; and/or, the target type image is a hand-drawn image.
3. The method of claim 1, wherein the first image is a facial image captured by an image capture device.
4. The method of claim 1, wherein the training the countermeasure network with the set of first type images and the set of target type images further comprises:
A second generator generates a third type of image according to the optimized second type of image, wherein the second generator is used for reconstructing the target type of image into the first type of image;
And acquiring and reversely transferring a difference value of the first type image generating the optimized second type image and the third type image generated based on the optimized second type image so as to reduce the difference value.
5. The method of claim 4, further comprising: performing five sense organ detection on the image acquired by the first generator to determine the position of the five sense organs;
The obtaining a difference value of a first type image generating the optimized second type image and a third type image generated based on the optimized second type image includes:
And determining the difference value according to the difference of the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, wherein the weight of the five sense organs is greater than that of other positions.
6. An image processing apparatus comprising:
An image acquisition unit configured to acquire a first image captured by an image capturing device;
a first generator configured to generate a second image based on the first image, wherein an antagonism network to which the first generator belongs is generated for training with a set of first type images and a set of target type images; performing total variation regular optimization constraint on the second image to reduce gradient, and obtaining an optimized second image;
The blurring processing unit is configured to perform edge detection and Gaussian blurring processing on images in the set of the target type images, add noise and acquire blurred target type images;
the first generator is further configured to generate a second type of image from images in the first type of image set; performing total variation regular optimization constraint on the first image to reduce gradient and obtain an optimized second image;
And the discriminator is configured to judge whether the optimized second type image and the fuzzy target type image belong to the target type image or not until the judgment result is that all the fuzzy target type image and the optimized second type image belong to the target type image.
7. The apparatus of claim 6, wherein the first type of image is an image captured by an image capture device; and/or, the target type image is a hand-drawn image.
8. The apparatus of claim 6, wherein the first image is a facial image captured by an image capture device.
9. The apparatus of claim 6, further comprising:
A second generator configured to generate a third type of image from the optimized second type of image, wherein the second generator is configured to reconstruct the target type of image into the first type of image;
And a difference acquisition unit configured to acquire and reversely transfer a difference value of a first type image generating the optimized second type image and a third type image generated based on the optimized second type image to reduce the difference value.
10. The apparatus of claim 9, further comprising:
The five sense organ position extraction unit is configured to perform five sense organ detection on the image acquired by the first generator and determine the five sense organ position;
The difference acquisition unit is configured to determine the difference value according to the difference between the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position and reversely transmit the difference value so as to reduce the difference value, wherein the weight of the five sense organs is larger than that of other positions.
11. An image processing apparatus comprising:
A memory; and
A processor coupled to the memory, the processor configured to perform the method of any of claims 1-5 based on instructions stored in the memory.
12. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910130830.9A CN111612699B (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910130830.9A CN111612699B (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111612699A CN111612699A (en) | 2020-09-01 |
| CN111612699B true CN111612699B (en) | 2024-05-17 |
Family
ID=72201350
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910130830.9A Active CN111612699B (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111612699B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304765A (en) * | 2017-12-11 | 2018-07-20 | 中国科学院自动化研究所 | Multitask detection device for face key point location and semantic segmentation |
| CN108537743A (en) * | 2018-03-13 | 2018-09-14 | 杭州电子科技大学 | A kind of face-image Enhancement Method based on generation confrontation network |
| CN108615073A (en) * | 2018-04-28 | 2018-10-02 | 北京京东金融科技控股有限公司 | Image processing method and device, computer readable storage medium, electronic equipment |
| CN108846793A (en) * | 2018-05-25 | 2018-11-20 | 深圳市商汤科技有限公司 | Image processing method and terminal device based on image style transformation model |
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN109190722A (en) * | 2018-08-06 | 2019-01-11 | 大连民族大学 | Font style based on language of the Manchus character picture migrates transform method |
| CN109191402A (en) * | 2018-09-03 | 2019-01-11 | 武汉大学 | The image repair method and system of neural network are generated based on confrontation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019019199A1 (en) * | 2017-07-28 | 2019-01-31 | Shenzhen United Imaging Healthcare Co., Ltd. | System and method for image conversion |
| US10753997B2 (en) * | 2017-08-10 | 2020-08-25 | Siemens Healthcare Gmbh | Image standardization using generative adversarial networks |
-
2019
- 2019-02-22 CN CN201910130830.9A patent/CN111612699B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN108304765A (en) * | 2017-12-11 | 2018-07-20 | 中国科学院自动化研究所 | Multitask detection device for face key point location and semantic segmentation |
| CN108537743A (en) * | 2018-03-13 | 2018-09-14 | 杭州电子科技大学 | A kind of face-image Enhancement Method based on generation confrontation network |
| CN108615073A (en) * | 2018-04-28 | 2018-10-02 | 北京京东金融科技控股有限公司 | Image processing method and device, computer readable storage medium, electronic equipment |
| CN108846793A (en) * | 2018-05-25 | 2018-11-20 | 深圳市商汤科技有限公司 | Image processing method and terminal device based on image style transformation model |
| CN109190722A (en) * | 2018-08-06 | 2019-01-11 | 大连民族大学 | Font style based on language of the Manchus character picture migrates transform method |
| CN109191402A (en) * | 2018-09-03 | 2019-01-11 | 武汉大学 | The image repair method and system of neural network are generated based on confrontation |
Non-Patent Citations (2)
| Title |
|---|
| Chen Yang 等.CartoonGAN:Generative Adeversarial Networks for Photo Cartoonization.《2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition》.全文. * |
| 基于深度特征插值的图像变换方法研究与实现;陈梓炜;《中国优秀硕博士学位论文 信息科技辑》;20180815;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111612699A (en) | 2020-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Practical deep raw image denoising on mobile devices | |
| Li et al. | Detail-enhanced multi-scale exposure fusion | |
| Liu et al. | Fast burst images denoising | |
| CN106778928B (en) | Image processing method and device | |
| Bugeau et al. | Variational exemplar-based image colorization | |
| Gallo et al. | Artifact-free high dynamic range imaging | |
| Xiao et al. | Fast image dehazing using guided joint bilateral filter | |
| CN106899781B (en) | Image processing method and electronic equipment | |
| US9042662B2 (en) | Method and system for segmenting an image | |
| Li et al. | Detail-preserving multi-exposure fusion with edge-preserving structural patch decomposition | |
| CN110276767A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN110248096A (en) | Focusing method and device, electronic equipment, computer readable storage medium | |
| CN104504745B (en) | A kind of certificate photo generation method split based on image and scratch figure | |
| Lin et al. | Real photographs denoising with noise domain adaptation and attentive generative adversarial network | |
| CN107610108A (en) | Image processing method and device | |
| JP2010045613A (en) | Image identifying method and imaging device | |
| CN105243371A (en) | Human face beauty degree detection method and system and shooting terminal | |
| JP2010244398A (en) | Image processing apparatus and image processing program | |
| Zhang et al. | Deep motion blur removal using noisy/blurry image pairs | |
| CN110276831A (en) | Constructing method and device, equipment, the computer readable storage medium of threedimensional model | |
| CN110611768A (en) | Multiple exposure photographic method and device | |
| Xiong et al. | An efficient underwater image enhancement model with extensive Beer-Lambert law | |
| Chen et al. | Focus manipulation detection via photometric histogram analysis | |
| CN117496019B (en) | Image animation processing method and system for driving static image | |
| Chugunov et al. | Shakes on a plane: Unsupervised depth estimation from unstabilized photography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |