CN111612699A - Image processing method, apparatus and computer-readable storage medium - Google Patents
Image processing method, apparatus and computer-readable storage medium Download PDFInfo
- Publication number
- CN111612699A CN111612699A CN201910130830.9A CN201910130830A CN111612699A CN 111612699 A CN111612699 A CN 111612699A CN 201910130830 A CN201910130830 A CN 201910130830A CN 111612699 A CN111612699 A CN 111612699A
- Authority
- CN
- China
- Prior art keywords
- image
- type
- images
- generator
- optimized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 63
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 27
- 238000005457 optimization Methods 0.000 claims abstract description 21
- 210000000697 sensory organ Anatomy 0.000 claims description 22
- 230000001815 facial effect Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 8
- 238000003708 edge detection Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 19
- 230000008569 process Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 4
- 210000003128 head Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G06T5/77—
Abstract
The disclosure provides an image processing method, an image processing device and a computer readable storage medium, and relates to the technical field of image processing. An image processing method of the present disclosure includes: acquiring a first image shot by an image acquisition device; a first generator of a countermeasure network generating a second image based on the first image, wherein the countermeasure network is generated for training with a set of first type images and a set of target type images; and the first generator performs total variation regularized optimization constraint on the second image to reduce the gradient and acquire an optimized second image. By the method, the countermeasure network can be trained, and the image shot by the image acquisition device is converted into the target type image; the image gradient can be reduced by utilizing the total variation regular optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a computer-readable storage medium.
Background
With the development and popularization of social networks, the creation of art and style processing on the basis of photos is popular, for example, a hand-drawing style head portrait based on real person self-photographing can attract a lot of eyes while protecting personal privacy.
At present, two techniques for generating a hand-drawing head portrait are mainly used, one is to use a method in the field of computer images to obtain each part by applying a theoretical segmentation of feature extraction to a human face, then to obtain corresponding styles of the parts by using technologies such as Non-Photorealistic Rendering (NPR), image geometric transformation and the like, and finally to recombine an image. Another relies on manual creation, where the artist manually adjusts and refines with the aid of image editing tools.
Disclosure of Invention
The inventor finds that although the automatic image processing method can quickly generate the image in the artistic form, the image effect, the quality and the like are not required; the manual method, although satisfactory for the user, costs a lot of time, money, manpower, and other resources.
It is an object of the present disclosure to improve the quality of automated image processing.
According to an aspect of the present disclosure, an image processing method is provided, including: acquiring a first image shot by an image acquisition device; a first generator of a countermeasure network generating a second image based on the first image, wherein the countermeasure network is generated for training with a set of first type images and a set of target type images; and the first generator performs total variation regularized optimization constraint on the second image to reduce the gradient and acquire an optimized second image.
In some embodiments, the first type of image is an image taken by an image acquisition device; and/or the target type image is a hand-drawn image.
In some embodiments, the first image is a facial image captured by the image capture device.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images includes: a first generator generating a second type of image from images in the set of first type of images; the first generator performs total variation regularized optimization constraint on the second type image to reduce the gradient and obtain an optimized second type image; and judging whether the optimized second type image belongs to the target type image or not by the discriminator of the countermeasure network until the judgment result shows that all the optimized second type images belong to the target type images.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images further comprises: the method comprises the steps that an anti-network carries out edge detection and Gaussian blur processing on images in a set of target type images, noise is added, and blurred target type images are obtained; the discriminator judges whether the fuzzy target type image belongs to the target type image or not until the judgment result shows that all the fuzzy target type images belong to the target type image.
In some embodiments, training the countermeasure network with the set of first type images and the set of target type images further comprises: generating a third type image according to the optimized second type image by a second generator, wherein the second generator is used for reconstructing the target type image into the first type image; and acquiring the difference value of the first type image generating the optimized second type image and the third type image generated based on the optimized second type image and transmitting the difference value in the opposite direction so as to reduce the difference value.
In some embodiments, the image processing method further comprises: performing facial feature detection on the image acquired by the first generator, and determining the position of the facial features; obtaining disparity values between a first type of image that generates an optimized second type of image and a third type of image that is generated based on the optimized second type of image comprises: and determining a difference value according to the difference of the first type image and the third type image at each relative pixel point position and the weight of the corresponding pixel point position, wherein the weight of the position of the five sense organs is greater than the weight of other positions.
By the method, the countermeasure network can be trained, and the image shot by the image acquisition device is converted into the target type image; the image gradient can be reduced by utilizing the total variation regular optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
According to an aspect of further embodiments of the present disclosure, there is provided an image processing apparatus including: an image acquisition unit configured to acquire a first image captured by an image capture device; a first generator configured to generate a second image based on a first image, wherein a countermeasure network to which the first generator belongs is generated by training with a set of first type images and a set of target type images; and performing total variation regularized optimization constraint on the second image to reduce the gradient and obtain an optimized second image.
In some embodiments, the first type of image is an image taken by an image acquisition device; and/or the target type image is a hand-drawn image.
In some embodiments, the first image is a facial image captured by the image capture device.
In some embodiments, the first generator is further configured to generate a second type of image from images in the set of first type images; performing total variation regularization optimization constraint on the second type image to reduce the gradient and obtain an optimized second type image; the image processing apparatus further includes: and the discriminator is configured to judge whether the optimized second type image generated by the first generator belongs to the target type image or not in the confrontation network training until all the judged results are that the optimized second type image belongs to the target type image.
In some embodiments, the image processing apparatus further comprises: the fuzzy processing unit is configured to perform edge detection and Gaussian fuzzy processing on the images in the set of target type images, add noise and acquire fuzzy target type images; the discriminator is further configured to judge whether the blurred target type image belongs to the target type image or not until the judgment result is that all the blurred target type images belong to the target type image.
In some embodiments, the image processing apparatus further comprises: a second generator configured to generate a third type of image from the optimized second type of image, wherein the second generator is used for reconstructing the target type of image into the first type of image; a disparity acquisition unit configured to acquire and reversely transfer disparity values of a first type image generating the optimized second type image and a third type image generated based on the optimized second type image to reduce the disparity values.
In some embodiments, the image processing apparatus further comprises: a facial feature position extraction unit configured to perform facial feature detection on the first image and determine the position of the facial feature; the difference acquisition unit is configured to determine a difference value according to the difference between the first type image and the third type image at each relative pixel point position and the weight of the corresponding pixel point position and transmit the difference value in a reverse direction to reduce the difference value, wherein the weight of the position of the five sense organs is greater than the weight of other positions.
According to an aspect of still further embodiments of the present disclosure, there is provided an image processing apparatus including: a memory; and a processor coupled to the memory, the processor configured to perform any of the image processing methods above based on instructions stored in the memory.
The image processing device can convert the image shot by the image acquisition device into a target type image by utilizing the trained confrontation network; the image gradient can be reduced by utilizing the total variation regular optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
According to an aspect of still further embodiments of the present disclosure, a computer-readable storage medium is proposed, on which computer program instructions are stored, which instructions, when executed by a processor, implement the steps of any of the image processing methods above.
By executing instructions on such a computer-readable storage medium, images captured by the image acquisition device can be converted into target-type images using the trained confrontational network; the image gradient can be reduced by utilizing the total variation regular optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and not to limit the disclosure. In the drawings:
fig. 1 is a flow diagram of some embodiments of an image processing method of the present disclosure.
Fig. 2 is a flow diagram of further embodiments of an image processing method of the present disclosure.
Fig. 3 is a flowchart of further embodiments of the image processing method of the present disclosure.
Fig. 4 is a schematic diagram of some embodiments of an image processing apparatus of the present disclosure.
Fig. 5 is a schematic diagram of other embodiments of an image processing apparatus of the present disclosure.
Fig. 6 is a schematic diagram of an image processing apparatus according to still other embodiments of the disclosure.
Fig. 7 is a schematic diagram of still other embodiments of an image processing apparatus of the present disclosure.
Detailed Description
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
A flow diagram of some embodiments of the image processing method of the present disclosure is shown in fig. 1.
In step 101, a first image captured by an image capture device is acquired. In some embodiments, the first image may be taken by a camera or may be taken from a photograph. In some embodiments, the first image may be a photographed image of the face of the user, such as a user self-photograph.
In step 102, the first generator generates a second image based on the first image, the first generator belongs to a confrontation network, the confrontation network is generated by using a set of first type images and a set of target type images, and the whole process is optimized through continuous training to finally obtain a stable generation model.
In step 103, the first generator of the countermeasure network applies a total variation regularized optimization constraint to the second image to reduce the gradient, and an optimized second image is obtained.
By the method, the countermeasure network can be trained, the image shot by the image acquisition device is converted into the target type image, and the image processing quality is improved. In addition, since the first image is formed by shooting, under the influence of angles and environments, interference factors such as uneven illumination and the like may cause color blocks in the image. Through the full-variation regular optimization constraint, the gradient distribution is more uniform, the noise is reduced, the influence of interference factors is avoided, the color is smoother and more uniform, and the overall beautiful result is obtained. Especially when processing the facial image, can make the face lines clean and neatly fall, clear pleasing to the eye, improve image quality.
In some embodiments, the image G generated by the first generator may be compared to the image generated by the second generatorAB(x) Adding a total variation constraint, namely:
In some embodiments, the set of images captured by the image capture device (i.e., the set of first type images), and the set of target type images (e.g., the set of hand-drawn style images) may be configured during the training process. Inputting a set of first type images into a first generator, wherein the first generator generates images, and after a total variation constraint, judging whether the generated images belong to a target type by a discriminator of a countermeasure network, wherein the training aim is to judge whether the generated images belong to the target type; a set of target type images may also be input to the discriminator, the goal of the training being that the input images belong to a target type.
By the method, the capacities of the first generator and the discriminator are continuously improved in the process of mutual confrontation and iteration, and when the capacity of the discriminator is finally improved to a certain degree and the data source is the first generator or the target type image cannot be judged, the image processing capacity of the first generator is determined to meet the requirement.
A flow diagram of further embodiments of the image processing method of the present disclosure is shown in fig. 2.
In step 201, the first generator generates a second type image according to images in the set of first type images, and performs a total variation regularization optimization constraint on the second type image to obtain an optimized second type image.
In step 202, the countermeasure network performs edge detection and gaussian blurring processing on the images in the set of target type images, adds noise, and obtains a blurred target type image.
And (3) randomly inputting the optimized second type image in the step 201 and the fuzzy target type image in the step 202 into a discriminator of the countermeasure network for judgment.
In step 203, the discriminator of the countermeasure network determines whether the optimized second type image, the blurred target type image, belongs to the target type image. And the discriminator feeds the judgment result back to the first generator until the judgment result indicates that all the fuzzy target type images belong to the target type images, and optimizes the second type images to belong to the target type images.
In some embodiments, the data input to the arbiter may include:
wherein A is a set of first type images, B is a set of target type images, Z is a set of blurred target type images, and for an image x belonging to A, an image y belonging to B, an image Z belonging to Z,
the representation takes the element x in the set a, and so on.
By the method, noise can be generated for the target type image needing to be judged in the discriminator, so that the noise identification capability of the discriminator is improved, the noise reduction capability of the generator is enhanced, and the image generated by the first generator is clearer and cleaner.
In some embodiments, two generators, a first generator G, may be included in the countermeasure networkABIs to convert the first type image into the target type, a second generator GBAFor based on GABThe generated image generates a first type of image. In some embodiments, the generator may employ a Unet256 architecture and the arbiter may employ PatchGAN.
In some embodiments, two discriminators, a first discriminator D, may be included in the countermeasure networkBFor judging GABWhether the generated image is a target type image, a second discriminator DAFor judging GBAWhether the generated image is a first type image. Inputting the set of target type images into a second generator, the second generator randomly generating images, from DAJudging whether the generated image belongs to a first type, wherein the training aims to judge that the generated image belongs to the first type; a set of images of the first type is also input DAThe goal of the training is that the input image is of a first type. By the method, the capacities of the second generator and the second discriminator are continuously improved in the process of mutual confrontation and iteration, and when the capacity of the second discriminator is improved to a certain degree finally and the data source cannot be judged to be the second generator or the first type image, the image processing capacity of the second generator is determined to meet the requirement.
In some embodiments, a first type image F1 may be input to the first generator, an output input to the second type image F2, and an input to the second generator F2, resulting in an image F3 of the third type; comparing F1 and F3, the difference between the images is obtained, and the difference is gradually reduced in the training process, so that the capacities of the first and second generators are optimized.
A flow chart of still further embodiments of the image processing method of the present disclosure is shown in fig. 3.
In step 301, the images obtained by the first generator are detected by the five sense organs, and the positions of the five sense organs are determined. In some embodiments, five sense organs detection may be performed on the image, such as:
E(x)＝[bbox1,…bboxj，…bbox0]，
wherein X ∈ X, E (X) is a method for detecting human face features, bboxjAnd (c) position information indicating the position of the five sense organ parts, specifically, the coordinates of the upper left vertex and the lower right vertex of the rectangular frame, and k is the number of detection parts. If the number k is larger, the resulting five sense organs are more refined. For example, k is 3, which represents three facial parts of the eyes, nose, and mouth, and after the detection is completed, the position information is stored.
In step 302, the first generator generates a second type of image from images in the set of first type images. And performing total variation regularization optimization constraint on the second type image to reduce the gradient and obtain an optimized second type image.
In step 303, the second generator generates a third type of image from the optimized second type of image.
In step 304, a difference value is determined according to the difference between the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, wherein the weight of the position of the five sense organs is greater than the weight of other positions. In some embodiments, the formula may be employed:
and determining the pixel-by-pixel distance of the two graphs. In the scene of generating the hand-drawn image of the face image, the fine granularity generated by the five sense organ parts directly determines the aesthetic property of the result, and in addition to the constraint of the whole image, the generation of single components of the eyes, the nose, the mouth and the like of the face is more concerned, so that the network generates more exquisite local features. At one endIn some embodiments, k may be 4,
respectively represent the whole picture, the eyes, the nose and the mouth,
is provided by step 301, i is the picture identification. The weight λ can be configured for each position as requiredjThe different weights represent the attention degree of the network, and the larger the weight is, the more attention of the network to the part is shown, and the finer the part is generated after the training is completed.
By the method, the part of the five sense organs can be focused more, the image of the part of the five sense organs is finer, and the definition and the accuracy of the part of the five sense organs in the image generated by the first generator are highlighted.
A schematic diagram of some embodiments of the image processing apparatus of the present disclosure is shown in fig. 4. The image acquisition unit 401 can acquire a first image captured by the image capture device. In some embodiments, the first image may be taken by a camera or may be taken from a photograph. In some embodiments, the first image may be a photographed image of the face of the user, such as a user self-photograph.
The first generator 402 is capable of generating a second image based on a first image, the first generator being attributed to a confrontation network generated for training with a set of images of a first type and a set of images of a target type; and then a first generator of the countermeasure network performs total variation regularized optimization constraint on the second image to reduce the gradient and obtain an optimized second image.
The image processing device can convert the image shot by the image acquisition device into a target type image by utilizing the trained confrontation network; the image gradient can be reduced by utilizing the total variation regular optimization constraint, the influence of color blocks on the image processing effect is avoided, and the image processing quality is improved.
Schematic diagrams of further embodiments of the image processing apparatus of the present disclosure are shown in fig. 5. In the training process, a set of first type images is input into a first generator 501, the first generator 502 generates second type images, and total variation regularization optimization constraint is performed on the second type images to obtain optimized second type images. Judging whether the generated optimized second type image belongs to a target type image by the discriminator 503 of the countermeasure network, the training being aimed at judging whether the generated image belongs to the target type image; the set of target type images is also input to the discriminator 503, the goal of the training being that the input images belong to a target type image.
In the training process, the capabilities of the first generator and the discriminator are continuously improved in the process of mutual confrontation and iteration, when the capability of the discriminator is finally improved to a certain degree and the data source is the first generator or the target type image cannot be judged, the image processing capability of the first generator is determined to meet the requirement, and after the image acquisition unit 501 inputs the acquired image into the first generator, the image belonging to the target type can be obtained.
In some embodiments, as shown in fig. 5, the image processing apparatus may further include a blurring processing unit 504, which is capable of performing blurring processing, such as edge detection and gaussian blurring processing, on images in a set of target type images prepared in advance, adding noise, and obtaining a blurred target type image. The blur processing unit 504 inputs the blurred target type image to the discriminator 503, and the discriminator determines whether or not the image belongs to the target type image, the target being the image determined to belong to the target type. The image processing device can improve the noise identification capability of the discriminator and enhance the noise reduction capability of the generator, thereby enabling the image generated by the first generator to be clearer and cleaner.
In some embodiments, as shown in fig. 5, the image processing apparatus may further include a second generator 505 and a difference acquisition unit 506. The second generator 505 is capable of generating a first type of image from a second type of image generated by the first generator 502. In some embodiments, a first type image F1 may be input to the first generator 502, an image F2 that is an optimized second type image may be output, and F2 may be input to the second generator 505, resulting in an image F3 that is a third type image; the difference obtaining unit 506 compares F1 and F3 to obtain the difference between the images, gradually reduces the difference in the training process, and optimizes the capabilities of the first and second generators, thereby improving the degree of fit between the image generated by the first generator 502 and the input image, and when the difference is used for generating the head portrait hand-drawn image, the degree of restoration of the shot head portrait can be improved, and the image generation effect is optimized.
In some embodiments, in order to highlight the lines of the five sense organs and blur other parts, a five sense organ position extracting unit 507 may be further included, which is capable of performing five sense organs detection on the image acquired by the first generator to determine the positions of the five sense organs. The difference obtaining unit 506 determines a difference value according to the difference between the first type image and the third type image at each relative pixel position and the weight of the corresponding pixel position, wherein the weight of the position of the five sense organs is greater than the weight of other positions, so that the part focusing on the five sense organs is realized, the image of the part of the five sense organs is finer, and the definition and the accuracy of the part of the five sense organs in the image generated by the first generator are highlighted.
A schematic structural diagram of some embodiments of the image processing apparatus of the present disclosure is shown in fig. 6. The image processing apparatus includes a memory 601 and a processor 602. Wherein: the memory 601 may be a magnetic disk, flash memory, or any other non-volatile storage medium. The memory is for storing instructions in the corresponding embodiments of the image processing method above. Processor 602 is coupled to memory 601 and may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 602 is configured to execute instructions stored in the memory, so as to avoid color blocks from affecting image processing effects and improve image processing quality.
In some embodiments, as also shown in fig. 7, the image processing apparatus 700 includes a memory 701 and a processor 702. Processor 702 is coupled to memory 701 by a BUS BUS 703. The image processing apparatus 700 may be connected to an external storage device 705 through a storage interface 704 to call external data, and may be connected to a network or another computer system (not shown) through a network interface 706. And will not be described in detail herein.
In the embodiment, the data instruction is stored in the memory, and the instruction is processed by the processor, so that the influence of color blocks on the image processing effect can be avoided, and the image processing quality is improved.
In further embodiments, a computer-readable storage medium has stored thereon computer program instructions which, when executed by a processor, implement the steps of the method in the corresponding embodiments of the image processing method. As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Thus far, the present disclosure has been described in detail. Some details that are well known in the art have not been described in order to avoid obscuring the concepts of the present disclosure. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
Finally, it should be noted that: the above examples are intended only to illustrate the technical solutions of the present disclosure and not to limit them; although the present disclosure has been described in detail with reference to preferred embodiments, those of ordinary skill in the art will understand that: modifications to the specific embodiments of the disclosure or equivalent substitutions for parts of the technical features may still be made; all such modifications are intended to be included within the scope of the claims of this disclosure without departing from the spirit thereof.
Claims (16)
1. An image processing method comprising:
acquiring a first image shot by an image acquisition device;
a first generator of a countermeasure network generating a second image based on the first image, wherein the countermeasure network is generated for training with a set of first type images and a set of target type images;
and the first generator performs full variation regularized optimization constraint on the second image to reduce the gradient and obtain an optimized second image.
2. The method of claim 1, wherein the first type of image is an image taken by an image acquisition device; and/or the target type image is a hand-drawn image.
3. The method of claim 1, wherein the first image is a facial image captured by an image capture device.
4. The method of claim 1 or 3, wherein training the countermeasure network with a set of first type images and a set of target type images comprises:
the first generator generating a second type of image from images in a set of first type of images;
the first generator performs total variation regularized optimization constraint on the second type image to reduce gradient and obtain an optimized second type image;
and the discriminator of the countermeasure network judges whether the optimized second type image belongs to the target type image or not until the judgment result shows that all the optimized second type images belong to the target type image.
5. The method of claim 4, wherein the training the countermeasure network with the set of first type images and the set of target type images further comprises:
the countermeasure network carries out edge detection and Gaussian blur processing on images in the set of target type images, noise is added, and a blurred target type image is obtained;
and the discriminator judges whether the fuzzy target type image belongs to the target type image or not and feeds back the judgment result to the first generator until the judgment result indicates that all the fuzzy target type images belong to the target type image.
6. The method of claim 4, wherein the training the countermeasure network with the set of first type images and the set of target type images further comprises:
a second generator for generating a third type image from the optimized second type image, wherein the second generator is used for reconstructing the target type image into the first type image;
and acquiring a difference value between the first type image generating the optimized second type image and the third type image generated based on the optimized second type image and transmitting the difference value in a reverse direction so as to reduce the difference value.
7. The method of claim 6, further comprising: performing facial feature detection on the image acquired by the first generator, and determining the position of the facial features;
the obtaining disparity values between a first type of image that generates the optimized second type of image and a third type of image that is generated based on the optimized second type of image comprises:
and determining the difference value according to the difference of the first type image and the third type image at each relative pixel point position and the weight of the corresponding pixel point position, wherein the weight of the position of the five sense organs is greater than the weight of other positions.
8. An image processing apparatus comprising:
an image acquisition unit configured to acquire a first image captured by an image capture device;
a first generator configured to generate a second image based on the first image, wherein a countermeasure network to which the first generator belongs is generated for training with a set of first type images and a set of target type images; and performing total variation regularized optimization constraint on the second image to reduce the gradient and obtain an optimized second image.
9. The apparatus of claim 8, wherein the first type of image is an image taken by an image acquisition device; and/or the target type image is a hand-drawn image.
10. The apparatus of claim 8, wherein the first image is a facial image captured by an image capture device.
11. The apparatus according to claim 8 or 10, wherein the first generator is further configured to generate a second type of image from images of the set of first type images; performing total variation regularized optimization constraint on the first image to reduce gradient and obtain an optimized second image;
further comprising:
a discriminator, configured to judge whether the optimized second type image generated by the first generator belongs to the target type image in confrontation network training until all judgment results indicate that the optimized second type image belongs to the target type image.
12. The apparatus of claim 11, further comprising:
the fuzzy processing unit is configured to perform edge detection and Gaussian fuzzy processing on the images in the set of target type images, add noise and acquire fuzzy target type images;
the discriminator is further configured to judge whether the blurred target type image belongs to the target type image or not until all the blurred target type images belong to the target type image as a result of the judgment.
13. The apparatus of claim 11, further comprising:
a second generator configured to generate a third type of image from the optimized second type of image, wherein the second generator is configured to reconstruct the target type of image into the first type of image;
a disparity acquisition unit configured to acquire disparity values of a first type image generating the optimized second type image and a third type image generated based on the optimized second type image and to transfer the disparity values in reverse to reduce the disparity values.
14. The apparatus of claim 13, further comprising:
a facial feature position extraction unit configured to perform facial feature detection on the image acquired by the first generator and determine the facial feature position;
the difference acquisition unit is configured to determine the difference value according to the difference of the first type image and the third type image at each relative pixel point position and the weight of the corresponding pixel point position, and transmit the difference value in the reverse direction to reduce the difference value, wherein the weight of the position of the five sense organs is greater than the weight of other positions.
15. An image processing apparatus comprising:
a memory; and
a processor coupled to the memory, the processor configured to perform the method of any of claims 1-7 based on instructions stored in the memory.
16. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910130830.9A CN111612699A (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910130830.9A CN111612699A (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111612699A true CN111612699A (en) | 2020-09-01 |
Family
ID=72201350
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910130830.9A Pending CN111612699A (en) | 2019-02-22 | 2019-02-22 | Image processing method, apparatus and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111612699A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304765A (en) * | 2017-12-11 | 2018-07-20 | 中国科学院自动化研究所 | Multitask detection device for face key point location and semantic segmentation |
| CN108537743A (en) * | 2018-03-13 | 2018-09-14 | 杭州电子科技大学 | A kind of face-image Enhancement Method based on generation confrontation network |
| CN108615073A (en) * | 2018-04-28 | 2018-10-02 | 北京京东金融科技控股有限公司 | Image processing method and device, computer readable storage medium, electronic equipment |
| CN108846793A (en) * | 2018-05-25 | 2018-11-20 | 深圳市商汤科技有限公司 | Image processing method and terminal device based on image style transformation model |
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN109190722A (en) * | 2018-08-06 | 2019-01-11 | 大连民族大学 | Font style based on language of the Manchus character picture migrates transform method |
| CN109191402A (en) * | 2018-09-03 | 2019-01-11 | 武汉大学 | The image repair method and system of neural network are generated based on confrontation |
| US20190035118A1 (en) * | 2017-07-28 | 2019-01-31 | Shenzhen United Imaging Healthcare Co., Ltd. | System and method for image conversion |
| US20190049540A1 (en) * | 2017-08-10 | 2019-02-14 | Siemens Healthcare Gmbh | Image standardization using generative adversarial networks |
-
2019
- 2019-02-22 CN CN201910130830.9A patent/CN111612699A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190035118A1 (en) * | 2017-07-28 | 2019-01-31 | Shenzhen United Imaging Healthcare Co., Ltd. | System and method for image conversion |
| US20190049540A1 (en) * | 2017-08-10 | 2019-02-14 | Siemens Healthcare Gmbh | Image standardization using generative adversarial networks |
| CN108875766A (en) * | 2017-11-29 | 2018-11-23 | 北京旷视科技有限公司 | Method, apparatus, system and the computer storage medium of image procossing |
| CN108304765A (en) * | 2017-12-11 | 2018-07-20 | 中国科学院自动化研究所 | Multitask detection device for face key point location and semantic segmentation |
| CN108537743A (en) * | 2018-03-13 | 2018-09-14 | 杭州电子科技大学 | A kind of face-image Enhancement Method based on generation confrontation network |
| CN108615073A (en) * | 2018-04-28 | 2018-10-02 | 北京京东金融科技控股有限公司 | Image processing method and device, computer readable storage medium, electronic equipment |
| CN108846793A (en) * | 2018-05-25 | 2018-11-20 | 深圳市商汤科技有限公司 | Image processing method and terminal device based on image style transformation model |
| CN109190722A (en) * | 2018-08-06 | 2019-01-11 | 大连民族大学 | Font style based on language of the Manchus character picture migrates transform method |
| CN109191402A (en) * | 2018-09-03 | 2019-01-11 | 武汉大学 | The image repair method and system of neural network are generated based on confrontation |
Non-Patent Citations (1)
| Title |
|---|
| 陈梓炜: "基于深度特征插值的图像变换方法研究与实现", 《中国优秀硕博士学位论文 信息科技辑》, 15 August 2018 (2018-08-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wadhwa et al. | Synthetic depth-of-field with a single-camera mobile phone | |
| Liu et al. | Fast burst images denoising | |
| Wang et al. | Selective light field refocusing for camera arrays using bokeh rendering and superresolution | |
| Gallo et al. | Artifact-free high dynamic range imaging | |
| Hu et al. | Deblurring low-light images with light streaks | |
| US20210082097A1 (en) | Image generation device, image generation method, and storage medium storing program | |
| CN106899781B (en) | Image processing method and electronic equipment | |
| Raman et al. | Reconstruction of high contrast images for dynamic scenes | |
| Li et al. | Detail-preserving multi-exposure fusion with edge-preserving structural patch decomposition | |
| Lin et al. | Real photographs denoising with noise domain adaptation and attentive generative adversarial network | |
| CN108830892B (en) | Face image processing method and device, electronic equipment and computer readable storage medium | |
| CN107025660B (en) | Method and device for determining image parallax of binocular dynamic vision sensor | |
| CN105243371A (en) | Human face beauty degree detection method and system and shooting terminal | |
| JP2010045613A (en) | Image identifying method and imaging device | |
| TW200840365A (en) | Motion-blur degraded image restoration method | |
| JP2010244398A (en) | Image processing apparatus and image processing program | |
| Zhang et al. | Deep motion blur removal using noisy/blurry image pairs | |
| Botezatu et al. | Fun selfie filters in face recognition: Impact assessment and removal | |
| Chen et al. | Focus manipulation detection via photometric histogram analysis | |
| Chen et al. | Hybrid saliency detection for images | |
| Kınlı et al. | Modeling the lighting in scenes as style for auto white-balance correction | |
| Jung et al. | Multispectral fusion of rgb and nir images using weighted least squares and convolution neural networks | |
| US20220398704A1 (en) | Intelligent Portrait Photography Enhancement System | |
| CN111612699A (en) | Image processing method, apparatus and computer-readable storage medium | |
| CN111062904B (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |