CN106570911B - Method for synthesizing facial cartoon based on daisy descriptor - Google Patents
Method for synthesizing facial cartoon based on daisy descriptor Download PDFInfo
- Publication number
- CN106570911B CN106570911B CN201610753192.2A CN201610753192A CN106570911B CN 106570911 B CN106570911 B CN 106570911B CN 201610753192 A CN201610753192 A CN 201610753192A CN 106570911 B CN106570911 B CN 106570911B
- Authority
- CN
- China
- Prior art keywords
- image
- pixel point
- cartoon
- synthesized
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 235000005633 Chrysanthemum balsamita Nutrition 0.000 title claims abstract description 18
- 230000001815 facial effect Effects 0.000 title claims abstract description 14
- 230000002194 synthesizing effect Effects 0.000 title claims abstract description 14
- 241000723353 Chrysanthemum Species 0.000 title claims abstract description 12
- 239000013598 vector Substances 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 16
- 238000006073 displacement reaction Methods 0.000 claims abstract description 9
- 241000132023 Bellis perennis Species 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 239000002131 composite material Substances 0.000 claims 1
- 230000015572 biosynthetic process Effects 0.000 abstract description 8
- 238000003786 synthesis reaction Methods 0.000 abstract description 8
- 230000000694 effects Effects 0.000 description 6
- 238000002939 conjugate gradient method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method for synthesizing facial cartoon based on daisy descriptor, which comprises the following steps: 1) acquiring the gray value of each pixel point in the face image to be synthesized; 2) establishing a day descriptor for each pixel point in the face image to be synthesized, extracting image block characteristics, and establishing a day descriptor for each pixel point of each face cartoon image in a training set; 3) adopting a patchmatch algorithm in pixel points of the face cartoon image in the training set to obtain K candidate pixel positions most similar to each pixel point of the face image to be synthesized; 4) obtaining K candidate values according to the corresponding displacement vectors, and giving weights to the K candidate values; 5) and acquiring a weighted value by adopting a conjugate gradient resolver, and synthesizing the cartoon image of the face to be synthesized by an RGB value of the cartoon image in the training set by adopting an SSD noise reduction method according to the weighted value. Compared with the prior art, the method has the advantages of high similarity, accurate synthesis and the like.
Description
Technical Field
The invention relates to the technical field of image processing and analysis, in particular to a method for synthesizing facial cartoon based on a day descriptor.
Background
The synthesis of facial cartoons has found widespread use in digital entertainment, and a great deal of research and commercial products have been devoted to the synthesis of facial cartoons. Although the styles are different, the generation of a high-quality and high-similarity facial cartoon image is pursued by all the works.
Currently, the synthesis of face sketch images has achieved good effect, and there are basically two methods for general sketch synthesis: image-based methods and example-based methods, image-based sketch synthesis methods generally cannot capture important facial details, while example-based methods reconstruct new sketch images from existing sketches, but require a lot of examples and have poor noise reduction effects, and the synthesized images are not accurate.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a method for synthesizing facial cartoon based on daisy descriptor, which has high similarity and accurate synthesis.
The purpose of the invention can be realized by the following technical scheme:
a method for synthesizing facial cartoon based on daisy descriptor includes the following steps:
1) acquiring the gray value of each pixel point in the face image to be synthesized, and storing the gray value in a two-dimensional matrix;
2) establishing a day descriptor for each pixel point in the face image to be synthesized, extracting image block characteristics, and establishing a day descriptor for each pixel point of each face cartoon image in a training set;
3) adopting a patchmatch algorithm in pixel points of the face cartoon image in the training set to obtain K candidate pixel positions most similar to each pixel point of the face image to be synthesized, and obtaining corresponding displacement vectors;
4) obtaining K candidate values according to the corresponding displacement vectors, and giving weights to the K candidate values to enable the K candidate values to be linearly combined into an input image block;
5) and acquiring a weighted value by adopting a conjugate gradient resolver, and synthesizing the cartoon image of the face to be synthesized by an RGB value of the cartoon image in the training set by adopting an SSD noise reduction method according to the weighted value.
The face image to be synthesized is equal to each face cartoon image in the training set in size, and the resolution ratio is the same.
In the step 2), the construction method of the daisy descriptor comprises the following steps:
21) selecting parameters of a daisy descriptor, including the farthest radius from a central pixel point, the number of convolution layers in each direction, the number of gradient directions in each layer and the number of straight bars of a gradient histogram;
22) calculating a plurality of orientation maps of the face image, and acquiring a plurality of corresponding convolution orientation maps by adopting a plurality of times of Gaussian kernel convolution;
23) synthesizing a plurality of convolution square maps into a vector hΣ(u, v), and obtain the daisy descriptor.
In the step 5), the calculation formula for obtaining the weighted value is as follows:
wherein,
is a weighted value, TpIs a vector containing the pixel values of the face image to be synthesized,
is a vector that includes K candidate picture pixel values.
In said step 5), the resultant value at the pixel position P
Calculated by the following formula:
wherein, | ΨpL is the image block ΨpThe number of pixels in (a) is,
and estimating a pixel point p for the pixel point q.
Compared with the prior art, the invention has the following advantages:
firstly, the similarity is high, and the synthesis is accurate: according to the method, the daisy descriptor is used for selecting the pixel point with the highest similarity with the picture to be synthesized, the image block is synthesized in a weighting mode, and then noise reduction synthesis is carried out through an SSD algorithm, so that the accuracy of the synthesized image is high.
Drawings
FIG. 1 is a flow chart of the method of operation of the present innovation.
Fig. 2 shows the original and the synthesized cartoon image of the embodiment, wherein the left image is the original and the right image is the synthesized cartoon image.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments.
Example (b):
as shown in fig. 1, the present embodiment includes the following steps:
first, an input image is loaded and its grayscale matrix is saved. In this example, the size of both our training set picture and the input picture are 500 x 360 pixels. In our training set, there are a total of 68 pairs of photo-cartoons.
And secondly, creating a Daisy descriptor, and calculating K-NN by using a patch match for each pixel point of the input image to obtain K candidate pixels and obtain corresponding displacement vectors. In this example, the value of K is 5, and the parameters of the Daisy descriptor are chosen as follows: the radius R farthest from the center pixel point is taken as 15, the number Q of convolution layers in each direction is taken as 3, the number T of gradient directions in each layer is taken as 8, and the number H of straight bars in a gradient histogram is taken as 8. The adopted parameter values are a group of values which are obtained after a plurality of experiments and have better average effect and average power consumption.
For an input image, we first calculate H orientation maps G by the formula
Where I denotes the input image, o denotes the calculated orientation (a total of 8 orientations), (a)+Denotes max (a, 0). Then convolving them with Gaussian kernel for several times to obtain convolution square map
Then, the convolution direction maps of all directions are combined into a vector as follows:
where (u, v) represents the location of the pixel. Thus descriptor D (u) of the entire Daisy0,v0) It is obtained.
Where Ij (u, v, R) represents a position in the j direction at a distance R from (u, v).
And then, for the Daisy local feature descriptors of the input pictures, performing batch matching on the Daisy local feature descriptors of the training set pictures one by one to obtain pixel positions with high similarity in each training set picture, selecting the most similar K values from the pixel positions, and storing the displacements in a displacement vector to record K candidate pixel positions.
And thirdly, obtaining K candidate values based on the calculated displacement vector, giving weights to the K candidate values to enable the K candidate values to be linearly combined into an input image block, and then calculating to obtain a weighted value by using a conjugate gradient solver. We want to solve the following linear equation to obtain the weighted values, namely
Where Tp represents a vector including input picture pixel values,
representing a vector containing K candidate picture pixel values.
We can solve this equation set to the desired weight value very efficiently using the conjugate gradient method.
And fourthly, synthesizing the target cartoon image by using the RGB value of the cartoon image in the training set by using an SSD noise reduction method according to the calculated coefficient. The resultant value at pixel position P
Calculated by the following formula
Where | ΨpI denotes the image block ΨpThe number of pixels in (1).
The estimation of the pixel point p by the pixel point q is shown, and the average of the estimation is the final result of the estimation. And the calculation of these estimates requiresAnd weighting the pixel values of the K candidate pictures at the pixel point q by using the weighting value finally obtained in the last step.
Effects of the implementation
According to the steps, the tested pictures are tested, and finally the cartoon image with good effect is synthesized. The result shows that the algorithm can achieve certain effect when the face photos are synthesized. The comparison of the test results is shown in fig. 2. It can be seen that the resultant map obtained according to our algorithm is still more similar to the original as a whole, but the resultant in some detail is also coarser. We believe that better results are expected from further attempts to modify some of the parameters, and that, because of the relatively small size of the data set and the single style, we consider trying a larger and more stylish cartoon image data set and expect a good improvement in the results.
Claims (4)
1. A method for synthesizing facial cartoon based on daisy descriptor is characterized by comprising the following steps:
1) acquiring the gray value of each pixel point in the face image to be synthesized, and storing the gray value in a two-dimensional matrix;
2) the method comprises the following steps of establishing a day descriptor for each pixel point in a face image to be synthesized, extracting image block features, and establishing the day descriptor for each pixel point of each face cartoon image in a training set, wherein the method for establishing the day descriptor comprises the following steps:
21) selecting parameters of a daisy descriptor, including the farthest radius from a central pixel point, the number of convolution layers in each direction, the number of gradient directions in each layer and the number of straight bars of a gradient histogram;
22) calculating a plurality of orientation maps of the face image, and acquiring a plurality of corresponding convolution orientation maps by adopting a plurality of times of Gaussian kernel convolution;
23) synthesizing a plurality of convolution square maps into a vector hΣ(u, v) thereofAnd (u, v) represents the position of the pixel point, and obtains a Daisy descriptor, wherein the parameters of the Daisy descriptor are selected as follows:
the radius R farthest from the central pixel point is taken as 15, the number Q of the convolution layers in each direction is taken as 3, the number T of the gradient directions of each layer is taken as 8, and the number H of straight bars of a gradient histogram is taken as 8;
for an input image, H direction maps G are calculated firstly, and the calculation formula is
Wherein I represents the input image, o represents the calculated orientation, a total of 8 orientations, (a)+Denotes max (a, 0);
3) adopting a patchmatch algorithm in pixel points of the face cartoon image in the training set to obtain K candidate pixel positions most similar to each pixel point of the face image to be synthesized, and obtaining corresponding displacement vectors;
4) obtaining K candidate values according to the corresponding displacement vectors, and giving weights to the K candidate values to enable the K candidate values to be linearly combined into an input image block;
5) and acquiring a weighted value by adopting a conjugate gradient resolver, and synthesizing the cartoon image of the face to be synthesized by an RGB value of the cartoon image in the training set by adopting an SSD noise reduction method according to the weighted value.
2. The method as claimed in claim 1, wherein the facial cartoon image to be synthesized has the same size and resolution as each facial cartoon image in the training set.
3. The method for synthesizing facial cartoon based on daisy descriptor as claimed in claim 1, wherein in step 5), the calculation formula for obtaining the weighting value is:
wherein,
is a weighted value, TpIs a vector containing the pixel values of the face image to be synthesized,
for a vector containing K candidate picture pixel values, i ∈ [1, K]Representing the ith candidate picture and p representing a pixel point.
4. The method as claimed in claim 1, wherein the step 5) is performed by using a composite value of the pixel position P
Calculated by the following formula:
wherein, | ΨpL is the image block ΨpThe number of pixels in (a) is,
and estimating a pixel point p for the pixel point q.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610753192.2A CN106570911B (en) | 2016-08-29 | 2016-08-29 | Method for synthesizing facial cartoon based on daisy descriptor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610753192.2A CN106570911B (en) | 2016-08-29 | 2016-08-29 | Method for synthesizing facial cartoon based on daisy descriptor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106570911A CN106570911A (en) | 2017-04-19 |
| CN106570911B true CN106570911B (en) | 2020-04-10 |
Family
ID=58532363
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610753192.2A Active CN106570911B (en) | 2016-08-29 | 2016-08-29 | Method for synthesizing facial cartoon based on daisy descriptor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106570911B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108428232B (en) * | 2018-03-20 | 2019-07-19 | 合肥工业大学 | A kind of blind appraisal procedure of cartoon image quality |
| CN109920021B (en) * | 2019-03-07 | 2023-05-23 | 华东理工大学 | Face sketch synthesis method based on regularized width learning network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1870049A (en) * | 2006-06-15 | 2006-11-29 | 西安交通大学 | Human face countenance synthesis method based on dense characteristic corresponding and morphology |
| CN102682420A (en) * | 2012-03-31 | 2012-09-19 | 北京百舜华年文化传播有限公司 | Method and device for converting real character image to cartoon-style image |
| CN103218427A (en) * | 2013-04-08 | 2013-07-24 | 北京大学 | Local descriptor extracting method, image searching method and image matching method |
| CN103559488A (en) * | 2013-11-13 | 2014-02-05 | 中南大学 | Cartoon figure facial feature extraction method based on qualitative space relation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW577227B (en) * | 2002-04-23 | 2004-02-21 | Ind Tech Res Inst | Method and apparatus for removing background of visual content |
-
2016
- 2016-08-29 CN CN201610753192.2A patent/CN106570911B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1870049A (en) * | 2006-06-15 | 2006-11-29 | 西安交通大学 | Human face countenance synthesis method based on dense characteristic corresponding and morphology |
| CN102682420A (en) * | 2012-03-31 | 2012-09-19 | 北京百舜华年文化传播有限公司 | Method and device for converting real character image to cartoon-style image |
| CN103218427A (en) * | 2013-04-08 | 2013-07-24 | 北京大学 | Local descriptor extracting method, image searching method and image matching method |
| CN103559488A (en) * | 2013-11-13 | 2014-02-05 | 中南大学 | Cartoon figure facial feature extraction method based on qualitative space relation |
Non-Patent Citations (1)
| Title |
|---|
| 基于DAISY 描述符和改进型权重核的快速局部立体匹配;刘天亮 等;《南京邮电大学学报》;20120831;第32卷(第4期);70-76 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106570911A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Scene-adaptive remote sensing image super-resolution using a multiscale attention network | |
| CN107330439B (en) | Method for determining posture of object in image, client and server | |
| Zhang et al. | Learning multiple linear mappings for efficient single image super-resolution | |
| Ren et al. | Single image super-resolution via adaptive high-dimensional non-local total variation and adaptive geometric feature | |
| JP3837575B2 (en) | Speeding up of super-resolution processing | |
| CN108073857A (en) | The method and device of dynamic visual sensor DVS event handlings | |
| CN107749987B (en) | Digital video image stabilization method based on block motion estimation | |
| US9247139B2 (en) | Method for video background subtraction using factorized matrix completion | |
| CN104933678B (en) | A kind of image super-resolution rebuilding method based on image pixel intensities | |
| US11915350B2 (en) | Training one-shot instance segmenters using synthesized images | |
| CN109376641B (en) | Moving vehicle detection method based on unmanned aerial vehicle aerial video | |
| CN105513033B (en) | A kind of super resolution ratio reconstruction method that non local joint sparse indicates | |
| CN111709307B (en) | Resolution enhancement-based remote sensing image small target detection method | |
| CN106934398B (en) | Image de-noising method based on super-pixel cluster and rarefaction representation | |
| CN104504672B (en) | Low-rank sparse neighborhood insertion ultra-resolution method based on NormLV features | |
| CN107862680A (en) | A kind of target following optimization method based on correlation filter | |
| CN116883588A (en) | Method and system for quickly reconstructing three-dimensional point cloud under large scene | |
| CN112598604A (en) | Blind face restoration method and system | |
| CN107392211B (en) | Salient target detection method based on visual sparse cognition | |
| Wan et al. | Drone image stitching using local mesh-based bundle adjustment and shape-preserving transform | |
| Zhou et al. | PADENet: An efficient and robust panoramic monocular depth estimation network for outdoor scenes | |
| Wang et al. | Group shuffle and spectral-spatial fusion for hyperspectral image super-resolution | |
| CN106570911B (en) | Method for synthesizing facial cartoon based on daisy descriptor | |
| CN115564639A (en) | Background blurring method and device, computer equipment and storage medium | |
| CN114663880A (en) | Three-dimensional target detection method based on multi-level cross-modal self-attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |