CN106780367B - HDR photo style transfer method dictionary-based learning - Google Patents
HDR photo style transfer method dictionary-based learning Download PDFInfo
- Publication number
- CN106780367B CN106780367B CN201611058620.6A CN201611058620A CN106780367B CN 106780367 B CN106780367 B CN 106780367B CN 201611058620 A CN201611058620 A CN 201611058620A CN 106780367 B CN106780367 B CN 106780367B
- Authority
- CN
- China
- Prior art keywords
- photo
- color
- hdr
- dictionary
- gradient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012546 transfer Methods 0.000 title claims abstract description 52
- 238000012549 training Methods 0.000 claims abstract description 29
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000013507 mapping Methods 0.000 claims description 15
- 238000005457 optimization Methods 0.000 claims description 13
- 238000010422 painting Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 239000000284 extract Substances 0.000 claims description 7
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 230000008901 benefit Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 239000004615 ingredient Substances 0.000 claims description 3
- 238000010428 oil painting Methods 0.000 claims description 3
- 238000011282 treatment Methods 0.000 claims description 3
- 231100000289 photo-effect Toxicity 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 7
- 230000002708 enhancing effect Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 102100032352 Leukemia inhibitory factor Human genes 0.000 description 1
- 108090000581 Leukemia inhibitory factor Proteins 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of HDR photo style transfer methods dictionary-based learning.Photo is referred to by a given HDR, by color transfer and dictionary learning, HDR style and features are transferred on the photo of source, to automatically generate HDR photo effect.HDR is transferred on the photo of source by the color transfer method for comprising the following steps that (1) and keeping by gradient of this method with reference to the color characteristic of photo；(2) minutia is extracted with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, forms the excessively complete wordbook of details；(3) sparse reconstruction is carried out to source photo using the excessively complete wordbook of details, generates and refers to the consistent details of photo eigen with HDR；(4) result that the result of color transfer and details are rebuild is merged, ultimately generates the new photo of HDR style.The method of the present invention clear process, structural integrity are realized high-efficient.
Description
Technical field
The present invention relates to a kind of HDR (High Dynamic Range, high dynamic range) photo wind dictionary-based learning
Lattice transfer method, belongs to field of image processing.
Background technique
Stylized technology, also known as feeling of unreality render (i.e. NPR, Non-Photorealistic Rendering), are meters
The research field that calculation machine technology and drawing techniques combine refers to the figure skill for generating using computer and having artistic style
Art.Currently, many high-performance cameras (such as: Canon 5D Mark III, Nikon D800 etc.) different journeys three times can be passed through
The exposure (owing exposure, normal, overexposure) of degree, is automatically synthesized a series of HDR photo.The photo of these HDR styles is aobvious with style
The features such as work, details are abundant, color is full, presents art features abundant, and typical HDR photo style includes: nature, floats
Carving, drawing, oil painting etc..It is different from the HDR effect of camera shooting, people prefer to realize a variety of HDR photos in common photo
Style makes common photo that can also show unique artistic charm, thus many HDR photos generation software (such as: HDR
Making, Dynamic Photo HDRI etc.) gradually come into vogue, wherein most be by manually to color and details into
Row adjustment come what is realized, not only interact it is cumbersome, time-consuming and laborious, but also generate HDR effect stability it is poor, in contrast to actual photographed
HDR effect gap it is larger, can not ideally show the artistry of photo.
Much methods about color of image transfer and image enhancement have been proposed in current researcher both domestic and external:
Reinhard et al. proposes the color branching algorithm based on statistics on the basis of color space l α β, in l α β color
Then the statistical information of each Color Channel of spatial match two images --- mean value and variance changes the color point in source images
Cloth makes it match the distribution of reference picture.
Chang et al. proposes a kind of color transfer method based on sample, according to the color characteristic of sample image to source
The color of image is adjusted, and source images is made to possess color similar with sample image, the calculation while keeping own profile
Method improves the accuracy of color transfer matching process, improves the effect of color transfer.
Xiao et al. proposes the color branching algorithm of gradient holding, the optimization mould kept by Histogram Matching and gradient
Type improves the effect of color transfer.
Luan et al. develops an interactive tool and shifts for local color, maintains global continuity.Wen
Et al. propose the color transfer system based on paintbrush, user can specify regional area and change its Color Style.
Tai et al. proposes the color branching algorithm based on image segmentation, mainly by image segmentation, cut zone match colors
With cut zone combination three parts composition.Wherein image segmentation part estimates pixel in mixed Gaussian using improved EM algorithm
Distribution of color under model.Improved EM obtains more accurate pixel color distribution by the way of iteration update.
The filter that Fattal et al. is kept using boundary does smoothing processing to image and obtains Primary layer and levels of detail, finally
The details enhancing of image is realized by adjusting zoom factor.
Qin et al. proposes that the Retinex structure light image of HSV color space enhances algorithm.It first will be on traditional rgb space
Multi-Scale Retinex Algorithm be transformed into hsv color space；Then enhanced in model by analysis hsv color spatial model
V component, while adjust S component adaptively with the enhancing of V component using related coefficient；Finally HSV model conversion is arrived
Rgb space makes enhanced color of image be maintained.
Xiao et al. proposes that a kind of rapid image based on different color spaces fusion enhances algorithm, which exists first
HSV space carries out enhancing processing to the channel luminance component V, while stretching obtained log-domain reflecting component to certain dynamic
When range, enhancing Dynamic gene is introduced, adjusts the enhancing degree of different brightness values to avoid noise amplification and color distortion phenomenon；
Then, in rgb color space, halation Producing reason is analyzed, and eliminates halation using improved Gaussian filter, and counting
When calculating reflecting component, the fidelity of color of image is adjusted by parameter.Finally, by the processing result in two kinds of different colours spaces into
Row is weighted and averaged enhancing image the most final.
Zhou et al. proposes a kind of Retinex Algorithm of color image enhancement based on improvement Mean Shift filtering, first
Soft image is decomposed by brightness and color two parts using Principal Component Analysis, passes through the existing Mean Shift that takes on a new look
Filtering method realizes the adaptive enhancing of illumination component, and restores to color channel, finally enterprising on global analysis basis
Row image compensation.
To solve the above-mentioned problems, based on above research, we are on the basis of color of image transfer and image enhancement
Propose a kind of HDR photo style transfer method dictionary-based learning.
Summary of the invention
The purpose of the present invention is be to provide a kind of HDR photograph dictionary-based learning aiming at the defects existing in the prior art
Piece style transfer method.The invention is paid close attention in terms of two: first, how to avoid complicated manual interaction；Second, how to give birth to
At the HDR effect of high quality.With reference to the processing technique based on sample, i.e., characteristic information is extracted from sample, source information is carried out certainly
Dynamic adjustment and processing, we generate it is also contemplated that interacting simplified and effect using HDR photo sample, i.e., given one has
The sample photo of HDR style, color required for therefrom extracting and minutia, and these features are automatically transferred to source and are shone
On piece, to generate the new photo with HDR style.
In order to achieve the above objectives, the present invention adopts the following technical scheme:
A kind of HDR photo style transfer method dictionary-based learning, specific steps are as follows:
(1) HDR is transferred on the photo of source by the color transfer method kept by gradient with reference to the color characteristic of photo；
(2) minutia is extracted with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, forms the excessively complete of details
Standby wordbook；
(3) sparse reconstruction is carried out to source photo using the excessively complete wordbook of details, generates and refers to photo eigen one with HDR
The details of cause；
(4) result that the result of color transfer and details are rebuild is merged, ultimately generates the new photo of HDR style.
Color transfer, the ginseng that user is admired are carried out to source HDR photo using the algorithm that gradient is kept in the step (1)
The style for examining photo is transferred on the photo of source, so that source photo is had the Color Style with reference to photo, while keeping the ladder of source photo
Spend feature, the specific steps are as follows:
By solving energy equation relevant to color, the two energy terms of gradient, retain the structural information of source photo, most
The color transfer for obtaining high-fidelity eventually solves following energy equation as a result, the problem of color is shifted is converted into:
Wherein Ec, EdEnergy term relevant to color, gradient, Φ, R are respectively indicated, at the end of I respectively indicates color transfer
Output image, reference picture, source images；H () expression does Histogram Matching to image, and G () indicates to obtain image gradient,
ωc, ωdRespectively indicate the weight of color component, gradient component；
1) color mapping
In (1) formula, the energy term E of color mapping is definedc, it is solved by following formula (2):
Wherein H () indicates that the Color histogram distribution of image, k indicate histogram serial number, and n indicates section sum；
Color mapping is as a result, the Color histogram distribution of output image must be with the Color histogram distribution phase of reference picture in order to obtain
Matching；In order to solve conveniently, the histogram of source images is matched with the histogram of reference picture, obtains a middle graph
Picture, the accurate distribution of color for maintaining reference picture of the image；Simultaneously in order to avoid the influence to structure, to intermediate image
It is decomposed, obtains levels of detail and Hue layer F, do color treatments in Hue layer, therefore formula (2) redefines are as follows:
Wherein Φ indicates that the image of output, p indicate the pixel of image, obtain the knot of color mapping by solution formula (3)
Fruit；
2) gradient is kept
Due to having separated levels of detail when color mapping, obtained result loses structural fidelity, according to solution formula
(1) the energy term E stated indThe Gradient Features of source images are able to maintain, are indicated with following formula (4):
WhereinIndicate gradient operator, p indicates that the pixel on image, m indicate that total pixel, λ indicate to adjust image
The parameter of contrast；As λ > 1, increase contrast；As λ < 1, reduce contrast；As λ=1, keep contrast constant；
Optimization can effectively keep the prototype structure feature of source images in gradient field, in order to keep the ladder of source images
Then according to the demand of user degree determines the parameter of contrast first with the gradient that Sobel difference operator calculates source images
λ, obtain source images gradient holding as a result, finding out EcAnd EdLater, it substitutes into formula (1) and obtains color transfer result.
The step (2) extracts minutia with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, obtains thin
The excessively complete wordbook D of section, specific steps are as follows:
It selects a HDR embossment or painting style photo as reference picture, in order to improve the quality of reconstruction image, needs
Reference picture is pre-processed, first extract its minutia with weighted least-squares method WLS filter, then take at random
The fritter of 100000 5x5, as training sample set Y；
By the training sample set Y of extraction, learn an excessively complete dictionary with HDR embossment or painting style feature out
D；Using the training of K_SVD algorithm, to the training process of HDR photo style, indicated using following formula:
Wherein D={ d1,d2…dk}∈Rr×kIt is trained that (k > r), which is by excessively complete redundant dictionary D, α that study obtains,
Sparse coefficient vector of the sample set Y on dictionary D, δ indicate αiThe upper limit of middle nonzero element number,Indicate training
The total reconfiguration error of sample set Y；
Using Y as the input of K_SVD algorithm, according to the two of dictionary learning stages, in the sparse coding stage, by initial
Change dictionary D, then normalizes dictionary D, obtain rarefaction representation coefficient square of the Y on dictionary D using orthogonal matching pursuit OMP algorithm
Battle array α；In the dictionary updating stage, according to the sparse matrix factor alpha found out, dictionary atom is updated by column；Meet k iteration or
When meeting the condition of convergence, the excessively complete dictionary D of final optimization pass is obtained；During updating dictionary D, it is assumed that dkExpression will be more
The kth column of new dictionary, αkIt indicates the corresponding sparse coefficient of the column, formula (5) is rewritten as follows:
Wherein D α is broken down into the sum for the matrix that k order is 1, it is assumed that wherein k-1 are fixed, and remaining 1 column are just
It is the kth to be updated column；(6) the matrix E in formulakIt indicates to remove removal atom dkError of the remaining ingredient in sample set Y afterwards；
E is updated using the method for singular value decomposition SVDkWithIt is found by this method and EkDifference is minimum and order is 1
Matrix, the global reconstruction error of training sample can be effectively reduced；But due to being obtained by singular value decompositionIt is not
Sparse, i.e.,Before updateThe position of nonzero element and in different size, the dictionary caused is not compact；Therefore,
Using only retainingIn nonzero term, recycle SVD method update αkAnd dk, until completing dictionary updating by column, by this
The dictionary D that method obtains can indicate the characteristic of sample well.
The step (3) uses the thought of sparse expression, is carried out using the dictionary that step (2) middle school's acquistion is arrived to source photo
Sparse reconstruction generates and refers to the consistent details of photo eigen, the specific steps are as follows:
The FSS algorithm that Lee is proposed is used for reference in rarefaction representation part, and algorithm flow is as follows:
(1) it inputs: the source photo X of common style, with reference to the training dictionary D of photo；
(2) minutia is gone out to source photo X benefit WLS filter equalizer, and carries out optimization processing, obtain image X1；
(3) bad process is followed: from image X1The image block x that size is 5 × 5 is successively taken out in the upper left corner, extracts to each fritter
Then each fritter is become 25 × 1 column signal by minutiaFind out rarefaction representation of each column signal at dictionary D
Factor alpha, specifically:
The average value m of step 1. calculating image block x；
D known to step 2. andSparse coefficient α is solved by following expression formula,
Wherein, λ1It indicates regularization parameter, is used to Equilibrium fitting error term and sparse constraint, λ1Value range is in 0 and 1
Between；
The relief style image block that step 3. is estimated is D α, and embossment/painting style image block of output is x*=m+D α；
(4) export: the fritter of sparse reconstruction restored by original position, obtain with details embossment abundant or
Person's painting style image X*。
The result that the result and details that the step (4) shifts color are rebuild merges, and ultimately generates the new photograph of HDR style
Piece, specific steps are as follows:
Since HDR photo has the feature of two aspects of color and details, pass through the color and details using reference picture
The tutorial message of aspect has respectively obtained the result of source photo color transfer and the result of the sparse reconstruction of source photo；Finally, needing
The two results are merged, source photo is made all to have the characteristics that HDR style on tone and details；Merging process reality etc.
It imitates in solving the energy equation comprising data item and gradient terms, wherein data item is color transfer as a result, and gradient terms
It is that details is rebuild as a result, being indicated with following formula:
Wherein Φ*, Φ, X*Respectively indicate the HDR style photo ultimately generated, source photo color transfer is as a result, source photo
Sparse reconstructed results；P indicates the pixel on photo, λ*Indicate that gradient constraint coefficient, ▽ indicate to obtain the gradient of photo, pass through
Optimization energy equation above merged as a result, obtaining new HDR style photo.
Compared with the prior art the present invention has a characteristic that
(1) realization algorithm flow provided by the invention is clear, structural integrity, realizes high-efficient.
(2) the minutia dictionary with reference to photo is obtained by the method training of dictionary learning.
(3) source images are rebuild using the method for sparse expression, makes source images that there is the minutia of reference picture.
Detailed description of the invention
Fig. 1 is HDR photo style transfer method program chart dictionary-based learning.
Fig. 2 is that HDR relief style transfer result (a) indicates that source photo (b) indicates to indicate color transfer with reference to photo (c)
As a result (d) indicates that the result (e) of sparse reconstruction indicates final output result.
Fig. 3 is that HDR painting style transfer result (a) indicates that source photo (b) indicates to indicate color transfer with reference to photo (c)
As a result (d) indicates that the result (e) of sparse reconstruction indicates final output result.
Specific embodiment
Preferred embodiments of the invention are described with reference to the drawings as follows:
Referring to Fig. 1, a kind of HDR photo style transfer method dictionary-based learning, specific steps are as follows:
(1) HDR is transferred on the photo of source by the color transfer method kept by gradient with reference to the color characteristic of photo；
Specific step is as follows:
By solving energy equation relevant to color, the two energy terms of gradient, retain the structural information of source photo, most
The color transfer for obtaining high-fidelity eventually solves following energy equation as a result, the problem of color is shifted is converted into:
Wherein Ec, EdEnergy term relevant to color, gradient, Φ, R are respectively indicated, at the end of I respectively indicates color transfer
Output image, reference picture, source images；H () expression does Histogram Matching to image, and G () indicates to obtain image gradient,
ωc, ωdRespectively indicate the weight of color component, gradient component；
1) color mapping
In (1) formula, the energy term E of color mapping is definedc, it is solved by following formula (2):
Wherein H () indicates that the Color histogram distribution of image, k indicate histogram serial number, and n indicates section sum；
Color mapping is as a result, the Color histogram distribution of output image must be with the Color histogram distribution phase of reference picture in order to obtain
Matching；In order to solve conveniently, the histogram of source images is matched with the histogram of reference picture, obtains a middle graph
Picture, the accurate distribution of color for maintaining reference picture of the image；Simultaneously in order to avoid the influence to structure, to intermediate image
It is decomposed, obtains levels of detail and Hue layer F, color treatments are carried out to Hue layer, therefore formula (2) redefines are as follows:
Wherein Φ indicates that the image of output, p indicate the pixel of image, obtain the knot of color mapping by solution formula (3)
Fruit；
2) gradient is kept
Due to having separated levels of detail when color mapping, obtained result loses structural fidelity, according to solution formula
(1) the energy term E stated indThe Gradient Features of source images are able to maintain, are indicated with following formula (4):
WhereinIndicate gradient operator, p indicates that the pixel on image, m indicate that the total pixel of image, λ indicate to adjust
The parameter of picture contrast；As λ > 1, increase contrast；As λ < 1, reduce contrast；As λ=1, contrast is kept not
Become；
Optimization can effectively keep the prototype structure feature of source images in gradient field, in order to keep the ladder of source images
Then according to the demand of user degree determines the parameter of contrast first with the gradient that Sobel difference operator calculates source images
λ, obtain source images gradient holding as a result, finding out EcAnd EdLater, it substitutes into formula (1) and obtains color transfer result.
(2) minutia is extracted with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, obtains the excessively complete of details
Standby wordbook D, specific steps are as follows:
It selects a HDR embossment or painting style photo as reference picture, in order to improve the quality of reconstruction image, needs
Reference picture is pre-processed, first extract its minutia with weighted least-squares method WLS filter, then take at random
The fritter of 100000 5x5, as training sample set Y；
By the training sample set Y of extraction, learn an excessively complete dictionary with HDR embossment or painting style feature out
D；Using the training of K_SVD algorithm, to the training process of HDR photo style, indicated using following formula:
Wherein D={ d1,d2…dk}∈Rr×kIt is trained that (k > r), which is by excessively complete redundant dictionary D, α that study obtains,
Sparse coefficient vector of the sample set Y on dictionary D, δ indicate αiThe upper limit of middle nonzero element number,Indicate training
The total reconfiguration error of sample set Y；
Using Y as the input of K_SVD algorithm, according to the two of dictionary learning stages, in the sparse coding stage, by initial
Change dictionary D, then normalizes dictionary D, obtain rarefaction representation coefficient square of the Y on dictionary D using orthogonal matching pursuit OMP algorithm
Battle array α；In the dictionary updating stage, according to the sparse matrix factor alpha found out, dictionary atom is updated by column；Meet k iteration or
When meeting the condition of convergence, the excessively complete dictionary D of final optimization pass is obtained；During updating dictionary D, it is assumed that dkExpression will be more
The kth column of new dictionary, αkIt indicates the corresponding sparse coefficient of the column, formula (5) is rewritten as follows:
Wherein D α is broken down into the sum for the matrix that k order is 1, it is assumed that wherein k-1 are fixed, and remaining 1 column are just
It is the kth to be updated column；(6) the matrix E in formulakIt indicates to remove removal atom dkError of the remaining ingredient in sample set Y afterwards；
E is updated using the method for singular value decomposition SVDkWithIt is found by this method and EkDifference is minimum and order is 1
Matrix, the global reconstruction error of training sample can be effectively reduced；But due to being obtained by singular value decompositionIt is not
Sparse, i.e.,Before updateThe position of nonzero element and in different size, the dictionary caused is not compact；Therefore,
Using only retainingIn nonzero term, recycle SVD method update αkAnd dk, until completing dictionary updating by column, by this
The dictionary D that method obtains can indicate the characteristic of sample well.
(3) sparse reconstruction is carried out to source photo using the excessively complete wordbook of details, generates and refers to photo eigen one with HDR
The details of cause；Specific step is as follows:
The FSS algorithm that Lee is proposed is used for reference in rarefaction representation part, and algorithm flow is as follows:
(1) it inputs: the source photo X of common style, with reference to the training dictionary D of photo；
(2) minutia is gone out to source photo X benefit WLS filter equalizer, and carries out optimization processing, obtain image X1；
(3) bad process is followed: from image X1The image block x that size is 5 × 5 is successively taken out in the upper left corner, extracts to each fritter
Then each fritter is become 25 × 1 column signal by minutiaFind out rarefaction representation system of each column signal at dictionary D
Number α, specifically:
The average value m of step 1. calculating image block x；
D known to step 2. andSparse coefficient α is solved by following expression formula,
Wherein, λ1It indicates regularization parameter, is used to Equilibrium fitting error term and sparse constraint, λ1Value range is in 0 and 1
Between；
The relief style image block that step 3. is estimated is D α, and embossment/painting style image block of output is x*=m+D α；
(4) export: the fritter of sparse reconstruction restored by original position, obtain with details embossment abundant or
Person's painting style image X*。
(4) result that the result of color transfer and details are rebuild is merged, ultimately generates the new photo of HDR style, specifically
Step are as follows:
Since HDR photo has the feature of two aspects of color and details, pass through the color and details using reference picture
The tutorial message of aspect has respectively obtained the result of source photo color transfer and the result of the sparse reconstruction of source photo；Finally, needing
The two results are merged, source photo is made all to have the characteristics that HDR style on tone and details；Merging process reality etc.
It imitates in solving the energy equation comprising data item and gradient terms, wherein data item is color transfer as a result, and gradient terms
It is that details is rebuild as a result, being indicated with following formula:
Wherein Φ*, Φ, X*Respectively indicate the HDR style photo ultimately generated, source photo color transfer is as a result, source photo
Sparse reconstructed results；P indicates the pixel on photo, λ*Indicate gradient constraint coefficient,The gradient for indicating acquisition photo, passes through
Optimization energy equation above merged as a result, obtaining new HDR style photo.
This method is adapted to all HDR style transfers, and user need to only pass through upper according to selected reference photo type
It states step to be handled, so that it may which source photo is become new HDR style photo.If the HDR photo style of reference is embossment
Style, then the experimental result of corresponding HDR relief style transfer is as shown in Figure 2.If the HDR photo style of reference is oil painting wind
Lattice, then the experimental result of corresponding HDR painting style transfer is as shown in Figure 3.
Claims (1)
1. a kind of HDR photo style transfer method dictionary-based learning, which is characterized in that specific steps are as follows:
(1) HDR is transferred on the photo of source by the color transfer method kept by gradient with reference to the color characteristic of photo；
(2) minutia is extracted with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, forms the excessively complete word of details
Allusion quotation collection；
(3) sparse reconstruction is carried out to source photo using the excessively complete wordbook of details, generated consistent with reference to photo eigen with HDR
Details；
(4) result that the result of color transfer and details are rebuild is merged, ultimately generates the new photo of HDR style；
Color transfer is carried out to source HDR photo using the algorithm that gradient is kept in the step (1), the reference that user is admired is shone
The style of piece is transferred on the photo of source, and source photo is made to have the Color Style with reference to photo, while keeping the gradient of source photo special
Sign, the specific steps are as follows:
By solving energy equation relevant to color, the two energy terms of gradient, the structural information of reservation source photo is final to obtain
Color transfer to high-fidelity solves following energy equation as a result, the problem of color is shifted is converted into:
Wherein Ec, EdEnergy term relevant to color, gradient, Φ, R are respectively indicated, I respectively indicates defeated at the end of color shifts
Image out, reference picture, source images；H () indicates that the Color histogram distribution of image, G () indicate to obtain image gradient,
ωc, ωdRespectively indicate the weight of color component, gradient component；
1) color mapping
In (1) formula, the energy term E of color mapping is definedc, it is solved by following formula (2):
Wherein H () indicates that the Color histogram distribution of image, k indicate histogram serial number, and n indicates section sum；In order to
Color mapping is obtained as a result, the Color histogram distribution of output image must be with the Color histogram distribution phase of reference picture
Match；In order to solve conveniently, the histogram of source images is matched with the histogram of reference picture, obtains an intermediate image,
The accurate distribution of color for maintaining reference picture of the image；Simultaneously in order to avoid the influence to structure, to intermediate image into
Row decomposes, and obtains levels of detail and Hue layer F, does color treatments in Hue layer, therefore formula (2) redefines are as follows:
Wherein Φ indicates that the image of output, p indicate the pixel of image, obtain the result of color mapping by solution formula (3)；
2) gradient is kept
Due to having separated levels of detail when color mapping, obtained result loses structural fidelity, according in solution formula (1)
The energy term E of statementdThe Gradient Features of source images are able to maintain, are indicated with following formula (4):
Wherein ▽ indicates gradient operator, and p indicates that the pixel on image, m indicate that total pixel, λ indicate to adjust image comparison
The parameter of degree；As λ > 1, increase contrast；As λ < 1, reduce contrast；As λ=1, keep contrast constant；
Optimization can effectively keep the prototype structure feature of source images in gradient field, first in order to keep the gradient of source images
It then according to the demand of user determines the parameter lambda of contrast first with the gradient that Sobel difference operator calculates source images, obtains
To the holding of source images gradient as a result, finding out EcAnd EdLater, it substitutes into formula (1) and obtains color transfer result；
The step (2) extracts minutia with reference to photo to HDR, carries out dictionary training using K_SVD algorithm, obtains details
Cross complete wordbook D, specific steps are as follows:
It selects a HDR embossment or painting style photo as reference picture, in order to improve the quality of reconstruction image, needs pair
Reference picture pre-processes, and first extracts its minutia with weighted least-squares method WLS filter, then takes 100000 at random
The fritter of 5x5, as training sample set Y；
By the training sample set Y of extraction, learn an excessively complete dictionary D with HDR embossment or painting style feature out；Benefit
With the training of K_SVD algorithm, to the training process of HDR photo style, indicated using following formula:
Wherein D={ d1, d2…dk}∈Rr×k, it is training sample set that k > r, which is by excessively complete redundant dictionary D, α that study obtains,
Sparse coefficient vector of the Y on dictionary D, δ indicate αiThe upper limit of middle nonzero element number,Indicate training sample set Y
Total reconfiguration error；
Using Y as the input of K_SVD algorithm, according to the two of dictionary learning stages, in the sparse coding stage, by initializing word
Then allusion quotation D normalizes dictionary D, obtains rarefaction representation coefficient matrix α of the Y on dictionary D using orthogonal matching pursuit OMP algorithm；
In the dictionary updating stage, according to the sparse matrix factor alpha found out, dictionary atom is updated by column；Meeting k iteration or satisfaction
When the condition of convergence, the excessively complete dictionary D of final optimization pass is obtained；During updating dictionary D, it is assumed that dkExpression will update word
The kth of allusion quotation arranges, αkIt indicates the corresponding sparse coefficient of the column, formula (5) is rewritten as follows:
Wherein D α is broken down into the sum for the matrix that k order is 1, it is assumed that wherein k-1 are fixed, and remaining 1 column are sought to
The kth of update arranges；(6) the matrix E in formulakIt indicates to remove removal atom dkError of the remaining ingredient in sample set Y afterwards；
E is updated using the method for singular value decomposition SVDkWithIt is found by this method and EkThe square that difference minimum and order are 1
Battle array, can effectively reduce the global reconstruction error of training sample；But due to being obtained by singular value decompositionIt is not sparse
, i.e.,Before updateThe position of nonzero element and in different size, the dictionary caused is not compact；Therefore, it uses
Only retainIn nonzero term, recycle SVD method update αkAnd dk, until completing dictionary updating by column, by this method
Obtained dictionary D can indicate the characteristic of sample well；
The step (3) uses the thought of sparse expression, and the dictionary arrived using step (2) middle school's acquistion carries out source photo sparse
It rebuilds, generate and refers to the consistent details of photo eigen, the specific steps are as follows:
A. it inputs: the source photo X of common style, with reference to the training dictionary D of photo；
B. minutia is gone out using WLS filter equalizer to source photo X, and carries out optimization processing, obtain image X1；
C. bad process is followed: from image X1The image block x that size is 5 × 5 is successively taken out in the upper left corner, and it is special to extract details to each fritter
Each fritter, is then become 25 × 1 column signal by signRarefaction representation coefficient α of each column signal at dictionary D is found out,
Specifically:
The average value m of step 1. calculating image block x；
D known to step 2. andSparse coefficient α is solved by following expression formula,
Wherein, λ1It indicates regularization parameter, is used to Equilibrium fitting error term and sparse constraint, λ1Value range is between 0 and 1；
The relief style image block that step 3. is estimated is D α, and embossment/painting style image block of output is x*=m+D α；
D. it exports: the fritter of sparse reconstruction being restored by original position, is obtained with details embossment abundant or oil
Painting style table images X*；
The result that the result and details that the step (4) shifts color are rebuild merges, and ultimately generates the new photo of HDR style,
Specific steps are as follows:
Since HDR photo has the feature of two aspects of color and details, in terms of the color and details using reference picture
Tutorial message, respectively obtained source photo color transfer result and the sparse reconstruction of source photo result；Finally, need by
The two results merge, and source photo is made all to have the characteristics that HDR style on tone and details；The merging process is practical to be equivalent to
The energy equation comprising data item and gradient terms is solved, wherein data item is color transfer as a result, and gradient terms are thin
Section rebuild as a result, being indicated with following formula:
Wherein Φ*, Φ, X*Respectively indicate the HDR style photo ultimately generated, the transfer of source photo color as a result, source photo it is sparse
Reconstructed results；P indicates the pixel on photo, λ*Indicate gradient constraint coefficient,The gradient for indicating acquisition photo, passes through optimization
It is that energy equation above is merged as a result, obtaining new HDR style photo.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058620.6A CN106780367B (en) | 2016-11-28 | 2016-11-28 | HDR photo style transfer method dictionary-based learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611058620.6A CN106780367B (en) | 2016-11-28 | 2016-11-28 | HDR photo style transfer method dictionary-based learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106780367A CN106780367A (en) | 2017-05-31 |
| CN106780367B true CN106780367B (en) | 2019-11-15 |
Family
ID=58912507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611058620.6A Active CN106780367B (en) | 2016-11-28 | 2016-11-28 | HDR photo style transfer method dictionary-based learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106780367B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107705242B (en) * | 2017-07-20 | 2021-12-17 | 广东工业大学 | Image stylized migration method combining deep learning and depth perception |
| WO2019023968A1 (en) * | 2017-08-02 | 2019-02-07 | 深圳传音通讯有限公司 | Image color adjustment system and color adjustment method for smart terminal |
| CN107730479B (en) * | 2017-08-30 | 2021-04-20 | 中山大学 | High dynamic range image artifact-removing fusion method based on compressed sensing |
| CN107895191B (en) | 2017-10-30 | 2022-02-22 | 上海寒武纪信息科技有限公司 | Information processing method and related product |
| CN109064434B (en) * | 2018-06-28 | 2021-01-08 | 广州视源电子科技股份有限公司 | Image enhancement method and device, storage medium and computer equipment |
| CN109308679B (en) * | 2018-08-13 | 2022-08-30 | 深圳市商汤科技有限公司 | Image style conversion method and device, equipment and storage medium |
| US20210133854A1 (en) | 2018-09-13 | 2021-05-06 | Shanghai Cambricon Information Technology Co., Ltd. | Information processing method and terminal device |
| EP3869466A1 (en) * | 2020-02-24 | 2021-08-25 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Method and device for picture generation, electronic device, and storage medium |
| CN112560960A (en) * | 2020-12-16 | 2021-03-26 | 北京影谱科技股份有限公司 | Hyperspectral image classification method and device and computing equipment |
| CN113269223B (en) * | 2021-03-16 | 2022-04-22 | 重庆市地理信息和遥感应用中心 | City style classification method based on spatial culture modular factorial analysis |
| CN114359083B (en) * | 2021-12-24 | 2022-11-29 | 北京航空航天大学 | High-dynamic thermal infrared image self-adaptive preprocessing method for interference environment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093445A (en) * | 2013-01-17 | 2013-05-08 | 西安电子科技大学 | Unified feature space image super-resolution reconstruction method based on joint sparse constraint |
| CN105989584A (en) * | 2015-01-29 | 2016-10-05 | 北京大学 | Image stylized reconstruction method and device |
| CN106023098A (en) * | 2016-05-12 | 2016-10-12 | 西安电子科技大学 | Image repairing method based on tensor structure multi-dictionary learning and sparse coding |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120133664A1 (en) * | 2010-11-29 | 2012-05-31 | Lotus Hill Institute For Computer Vision And Information Science | System and method for painterly rendering based on image parsing |
-
2016
- 2016-11-28 CN CN201611058620.6A patent/CN106780367B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093445A (en) * | 2013-01-17 | 2013-05-08 | 西安电子科技大学 | Unified feature space image super-resolution reconstruction method based on joint sparse constraint |
| CN105989584A (en) * | 2015-01-29 | 2016-10-05 | 北京大学 | Image stylized reconstruction method and device |
| CN106023098A (en) * | 2016-05-12 | 2016-10-12 | 西安电子科技大学 | Image repairing method based on tensor structure multi-dictionary learning and sparse coding |
Non-Patent Citations (4)
| Title |
|---|
| "基于区域自适应的图像间颜色迁移算法与性能评价";马平平;《中国优秀硕士学位论文全文数据库信息科技辑》;20140715;第2014年卷(第7期);I138-765 * |
| "基于实例的颜色处理新技术研究";肖学中;《中国博士学位论文全文数据库信息科技辑》;20100715;第2010年卷(第7期);I138-34 * |
| "基于配准融合及色彩迁移的新图像生成";王丽丹;《中国优秀硕士学位论文全文数据库信息科技辑》;20070215;第2007年卷(第2期);I138-754 * |
| "异类图像多级混合融合技术研究";陈磊;《中国优秀硕士学位论文全文数据库信息科技辑》;20140815;第2014年卷(第8期);I138-1279 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106780367A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780367B (en) | HDR photo style transfer method dictionary-based learning | |
| Hao et al. | Low-light image enhancement with semi-decoupled decomposition | |
| Ying et al. | A bio-inspired multi-exposure fusion framework for low-light image enhancement | |
| CN110781895B (en) | Image semantic segmentation method based on convolutional neural network | |
| Huang et al. | Deep learning for image colorization: Current and future prospects | |
| CN109064423B (en) | Intelligent image repairing method for generating antagonistic loss based on asymmetric circulation | |
| WO2021022929A1 (en) | Single-frame image super-resolution reconstruction method | |
| CN110443763B (en) | Convolutional neural network-based image shadow removing method | |
| CN107798661B (en) | Self-adaptive image enhancement method | |
| CN111626918B (en) | Method and system for carrying out style change on digital image based on semantic segmentation network technology | |
| CN110675462A (en) | Gray level image colorizing method based on convolutional neural network | |
| CN114723643B (en) | Low-light image enhancement method based on reinforcement learning and aesthetic evaluation | |
| CN113222875B (en) | Image harmonious synthesis method based on color constancy | |
| CN109829925A (en) | A kind of method and model training method for extracting clean prospect in scratching figure task | |
| CN108537720A (en) | A kind of image processing method and device | |
| CN113658057A (en) | Swin transform low-light-level image enhancement method | |
| CN110322530A (en) | It is a kind of based on depth residual error network can interaction figure picture coloring | |
| CN110276753A (en) | Objective self-adapting hidden method based on the mapping of feature space statistical information | |
| CN111127377B (en) | Weak light enhancement method based on multi-image fusion Retinex | |
| Li et al. | Flexible piecewise curves estimation for photo enhancement | |
| Lei et al. | Low-light image enhancement using the cell vibration model | |
| CN113393546B (en) | Fashion clothing image generation method based on clothing type and texture pattern control | |
| Li et al. | A large-scale film style dataset for learning multi-frequency driven film enhancement | |
| CN113052783A (en) | Face image fusion method based on face key points | |
| Liu et al. | Halder: Hierarchical attention-guided learning with detail-refinement for multi-exposure image fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |