CN107133953B - Intrinsic image decomposition method based on partial differential equation learning - Google Patents

Intrinsic image decomposition method based on partial differential equation learning Download PDF

Info

Publication number
CN107133953B
CN107133953B CN201710138216.8A CN201710138216A CN107133953B CN 107133953 B CN107133953 B CN 107133953B CN 201710138216 A CN201710138216 A CN 201710138216A CN 107133953 B CN107133953 B CN 107133953B
Authority
CN
China
Prior art keywords
equation
function
adjoint
image
partial differential
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710138216.8A
Other languages
Chinese (zh)
Other versions
CN107133953A (en
Inventor
赖剑煌
赵兰杰
谢晓华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
SYSU CMU Shunde International Joint Research Institute
Original Assignee
Sun Yat Sen University
SYSU CMU Shunde International Joint Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University, SYSU CMU Shunde International Joint Research Institute filed Critical Sun Yat Sen University
Priority to CN201710138216.8A priority Critical patent/CN107133953B/en
Publication of CN107133953A publication Critical patent/CN107133953A/en
Application granted granted Critical
Publication of CN107133953B publication Critical patent/CN107133953B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an intrinsic image decomposition method based on partial differential equation learning, which does not depend on determined prior constraint on the estimation of intrinsic components of processed images and adopts a data-driven mode to construct a partial differential equation; determining a search direction by using a conjugate gradient method, wherein the method combines the conjugation with the steepest descent method relative to the steepest descent method and the Newton method; constructing a group of conjugate directions by utilizing the gradient of a known point, searching along the group of directions, and solving a minimum point of an objective function to determine the optimal searching direction; the method can effectively realize the intrinsic decomposition of the images under different illumination conditions to obtain the reflection components and the shadow components of the images.

Description

Intrinsic image decomposition method based on partial differential equation learning
Technical Field
The invention relates to the field of digital image processing, in particular to an intrinsic image decomposition method based on partial differential equation learning.
Background
The image of an object in the real world presented in the human eye depends on the intrinsic properties of the scene, such as the illumination of the scene, the shape of the object surface, the material of the object surface, etc. Intrinsic image decomposition is a fundamental problem in computer vision, given an input image, the corresponding reflection and shadow components need to be decomposed. With the development of digital image processing technology, images are decomposed to obtain intrinsic components, which play an increasingly important role in the fields of computer vision and image processing. It is difficult to obtain the reflection component and the shadow component of an image with different illumination conditions in the same scene by intrinsic decomposition.
At present, in the intrinsic image decomposition task, the traditional method generally designs a corresponding constraint optimization equation according to various prior knowledge, and solves and obtains a reflection component and a shadow component by optimizing an objective equation. Common prior assumptions include images with different illumination in the same scene, smooth surface of input image, balanced color of image surface, natural illumination of image, etc. In addition, some current research algorithms rely on additional information from multiple input images, user input, or depth cues. The method mainly aims at input images with different illumination conditions in the same scene so as to recover the reflection component and the shadow component of the input images. The above methods have some disadvantages, and the existing methods are generally based on prior assumptions and generally have no universality.
Disclosure of Invention
The invention provides an intrinsic image decomposition method based on partial differential equation learning, which can realize intrinsic decomposition of images under different illumination conditions to obtain reflection components and shadow components of the images.
In order to achieve the technical effects, the technical scheme of the invention is as follows:
an intrinsic image decomposition method based on partial differential equation learning comprises the following steps:
s1: inputting a training data pair;
s2: initializing the control function, since the objective function is non-convex, the convergence direction of the minimization process depends on the initialized local minimum, and when the initialization process does not converge, turning to step S3, executing a loop; otherwise go to step S8;
s3: solving an optimal control equation with PDE constraints;
s4: solving an adjoint equation when the adjoint function takes a specific value;
s5: the derivative of the translational rotational invariants at j 0, 1.
Figure BDA0001242032760000021
Wherein J is the translation and rotation invariant, J is the number of the translation and rotation invariant, aj，bjIs a control function, λjAnd ujIs a positive weighting parameter that is,
Figure BDA0001242032760000026
and phimIs the adjoint function, u is the output image, Ω is the rectangular area occupied by the input image, fΩIs an initial function of Ω, M1, 2.., M is the logarithm of the input data samples, inv (u, v) represents the inversion of the matrix (u, v), v is an indicator function, which is introduced with the purpose of collecting large scale information in the image in order to guide the evolution of u.
S6: determining a search direction using a conjugate gradient method;
s7: performing golden section search along the search direction, continuously updating the system function, and performing the next cycle until j is 16 for training;
s8: terminating the loop and outputting a system function;
s9: preparing application data, wherein the data picture is characterized by black background and single and prominent target object;
s10: and performing eigen decomposition application on the given data by using the obtained system function to obtain a reflection component and a shadow component of the image.
Further, in the step S2, the control function a is controlled by solving the equation (1)j(t), t ═ 0, Δ t, ·,1- Δ t, initialized, at which time b is fixedj(t),j＝0,1,···,16，
F·a(t)＝d(1)
Figure BDA0001242032760000022
Figure BDA0001242032760000023
Wherein, aj(t) and bj(t) is a control function, a (t) { a ═ bj(t) } and b (t) ═ bj(t) is a set of functions defined on Q, respectively for controlling the evolution of u and v, fuAnd fvAre the initial functions of u and v, respectively.
Further, the step S3 calculating the addendum derivative of the jth translational and rotational invariant by introducing the adjoint equation of the formula (2), thereby obtaining a local optimum
Figure BDA0001242032760000024
And
Figure BDA0001242032760000025
calculated by a gradient-based algorithm, for umAnd vmWhen M is 1,2, …, M, formula (2) is solved:
Figure BDA0001242032760000031
wherein T is the time when the PDE system completes the visual information processing and outputs the result, Q is Ω x (0, T), and Γ is
Figure BDA0001242032760000032
Further, the adjoint equation at the time of solving the adjoint function specific value in the step S4 is:
for the
Figure BDA0001242032760000039
And phimM1, 2, …, M, the adjoint equation is solved, where
Figure BDA00012420327600000310
And phimThe adjoint equation of (c) is as shown in equation (3):
Figure BDA0001242032760000033
Figure BDA0001242032760000034
Figure BDA0001242032760000035
Figure BDA0001242032760000036
Figure BDA0001242032760000037
Figure BDA0001242032760000038
Figure BDA0001242032760000041
Figure BDA0001242032760000042
wherein, OmIs the desired output image, p, q belong to the index set of local variations of { (0,0), (0,1), (0,2), (1,1), (2,0) }.
Further, the derivative of the translational rotational invariants at j 0, 1.., 16 is calculated using equation (4); calculating the derivative of the translational rotational invariant of the control function when j is 0,1
Figure BDA0001242032760000043
And
Figure BDA0001242032760000044
with the help of the adjoint equation for aj(t) and bj(t), the derivative of J in each iteration is as follows:
Figure BDA0001242032760000045
wherein the adjoint function
Figure BDA0001242032760000046
And phimIs the answer to equation (3).
Further, in step S7, performing a golden section search along the search direction, and continuously updating the system function to perform the next cycle;
performing a golden section search along a search direction, updating the control function aj(t) and bj(t), j-0, 1.., 16, and continue with the next cycle until j-16 is trained.
Further, in step S8, the loop is terminated, and the system function is output
Figure BDA0001242032760000047
And
Figure BDA0001242032760000048
compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the method does not depend on the determined prior constraint on the estimation of the intrinsic components of the processed image, and adopts a data-driven mode to construct a partial differential equation; determining a search direction by using a conjugate gradient method, wherein the method combines the conjugation with the steepest descent method relative to the steepest descent method and the Newton method; constructing a group of conjugate directions by utilizing the gradient of a known point, searching along the group of directions, and solving a minimum point of an objective function to determine the optimal searching direction; the method can effectively realize the intrinsic decomposition of the images under different illumination conditions to obtain the reflection components and the shadow components of the images.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 1, an eigen image decomposition method based on partial differential equation learning includes the following steps:
1) and a training stage:
step 1: inputting a training data pair;
the input training data pairs are Images in an MIT-Intrasic Images library, the training input data comprise 20 types of pictures, each type comprises 10 pictures with the same scene and different illumination, the number of the pictures is 220, and the training output data are shadow components collected corresponding to the input pictures.
Step 2: initializing a control function, and since the target function is non-convex, the convergence direction of the minimization process depends on the initialized local minimum, and when the initialization process is not converged, turning to step 3, executing a loop; otherwise, turning to step 8;
control function a is controlled by solving equation (1)j(t), t ═ 0, Δ t, ·,1- Δ t, initialized, at which time b is fixedj(t),j＝0,1,···,16，
F·a(t)＝d (1)
Figure BDA0001242032760000051
Figure BDA0001242032760000052
Where u is the output image and v is an indicator function, the purpose of introducing the indicator function is to search for large scale information in the image in order to correctly guide the evolution of u. Since the objective function is non-convex, the convergence of the minimization process is oriented to the initialized local minimum, when the control function does not converge, the loop is executed, step 3, otherwise, step 8.
And step 3: solving an optimal control equation with PDE constraints;
by introducing the adjoint equation of the formula (2), the addition derivative of the jth translational and rotational invariant is calculated, so that the local optimum value
Figure BDA0001242032760000053
And
Figure BDA0001242032760000054
can be calculated by a gradient-based algorithm, for umAnd vmWhen M is 1, 2., M, formula (2) is solved:
Figure BDA0001242032760000061
where Ω is a rectangular area occupied by the input image, T is a time taken for the PDE system to complete the visual information processing and output the result, and is an initial function of u and v, respectively. For computational problems and mathematical deductions involved, the present invention will fill its surroundings with zero values of a few pixels width. Since the time unit can be changed, T is 1, and fixed as a set of functions defined on Q, are used to control the evolution of u and v, respectively.
And 4, step 4: solving an adjoint equation when the adjoint function takes a specific value;
for the
Figure BDA0001242032760000062
And phimM1, 2, M, an adjoint equation is solved, wherein
Figure BDA0001242032760000063
And phimThe adjoint equation of (c) is as shown in equation (3):
Figure BDA0001242032760000064
Figure BDA0001242032760000065
Figure BDA0001242032760000066
Figure BDA0001242032760000067
Figure BDA0001242032760000068
Figure BDA0001242032760000071
Figure BDA0001242032760000072
Figure BDA0001242032760000073
and 5: calculating the derivative of the translational rotation invariant at j 0,1, 16 using equation (4);
calculating the derivative of the translational rotational invariants of the control function at j 0,1, 16 by equation (4)
Figure BDA0001242032760000074
And
Figure BDA0001242032760000075
with the help of the adjoint equation for aj(t) and bj(t), the derivative of J in each iteration is as follows:
Figure BDA0001242032760000076
wherein λ isjAnd ujIs a positive weighting parameter, a adjoint function
Figure BDA0001242032760000077
And phimIs the answer to equation (3).
Step 6: determining a search direction using a conjugate gradient method;
the search direction is determined using a conjugate gradient method, which combines conjugacy with the steepest descent method. A set of conjugate directions is constructed by using the gradient of the known point, and a search is carried out along the set of directions to find the minimum point of the objective function so as to determine the optimal search direction.
And 7: executing golden section search along the search direction, continuously updating system functions, and performing the next cycle;
performing golden section search along the search direction to update system function aj(t) and bj(t), j is 0,1, 16, and the next cycle is continued until j is 16;
and 8: and terminating the loop and outputting a system function.
2) And an application stage:
and step 9: preparing application data, wherein the data picture is characterized by black background and single and prominent target object;
step 10: the intrinsic decomposition is applied to the given data to obtain the reflection component and the shadow component of the image.
The same or similar reference numerals correspond to the same or similar parts;
the positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (5)

1. An eigen image decomposition method based on partial differential equation learning is characterized by comprising the following steps:
s1: inputting a training data pair;
s2: initializing the control function, since the objective function is non-convex, the convergence direction of the minimization process depends on the initialized local minimum, and when the initialization process does not converge, turning to step S3, executing a loop; otherwise go to step S8;
s3: solving an optimal control equation with PDE constraints;
s4: solving an adjoint equation when the adjoint function takes a specific value;
s5: the derivative of the translational rotational invariants at j 0, 1.
Figure FDA0002301030580000011
Wherein J is the translation and rotation invariant, J is the number of the translation and rotation invariant, aj，bjIs a control function, λjAnd ujIs a positive weighting parameter that is,
Figure FDA0002301030580000012
and phimIs the adjoint function, u is the output image, Ω is the rectangular area occupied by the input image, fΩAn initial function of Ω, M1, 2.., M is the logarithm of the input data samples, inv (u, v) represents the inversion of the matrix (u, v), v is an indicator function, which is introduced for the purpose of collecting large-scale information in the image in order to guide the evolution of u;
s6: determining a search direction using a conjugate gradient method;
s7: performing golden section search along the search direction, continuously updating the system function, and performing the next cycle until j is 16 for training;
s8: terminating the loop and outputting a system function;
s9: preparing application data, wherein the data picture is characterized by black background and single and prominent target object;
s10: carrying out intrinsic decomposition application on the given data by using the obtained system function to obtain a reflection component and a shadow component of the image;
in the step S2, the control function a is controlled by solving the equation (1)j(t), t ═ 0, Δ t, ·,1- Δ t, initialized, at which time b is fixedj(t),j＝0,1,···,16，
F·a(t)＝d (1)
Figure FDA0002301030580000013
Figure FDA0002301030580000014
Wherein, aj(t) and bj(t) is a control function, a (t) { a ═ bj(t) } and b (t) ═ bj(t) is a set of functions defined on Q, respectively for controlling the evolution of u and v, fuAnd fvAre the initial functions of u and v, respectively;
in the step S3, the addition derivative of the jth translational rotation invariant is calculated by introducing the adjoint equation of the formula (2), so that the local optimal value
Figure FDA0002301030580000021
And
Figure FDA0002301030580000022
calculated by a gradient-based algorithm, for umAnd vmWhen M is 1,2, …, M, formula (2) is solved:
Figure FDA0002301030580000023
wherein T is the time when the PDE system completes the visual information processing and outputs the result, Q is Ω x (0, T), and Γ is
Figure FDA0002301030580000024
2. The intrinsic image decomposition method based on partial differential equation learning according to claim 1, wherein the adjoint equation at adjoint function specific value is solved in the step S4:
for the
Figure FDA0002301030580000025
And phimM1, 2, …, M, the adjoint equation is solved, where
Figure FDA0002301030580000026
And phimThe adjoint equation of (c) is as shown in equation (3):
Figure FDA0002301030580000027
Figure FDA0002301030580000028
Figure FDA0002301030580000029
Figure FDA0002301030580000031
Figure FDA0002301030580000032
Figure FDA0002301030580000033
Figure FDA0002301030580000034
Figure FDA0002301030580000035
wherein, OmIs the desired output image, p, q belong to the index set of local variations of { (0,0), (0,1), (0,2), (1,1), (2,0) }.
3. The intrinsic image decomposition method based on partial differential equation learning of claim 2, whichCharacterized in that the derivative of the translational rotational invariants at j 0, 1.., 16 is calculated using equation (4); calculating the derivative of the translational rotational invariant of the control function when j is 0,1
Figure FDA0002301030580000036
And
Figure FDA0002301030580000037
with the help of the adjoint equation for aj(t) and bj(t), the derivative of J in each iteration is as follows:
Figure FDA0002301030580000038
wherein the adjoint function
Figure FDA0002301030580000039
And phimIs the answer to equation (3).
4. The method for decomposing an intrinsic image based on partial differential equation learning according to claim 3, wherein said step S7 is performed by performing a golden section search along the search direction and continuously updating the system function for the next cycle;
performing a golden section search along a search direction, updating the control function aj(t) and bj(t), j-0, 1.., 16, and continue with the next cycle until j-16 is trained.
5. The method for decomposing an intrinsic image based on partial differential equation learning according to claim 4, wherein in step S8, the loop is terminated and the system function is outputted
Figure FDA00023010305800000310
And
Figure FDA00023010305800000311
CN201710138216.8A 2017-03-09 2017-03-09 Intrinsic image decomposition method based on partial differential equation learning Active CN107133953B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710138216.8A CN107133953B (en) 2017-03-09 2017-03-09 Intrinsic image decomposition method based on partial differential equation learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710138216.8A CN107133953B (en) 2017-03-09 2017-03-09 Intrinsic image decomposition method based on partial differential equation learning

Publications (2)

Publication Number Publication Date
CN107133953A CN107133953A (en) 2017-09-05
CN107133953B true CN107133953B (en) 2020-04-24

Family

ID=59720752

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710138216.8A Active CN107133953B (en) 2017-03-09 2017-03-09 Intrinsic image decomposition method based on partial differential equation learning

Country Status (1)

Country Link
CN (1) CN107133953B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104700109A (en) * 2015-03-24 2015-06-10 清华大学 Method and device for decomposing hyper-spectral intrinsic images
CN106355601A (en) * 2016-08-31 2017-01-25 上海交通大学 Intrinsic image decomposition method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9299188B2 (en) * 2013-08-08 2016-03-29 Adobe Systems Incorporated Automatic geometry and lighting inference for realistic image editing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104700109A (en) * 2015-03-24 2015-06-10 清华大学 Method and device for decomposing hyper-spectral intrinsic images
CN106355601A (en) * 2016-08-31 2017-01-25 上海交通大学 Intrinsic image decomposition method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于先验知识的快速人脸光照迁移算法;胡可鑫 等;《计算机辅助设计与图形学学报》;20150930;第27卷(第9期);第1716-1724页 *

Also Published As

Publication number Publication date
CN107133953A (en) 2017-09-05

Similar Documents

Publication Publication Date Title
CN107392097B (en) Three-dimensional human body joint point positioning method of monocular color video
Paragios Geodesic active regions and level set methods: Contributions and applications in artificial vision
Güldenring et al. Self-supervised contrastive learning on agricultural images
CN106446927B (en) It is a kind of to enhance image classification method and system from step
CN110674323B (en) Unsupervised cross-modal Hash retrieval method and system based on virtual label regression
Perrin et al. Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems
Lonini et al. Robust active binocular vision through intrinsically motivated learning
Ptucha et al. LGE-KSVD: Robust sparse representation classification
Yin et al. Sparse representation over discriminative dictionary for stereo matching
CN108510013A (en) The steady tensor principal component analytical method of improvement based on low-rank kernel matrix
Rakotomamonjy Applying alternating direction method of multipliers for constrained dictionary learning
CN112084358B (en) Image-text matching method based on area strengthening network with subject constraint
Yan et al. Monocular depth estimation with guidance of surface normal map
CN110197255A (en) A kind of deformable convolutional network based on deep learning
Huang et al. Contrast-weighted dictionary learning based saliency detection for VHR optical remote sensing images
CN110807741A (en) Training method of image processing network, image denoising method and device
CN112699791A (en) Face generation method, device and equipment of virtual object and readable storage medium
Lu et al. Unsupervised articulated skeleton extraction from point set sequences captured by a single depth camera
CN112164017A (en) Deep learning-based polarization colorization method
Zhang et al. Multi-local-task learning with global regularization for object tracking
Li et al. A self-learning image super-resolution method via sparse representation and non-local similarity
Zhang et al. Incomplete multiview nonnegative representation learning with multiple graphs
CN109685830A (en) Method for tracking target, device and equipment and computer storage medium
Cao et al. PCNet: A structure similarity enhancement method for multispectral and multimodal image registration
Zhou et al. Illumination correction of dyed fabrics approach using Bagging-based ensemble particle swarm optimization–extreme learning machine

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant