CN113870372A - Video hair color conversion method based on deep learning - Google Patents

Video hair color conversion method based on deep learning Download PDF

Info

Publication number
CN113870372A
CN113870372A CN202111012366.7A CN202111012366A CN113870372A CN 113870372 A CN113870372 A CN 113870372A CN 202111012366 A CN202111012366 A CN 202111012366A CN 113870372 A CN113870372 A CN 113870372A
Authority
CN
China
Prior art keywords
hair
hair color
image
target image
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111012366.7A
Other languages
Chinese (zh)
Other versions
CN113870372B (en
Inventor
伍克煜
郑友怡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202111012366.7A priority Critical patent/CN113870372B/en
Priority to PCT/CN2021/126912 priority patent/WO2023029184A1/en
Publication of CN113870372A publication Critical patent/CN113870372A/en
Application granted granted Critical
Publication of CN113870372B publication Critical patent/CN113870372B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a video hair color conversion method based on a deep neural network, which introduces a brightness map (Luminance map) to represent the geometric structure of hair for the first time and provides standardization on time and space to further ensure the stability of continuous frames and the consistency of a generated result and a reference image. The invention also designs three condition modules of color, structure and background to further decouple the hair property. Due to the fact that the luminescence map is used for highly decoupling colors, corresponding modules are independently designed according to various attributes of the hair, and the colors, the structures and the illumination of the hair and the background of the image are highly decoupled, the method achieves high-fidelity hair color conversion by utilizing the advantages. The invention also introduces a discriminator and a circular consistency loss function to generate a more realistic result with stronger time coherence.

Description

Video hair color conversion method based on deep learning
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a video hair color conversion method based on deep learning.
Background
Hair is one of the most important components in portraiture, which also brings about a great deal of excellent work in the field of computer vision, however, the existing work only stays on static hair, the related work on video sequences is still insufficient, and in addition, the hair is very delicate, changeable and complex unlike most parts of human faces. It consists of thousands of thin lines and is affected by light, motion and occlusion, and is therefore difficult to analyze, represent and generate. Existing studies have used gabor filters to extract the direction of hair growth as the geometry of the hair and decoupled from the color, however, one common drawback of this approach is: the gabor filter loses much detail information, resulting in an insecure and easily jittery generated video sequence.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a video hair color conversion method based on deep learning, the standardized luminescence map is used for representing the geometric structure of the hair, three condition modules are designed aiming at each attribute of the hair, the hair attributes are highly decoupled and recombined to achieve the effect of hair color conversion, and the whole training process does not need any paired data. A loop consistency loss (loop consistency loss) is proposed and a discriminator (discriminator) is used to remove the jitter that may be caused during the generation and to enhance the consistency of the generated result with the reference image.
The invention is realized by the following technical scheme:
a video hair color conversion method based on deep learning comprises the following steps:
the method comprises the following steps: converting each frame of a video containing a target image of the hair color to be converted from an RGB space to an LAB space, extracting an L space, standardizing the L space in time and space, and acquiring a characteristic diagram of the target image containing the hair structure and illumination by using a structural characteristic extraction module.
The L space (Luminance map) represents brightness, the structure of the hair can be expressed by the intensity of the brightness, the original illumination is reserved, and the most important point is that the Luminance map can reserve all tiny details to reconstruct a more real image. Because the brightness also relates to the depth of the color, in order to make up for the influence brought by different illumination conditions of different images, the invention provides the method for standardizing the illumination map in time and space and expressing the geometric structure of the hair by using the standardized result, thereby not only ensuring the structure of the hair, but also ensuring the original illumination conditions and generating a more real result.
Step two: selecting a reference image with pre-converted hair colors, extracting hair color features of the reference image by using a color feature extraction module, and superposing the hair color features on a hair mask of a target image to obtain a hair color mask;
step three: extracting a background region characteristic diagram of the target image except hairs by using a background region characteristic extraction module according to the target image hair mask;
step four: inputting the target image extracted in the first step, which comprises the hair structure and the characteristic diagram of illumination, the hair color mask extracted in the second step and the background region characteristic diagram extracted in the third step, into a backbone generation network to integrate and generate the target image with the hair color of the reference image.
The structure feature extraction module, the color feature extraction module, the background region feature extraction module and the backbone generation network are obtained through collected video training.
Further, in the step 1, each frame in the video including the target image of the hair color to be converted is converted from an RGB space to an LAB space, an L space is extracted and normalized in time and space, specifically:
(1.1) converting each frame image of the video containing the target image of the hair color to be converted from CIE RGB to CIE XYZ color space and then from CIE XYZ to LAB color space.
(1.2) extracting L space and calculating the mean and variance of L values of all pixel points of a whole video sequence by using a formula Lt norm＝(LtMean (L)/V normalizes the L space. Where mean (L) represents the mean of L, V represents the variance, and t is the index of the image.
Further, in the step (1.2), a mean and a variance of L values of pixels corresponding to a hair region in the entire video sequence are calculated.
Further, the color feature extraction module includes a partial convolution network (partial convolution) with 4 layers of downsampling and an instance-level averaging pooling layer (instance-wise averaging).
The method comprises the steps of extracting features of a hair region of a reference image based on the advantage of mask extraction by using partial constraint, avoiding feature interference outside the hair region, and compressing the features into a feature vector by using an instance-level average pooling layer to obtain the global color features of the reference image.
Furthermore, the structural feature extraction module comprises a plurality of up-sampling modules and residual blocks which are connected in sequence, the backbone generation network comprises a plurality of residual blocks and down-sampling modules which are connected in sequence, and the structural feature extraction module and the backbone generation network are in a symmetrical structure. In the fourth step, the target image extracted in the first step contains a hair structure and illumination feature map, the hair color mask extracted in the second step is input into a plurality of residual blocks of a backbone generation network after being connected on a feature channel to extract features, the extracted features are input into an up-sampling module, the up-sampling module and the down-sampling module of the structure feature extraction module are connected through jumping to obtain multi-scale features, the target image with the hair color of the reference image is finally obtained by combining the background region feature maps extracted in the third step in the last n down-sampling modules, and n is the number of the background region feature maps.
Further, when the target images comprise continuous multiple frames of target images of the hair color to be converted, the conversion result of the first k frames of target images of the target image of the current hair color to be converted is fed back to the structural feature extraction module and is used as common input with the target image of the current hair color to be converted so as to ensure the time coherence of the generated video sequence.
Further, the loss function adopted by the training comprises: the generated target image with reference image hair colors and the true value of the L1 loss and the perception loss, the network generation countermeasure loss, the generated target image with reference image hair colors and the true value of the feature matching loss, the temporal coherence loss and the cycle coherence loss are expressed as:
Figure BDA0003239427410000031
wherein,
Figure BDA0003239427410000032
the L1 loss and the perceptual loss of the network output image and the true value respectively,
Figure BDA0003239427410000033
is the creation of a network to combat the loss,
Figure BDA0003239427410000034
is the loss of feature matching of the network output image with the true value,
Figure BDA0003239427410000035
is the Temporal coherence loss (Temporal coherence),
Figure BDA0003239427410000036
and
Figure BDA0003239427410000037
is a loss of cyclic consistency; λ is the weight corresponding to the loss, and is a real number.
Further, the cyclic consistency loss training specifically includes, for two video sequences X and Y, first taking the video sequence Y as a reference image, using the above steps of the method of the present invention to convert the hair color of the video sequence X to generate a video sequence X, then taking the video sequence X as a reference image, taking the video sequence Y as a video sequence of the hair color to be converted, and repeating the above steps to generate a new video sequence Y, where the loss function is as follows:
Figure BDA0003239427410000038
Figure BDA0003239427410000039
Figure BDA0003239427410000041
Figure BDA0003239427410000042
wherein I is the target image, IrefRepresenting a reference image, G represents the output of the backbone generation network,
Figure BDA0003239427410000045
shows the result of a hair color conversion, IcycRepresenting the result obtained by two times of conversion, and k +1 represents the number of the fed-back input images;
Figure BDA0003239427410000043
representing the output of the j-th layer of the VGG network, t is the index of the video sequence, and the subscript l represents the l space of the corresponding image, i.e. the luminance map of the image I; d denotes an output of the discriminator,
Figure BDA0003239427410000044
indicating a desire; m denotes the hair mask of the template image, MrefA hair mask representing a reference image, | Y phosphor1Indicating L1 regularization.
The cyclic consistency loss further ensures consistency of the colors of the generated result with the reference image.
The outstanding contributions of the invention are:
the invention provides iHairRecolor, the first method for converting the hair color of a reference image into a video based on deep learning, and the first method for converting the hair color of the reference image into the video based on deep learning, wherein an L space in an LAB space is used for replacing a traditional orientation map as the structure of the hair, and the structure is subjected to temporal and spatial standardization, so that a luminescence map can not only keep fine structural characteristics and original illumination conditions, but also can keep the color height structure of the hair. In addition, a novel cyclic consistency loss is employed to better match the colors between the generated result and the reference image. The invention can train without depending on any pair data and can still be robust and stable on test data. The invention also introduces a discriminator to ensure the time consistency of the generated video sequence, and obtains a more real and smoother result, which is superior to all the existing methods at present.
Drawings
FIG. 1 is a diagram of the network pipeline architecture of the present invention;
fig. 2 is a graph of the hair color conversion results of the present invention.
Detailed Description
Due to the very delicate, variable and complex hair. It consists of thousands of thin lines and is affected by light, motion and occlusion, and is therefore difficult to analyze, represent and generate. The aim of the invention is to obtain a new video sequence in which the hair structure is highly reducing the original video and the hair color remains identical to the reference image. This requires a high degree of structural and color decoupling, typically using gabor filter calculations. The present invention addresses the above problems by proposing temporal and spatial normalization using a luminescence map to represent the structure of hair and align. And three well-designed condition modules such as partial constraint and the like and the proposed loss function are utilized to complete the training of the video hair color conversion model. The method specifically comprises the following steps:
the method comprises the following steps: converting each frame of a video containing a target image of the hair color to be converted from an RGB space to an LAB space, extracting an L space, standardizing the L space in time and space, and acquiring a characteristic diagram of the target image containing the hair structure and illumination by using a structural characteristic extraction module.
Step two: selecting a reference image with pre-converted hair colors, extracting hair color features of the reference image by using a color feature extraction module, and superposing the hair color features on a hair mask of a target image to obtain a hair color mask;
step three: extracting a background region characteristic diagram of the target image except hairs by using a background region characteristic extraction module according to the target image hair mask;
step four: inputting the target image extracted in the first step, which comprises the hair structure and the characteristic diagram of illumination, the hair color mask extracted in the second step and the background region characteristic diagram extracted in the third step, into a backbone generation network to integrate and generate the target image with the hair color of the reference image.
The structure feature extraction module, the color feature extraction module, the background region feature extraction module and the backbone generation network are obtained through collected video training.
Fig. 1 illustrates the structure and data flow direction of the network in the present invention. The process of the invention is further illustrated below with reference to a specific example:
video sequence I (T) { I) for a given T frame1，I2，...，ITAnd reference picture IrefOur goal is to remove the original hair color in i (t) and convert to the same color as the hair in the reference image, while keeping the other hair attributes unchanged. Therefore, the attributes of the hair in the video sequence are first decomposed into shapes, structures, lighting, colors and areas outside the hair.
Wherein the extraction of structural features uses a high degree of decoupling of the luminance and color unique to the LAB space, and the L space is normalized using the following formula:
Figure BDA0003239427410000051
Figure BDA0003239427410000052
where M denotes a mask of the hair region of the target image,
Figure BDA0003239427410000053
a luminance map (luminance map) corresponding to the T-th frame image, i represents a certain pixel in the luminance map,
Figure BDA0003239427410000054
a luminance map corresponding to the T-th frame image after normalization is shown. The effect of background on brightness icon normalization is removed by using M x L in the invention. Simultaneously, all frames of a video sequence are simultaneously calculated to obtain a luminance map, the normalized variance and mean are unified, and the luminance map sequence is eliminatedTo produce a smoother result. The luminance map generation result is shown in fig. 1 (Clip X). Finally, extracting the brightness graph of the target image by a structural feature extraction module (luminance module) to obtain a feature graph containing hair structure and illumination; in this embodiment, the luma Block consists of 3-layer downsampling and 4 residual Block (Resnet Block) structures.
For a reference image with pre-converted hair Color, using a Color feature extraction Module (Color Module) as shown in fig. 1, the Color feature extraction Module includes a partial convolution network (partial convolution) with 4 layers of down-sampling and an instance-level average pooling layer (instance-level averaging), firstly, using a hair segmentation network to obtain a hair mask of a hair region of the reference image, and after the partial convolution with 4 layers of down-sampling, each layer of down-sampling will reduce the resolution of the image, and accordingly, each time the hair mask is updated to avoid interference of features other than hair. The feature map (feature map) obtained after 4 downsampling is further compressed into a 512-dimensional feature vector (feature vector) using an example-level average pooling layer. This not only preserves the global information of hair color but also removes the effects of hair shape and structure differences. Further, the extracted feature vectors are superimposed with the hair mask of the target image by the following formula:
Figure BDA0003239427410000061
wherein A'refRepresenting the feature vectors extracted from the reference image, MrefThe hair mask representing the reference image and M representing the hair mask of the target image. A is a characteristic diagram of colors superposed on M by the feature vector, namely a hair color mask.
Further, a Background region feature extraction Module (Background Module) shown in fig. 1 is used to extract a Background region feature map of the target image except for hair, specifically, firstly, a hair region is removed by using a hair mask of the target image obtained by segmentation, the rest regions are kept unchanged, the rest regions are input into a neural Background region feature extraction Module, and features with different granularities are obtained after two times of downsampling and are combined with newly generated hair features in the last two layers of a backbone generation network (backbone Generator).
As shown in fig. 1, the backbone generation network includes 4 resblocks and 3 layers of upsampling, has a symmetric network structure with the luminence Module, and is input to obtain combined features of a feature map containing hair structures and illumination and a hair color mask after the connection on feature channels of a target image, further extracts features through 4 resblocks, combines multi-scale features in the downsampling in the luminence Module through skip connection, and combines with background features at the last 2 layers to generate an image with reference image hair colors.
As a preferable scheme, if it is necessary to continuously convert hair colors of multiple target images in a video, the images generated after the hair color conversion of the first k frames of images of the target image to be converted can be used as the input of the network together, so that a video sequence with smoother and stronger temporal coherence can be generated. In order to ensure that the input is consistent, the input of the first image and the input of the second image are the same three images.
The structural feature extraction module, the color feature extraction module, the background region feature extraction module and the backbone generation network are obtained according to the collected video training by the following method:
as shown in fig. 1, for two video sequences X and Y, first using the video sequence Y as a reference image, the above steps of the method of the present invention are used to convert the hair color of the video sequence X into a video sequence X, then using the video sequence X as a reference image and the video sequence Y as a video sequence of the hair color to be converted, and repeating the above steps to generate a new video sequence Y, Y should be the same as the video sequence Y, so that the following loss function constraints can be utilized:
Figure BDA0003239427410000071
Figure BDA0003239427410000072
Figure BDA0003239427410000073
Figure BDA0003239427410000074
Figure BDA0003239427410000075
Figure BDA0003239427410000076
wherein,
Figure BDA0003239427410000077
the L1 loss and the perceptual loss of the network output image and the true value respectively,
Figure BDA0003239427410000078
is the creation of a network to combat the loss,
Figure BDA0003239427410000079
is the loss of feature matching of the network output image with the true value,
Figure BDA00032394274100000710
is the Temporal coherence loss (Temporal coherence).
I is the target image, IrefRepresenting a reference image, G represents the output of the backbone generation network,
Figure BDA00032394274100000713
shows the result of a hair color conversion, Icyce.Y represents the result obtained by two times of conversion, and k +1 represents the number of the input images of the structural feature extraction module, which is 3 in the embodiment;
Figure BDA00032394274100000711
represents the output of the j-th layer of the VGG network, and the subscript l represents the l space of the corresponding image, i.e. the luminance map of the image I; d denotes an output of the discriminator,
Figure BDA00032394274100000712
indicating a desire.
λ is the weight corresponding to the loss, and in this embodiment, λ is taken separately1＝10、λp＝10、λadv＝0.1、
Figure BDA0003239427410000081
λchromatic＝10、λstable1, finally training to obtain a network, and fig. 2 is a graph of the hair color conversion result by using the invention, and as can be seen from the graph, the invention can adapt to different hairstyles, from long to short, simple to complex, and also can adapt to various colors, including some mixed colors, and the generated target image can keep consistent with the reference image, and meanwhile, the generated target image also keeps the illumination condition in the source image.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.

Claims (8)

1. A video hair color conversion method based on deep learning is characterized by comprising the following steps:
the method comprises the following steps: converting each frame of a video containing a target image of the hair color to be converted from an RGB space to an LAB space, extracting an L space, standardizing the L space in time and space, and acquiring a characteristic diagram of the target image containing the hair structure and illumination by using a structural characteristic extraction module.
Step two: selecting a reference image with pre-converted hair colors, extracting hair color features of the reference image by using a color feature extraction module, and superposing the hair color features on a hair mask of a target image to obtain a hair color mask;
step three: extracting a background region characteristic diagram of the target image except hairs by using a background region characteristic extraction module according to the target image hair mask;
step four: inputting the target image extracted in the step one including the hair structure and the characteristic diagram of illumination, the hair color mask obtained in the step two and the background region characteristic diagram extracted in the step three into a main generating network to integrate and generate the target image with the hair color of the reference image.
The structure feature extraction module, the color feature extraction module, the background region feature extraction module and the backbone generation network are obtained through collected video training.
2. The method for converting the hair color of the video based on the deep learning as claimed in claim 1, wherein in the step 1, each frame of the video including the target image of the hair color to be converted is converted from RGB space to LAB space, L space is extracted and normalized in time and space, specifically:
(1.1) converting each frame image of the video containing the target image of the hair color to be converted from CIE RGB to CIE XYZ color space and then from CIE XYZ to LAB color space.
(1.2) extracting L space and calculating the mean and variance of L values of all pixel points of a whole video sequence by using a formula Lt norm＝(LtMean (L)/V normalizes the L space. Where mean (L) represents the mean of L, V represents the variance, and t is the index of the image.
3. The method according to claim 1, wherein in step (1.2), the mean and variance of the L values of the pixels corresponding to the hair region in the whole video sequence are calculated.
4. The deep learning based video hair color conversion method according to claim 1, wherein the color feature extraction module comprises a 4-layer down-sampled partial convolution network and an instance-level average pooling layer.
5. The method according to claim 1, wherein the structural feature extraction module comprises a plurality of upsampling modules and residual blocks which are connected in sequence, the backbone generation network comprises a plurality of residual blocks and downsampling modules which are connected in sequence, the structural feature extraction module and the backbone generation network are in a symmetrical structure, and the corresponding upsampling modules are connected with the downsampling modules in a jumping manner. In the fourth step, the target image extracted in the first step comprises a hair structure and illumination characteristic diagram, the hair color mask extracted in the second step is input into a plurality of residual blocks of a backbone generation network after being connected on a characteristic channel to extract characteristics, the extracted characteristics are input into an upper sampling module to sequentially obtain multi-scale characteristics, and the target image with the hair color of the reference image is finally obtained by combining the background area characteristic diagram extracted in the third step in the last n lower sampling modules.
6. The method for converting hair color of video based on deep learning of claim 1, wherein when the method comprises a plurality of consecutive target images of hair color to be converted, the conversion result of the first k frames of target images of the target image of hair color to be converted is fed back to the structural feature extraction module and the target image of hair color to be converted is used as common input to ensure temporal coherence of the generated video sequence.
7. The method of claim 1, wherein the training uses a loss function comprising: the generated target image with reference image hair color and the true value of the L1 loss and the perception loss, the network generation countermeasure loss, the generated target image with reference image hair color and the true value of the feature matching loss, the temporal coherence loss and the cycle coherence loss.
8. The method according to claim 7, wherein the cyclic consistency loss training specifically comprises, for two video sequences X and Y, first using the video sequence Y as a reference image, using the above steps to transform the hair color of the video sequence X into the video sequence X, then using the video sequence X as a reference image, using the video sequence Y as a video sequence of the hair color to be transformed, and repeating the above steps to generate a new video sequence Y, wherein the loss function is as follows:
Figure FDA0003239427400000021
Figure FDA0003239427400000022
Figure FDA0003239427400000031
Figure FDA0003239427400000032
Figure FDA0003239427400000033
wherein I is the target image, IrefRepresenting a reference image, G represents the output of the backbone generation network,
Figure FDA0003239427400000034
shows the result of a hair color conversion, IcycRepresenting the result obtained by two times of conversion, and k +1 represents the number of the fed-back input images;
Figure FDA0003239427400000035
representing the output of the j-th layer of the VGG network, t is the index of the video sequence, and the subscript l represents the l space of the corresponding image, i.e. the luminance map of the image I; d denotes an output of the discriminator,
Figure FDA0003239427400000036
indicating a desire; m denotes the hair mask of the template image, MrefA hair mask representing a reference image, | Y phosphor1Indicating L1 regularization.
CN202111012366.7A 2021-08-31 2021-08-31 Video hair color conversion method based on deep learning Active CN113870372B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202111012366.7A CN113870372B (en) 2021-08-31 2021-08-31 Video hair color conversion method based on deep learning
PCT/CN2021/126912 WO2023029184A1 (en) 2021-08-31 2021-10-28 Video hair color conversion method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111012366.7A CN113870372B (en) 2021-08-31 2021-08-31 Video hair color conversion method based on deep learning

Publications (2)

Publication Number Publication Date
CN113870372A true CN113870372A (en) 2021-12-31
CN113870372B CN113870372B (en) 2024-06-21

Family

ID=78988977

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111012366.7A Active CN113870372B (en) 2021-08-31 2021-08-31 Video hair color conversion method based on deep learning

Country Status (2)

Country Link
CN (1) CN113870372B (en)
WO (1) WO2023029184A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596370A (en) * 2022-03-04 2022-06-07 深圳万兴软件有限公司 Video color conversion method, device, computer equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881011A (en) * 2012-08-31 2013-01-16 北京航空航天大学 Region-segmentation-based portrait illumination transfer method
CN108629819A (en) * 2018-05-15 2018-10-09 北京字节跳动网络技术有限公司 Image hair dyeing treating method and apparatus
CN110458906A (en) * 2019-06-26 2019-11-15 重庆邮电大学 A kind of medical image color method based on depth color transfer
WO2020037965A1 (en) * 2018-08-21 2020-02-27 北京大学深圳研究生院 Method for multi-motion flow deep convolutional network model for video prediction
US11017560B1 (en) * 2019-04-15 2021-05-25 Facebook Technologies, Llc Controllable video characters with natural motions extracted from real-world videos
CN113313657A (en) * 2021-07-29 2021-08-27 北京航空航天大学杭州创新研究院 Unsupervised learning method and system for low-illumination image enhancement

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10679328B2 (en) * 2018-05-23 2020-06-09 Adobe Inc. Machine learning techniques for increasing color consistency across videos
CN109658330B (en) * 2018-12-10 2023-12-26 广州市久邦数码科技有限公司 Color development adjusting method and device
CN110969631B (en) * 2019-11-25 2023-04-11 杭州小影创新科技股份有限公司 Method and system for dyeing hair by refined photos
CN111524205A (en) * 2020-04-23 2020-08-11 北京信息科技大学 Image coloring processing method and device based on loop generation countermeasure network
CN112614060B (en) * 2020-12-09 2024-10-18 深圳数联天下智能科技有限公司 Face image hair rendering method and device, electronic equipment and medium
CN112529914B (en) * 2020-12-18 2021-08-13 北京中科深智科技有限公司 Real-time hair segmentation method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102881011A (en) * 2012-08-31 2013-01-16 北京航空航天大学 Region-segmentation-based portrait illumination transfer method
CN108629819A (en) * 2018-05-15 2018-10-09 北京字节跳动网络技术有限公司 Image hair dyeing treating method and apparatus
WO2020037965A1 (en) * 2018-08-21 2020-02-27 北京大学深圳研究生院 Method for multi-motion flow deep convolutional network model for video prediction
US11017560B1 (en) * 2019-04-15 2021-05-25 Facebook Technologies, Llc Controllable video characters with natural motions extracted from real-world videos
CN110458906A (en) * 2019-06-26 2019-11-15 重庆邮电大学 A kind of medical image color method based on depth color transfer
CN113313657A (en) * 2021-07-29 2021-08-27 北京航空航天大学杭州创新研究院 Unsupervised learning method and system for low-illumination image enhancement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
伍克煜;刘峰江;许浩;张浩天;王贝贝: "合成生物学基因设计软件:iGEM设计综述", 生物信息学, no. 001, 31 December 2020 (2020-12-31) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596370A (en) * 2022-03-04 2022-06-07 深圳万兴软件有限公司 Video color conversion method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN113870372B (en) 2024-06-21
WO2023029184A1 (en) 2023-03-09

Similar Documents

Publication Publication Date Title
CN112734646B (en) Image super-resolution reconstruction method based on feature channel division
Huang et al. Deep learning for image colorization: Current and future prospects
CN108921786A (en) Image super-resolution reconstructing method based on residual error convolutional neural networks
CN111783658B (en) Two-stage expression animation generation method based on dual-generation reactance network
CN111445564B (en) Face texture image generation method, device, computer equipment and storage medium
CN111145290B (en) Image colorization method, system and computer readable storage medium
CN105447840B (en) The image super-resolution method returned based on active sampling with Gaussian process
CN105550989B (en) The image super-resolution method returned based on non local Gaussian process
CN113283444B (en) Heterogeneous image migration method based on generation countermeasure network
CN110097609A (en) A kind of fining embroidery texture moving method based on sample territory
CN111696035A (en) Multi-frame image super-resolution reconstruction method based on optical flow motion estimation algorithm
CN111784582A (en) DEC-SE-based low-illumination image super-resolution reconstruction method
CN113284061B (en) Underwater image enhancement method based on gradient network
WO2021052103A1 (en) Image generation method and apparatus, and computer
CN112991371B (en) Automatic image coloring method and system based on coloring overflow constraint
CN110443865A (en) Multispectral imaging method and apparatus based on RGB camera and deep neural network
Wang et al. No-reference stereoscopic image quality assessment using quaternion wavelet transform and heterogeneous ensemble learning
CN113870372B (en) Video hair color conversion method based on deep learning
CN115239885A (en) Face reconstruction method and device based on key point recognition
CN110415816B (en) Skin disease clinical image multi-classification method based on transfer learning
CN117196940A (en) Super-resolution reconstruction method suitable for real scene image based on convolutional neural network
Neagoe An optimum 2d color space for pattern recognition.
Pucci et al. Pro-ccaps: progressively teaching colourisation to capsules
CN113077385A (en) Video super-resolution method and system based on countermeasure generation network and edge enhancement
Kim et al. RPF: Reference-based progressive face super-resolution without losing details and identity

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant