WO2017181332A1 - 一种基于单幅图像的全自动三维头发建模方法 - Google Patents

一种基于单幅图像的全自动三维头发建模方法 Download PDF

Info

Publication number
WO2017181332A1
WO2017181332A1 PCT/CN2016/079613 CN2016079613W WO2017181332A1 WO 2017181332 A1 WO2017181332 A1 WO 2017181332A1 CN 2016079613 W CN2016079613 W CN 2016079613W WO 2017181332 A1 WO2017181332 A1 WO 2017181332A1
Authority
WO
WIPO (PCT)
Prior art keywords
hair
sample
dimensional
mask
neural network
Prior art date
Application number
PCT/CN2016/079613
Other languages
English (en)
French (fr)
Inventor
周昆
柴蒙磊
Original Assignee
浙江大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江大学 filed Critical 浙江大学
Priority to PCT/CN2016/079613 priority Critical patent/WO2017181332A1/zh
Publication of WO2017181332A1 publication Critical patent/WO2017181332A1/zh
Priority to US16/163,556 priority patent/US10665013B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24143—Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Definitions

  • the present invention relates to the field of three-dimensional modeling based on a single image, and more particularly to a method for automatically three-dimensional modeling of hair of a portrait picture.
  • Image-based hair modeling is an effective way to create high quality hair geometry. Hair collection techniques based on multi-view images often require complex equipment configurations and long processing cycles (LUO, L., LI, H., AND RUSINKIEWICZ, S. 2013. Structure-aware hair capture.
  • ACM Transactions on Graphics (TOG) 32,4,76.) (ECHEVARRIA,JI,BRADLEY,D.,GUTIERREZ,D.,AND BEELER,T.2014.Capturing and stylizing hair for 3d fabrication.ACM Transactions on Graphics(TOG)33,4,125.) (HU, L., MA, C., LUO, L., AND LI, H.2014. Robust hair capture using simulated examples.
  • ACM Transactions on Graphics (TOG) 33, 4, 126.) (HU, L., MA, C., LUO, L., WEI, L.-Y., AND LI, H.2014. Capturing braided hairstyles.
  • ACM Transactions on Graphics(TOG)33,6,225.) so it is not suitable for ordinary users, but also to generate A lot of 3D hair models are too expensive.
  • Recent hair image modeling techniques based on single images have yielded impressive results.
  • the prior art uses different types of prior knowledge to perform modeling, such as using layer boundaries and occlusion (CHAI, M., WANG, L., WENG, Y., YU, Y., GUO, B., AND ZHOU, K.2012.Single-view hair modeling for portrait manipulation.
  • ACM Transactions on Graphics(TOG)34,4,125.) and using shadow cues (CHAI,M.,LUO,L.,SUNKAVALLI,K.,CARR,N. , HADAP, S., 747AND ZHOU, K.2015. High-quality hair modeling from a single portrait photo.
  • the object of the present invention is to provide a new fully automatic three-dimensional hair modeling method based on single image in view of the deficiencies of the prior art, and obtain a robust and high-precision hair segmentation result by layered deep convolutional neural network. And direction estimation, and then use the data-driven method to match the three-dimensional hair sample model to the segmented hair and pattern to obtain the final hair model expressed in the hair bundle.
  • the result of this method is comparable to the results of current methods of user interaction, and has high practical value.
  • the invention is realized by the following technical solution, a fully automatic three-dimensional hair modeling method based on a single image, comprising the following steps:
  • Pretreatment of hair training data Mark the two-dimensional mask and growth direction of hair, and obtain the classification of different hairstyles by unsupervised clustering method.
  • the invention has the beneficial effect that the invention firstly proposes a fully automatic three-dimensional hair modeling method based on a single image, and completes high-precision and robust hair segmentation and growth direction estimation by means of a deep neural network, and is completed by a data driven method. Efficient hair matching and modeling.
  • the effect achieved by the present invention is comparable to that achieved by the method of user interaction today, and is automatic and efficient, and can be used for modeling a wide range of Internet portrait pictures.
  • Figure 1 is a marker diagram of training data; left column: original image; middle column: hair segmentation mask map; right column: direction-based sub-region segmentation and pattern;
  • FIG. 2 is a flow chart of hair styling, hair segmentation and direction estimation; for the estimation of the hair region, the category of the hairstyle is first identified, and then the corresponding segmentation network and direction estimation network are selected according to the category to obtain a segmentation mask and a pattern;
  • Figure 3 is a process diagram of segmentation and recombination of a three-dimensional hair sample to generate a new sample; each row of left column: two original hair samples; three columns to the right of each row: three new hair samples generated by decomposing the original hair bundle and then combining;
  • Figure 4 is a three-dimensional hair result diagram automatically modeled from a single image of the present invention; each line from left to right: input image, automatically segmented hair mask map and direction estimation map, deformed matching hair sample, and finally 3 Final hair from different perspectives A bundle of hair models.
  • the core technology of the invention utilizes a deep neural network to perform fully automatic high-precision hair segmentation and direction estimation, and uses a data-driven hair matching method to complete high-quality hair three-dimensional modeling.
  • the method is mainly divided into the following four main steps: preprocessing of hair training data, hair segmentation and direction estimation based on deep neural network, generation and organization of 3D hair samples, and data-driven 3D hair modeling.
  • Pretreatment of hair training data marking the two-dimensional mask and growth direction of the hair, and obtaining the classification of different hairstyles by unsupervised clustering method;
  • the continuous direction range [0, 2p) is discretized into four intervals ([0, 0.5p), [0.5p, p), [p, 1.5p), [1.5p, 2p)), and then These four markers are assigned to each pixel to obtain a direction marker map M d . Pixels that are not in the hair area will also have a marker.
  • Figure 1 shows an example of marking a hair mask and a pattern.
  • the K-means clustering method is used to classify the hairstyles of the training pictures into four categories.
  • Two histograms H a, H b using L1 distance from Earth Mover paradigm (EMD) (LING, H., AND OKADA, K.2007.An efficient earth mover's distance algorithm for robust histogram comparison.Pattern Analysis and Machine Intelligence , IEEE Transactions on 29, 5, 820 840–853.) Calculation.
  • Each cluster center g is the smallest member of the sum of the distances from other members in the class.
  • Hair segmentation and direction estimation based on deep neural network training deep neural network based on the marker data of step 1, using the layered deep convolutional neural network obtained by training to complete hair type recognition, hair segmentation and hair growth direction estimation The algorithm flow is shown in Figure 2.
  • step 1.2 Given a portrait photo, first use the face alignment method in step 1.2 to detect a series of face feature points and align the photo to the reference face coordinate system. Next, for each type of hair distribution, 20 typical hair wrapping boxes are selected and a set of candidate hair areas are created by rotating and zooming the face area of the photo. A typical hair wrapping box is created by pre-clustering the bounding boxes of each type of hair. These candidate areas will be cropped and independently passed to subsequent recognizers for hairstyle recognition.
  • Hair styling is performed based on the hair area obtained in step 2.1.
  • Hairstyle recognition is a deep convolutional neural network structure using R-CNN (GIRSHICK, R., DONAHUE, J., DARRELL, T., AND MALIK, J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, IEEE, 580-587.), and further learning on the hair training data labeled in step 1.1.
  • the segmentation and direction estimation of the hair region is performed. Hair segmentation and hair direction estimation are based on the public deep neural network VGG16 (SIMONYAN, K., AND ZISSERMAN, A.2014.
  • the network is Based on the public data set ImageNet pre-trained to identify 1000 categories of classification networks, the present invention has been modified based on this to make the output of the network is the mark of each pixel (the output number of the splitter is 2, the direction estimator The number of output markers is 5).
  • the last two layers of the 2. 2 max-pooling layer are removed to improve the layer resolution of the network, and the acceptance domain following the convolutional layer is also extended from 3. 3 and 7. 7 to 5. 5 and 25 ⁇ 25 (filled with 0).
  • the loss layer calculates the sum of the cross entropy between the output marker and the manual marker on the entire image (since there are three max-pooling layers in VGG16, the resolution of the image is downsampled by eight times).
  • the output marker map is upsampled to the original image size by a bilinear difference and improved using a fully connected CRF.
  • the image size of the present invention during the training and testing phases is 512 ⁇ 512.
  • the face detector first aligns the image into the face coordinate system and produces a set of hair region estimates around the face.
  • the hairstyle recognition network tests each candidate area and selects the highest score as the hair style of the hair.
  • the hair segmentation network and direction estimation network corresponding to this category are sequentially applied to I.
  • the output of the segmentation network is the hair segmentation mask M I (with an alpha channel) of the same size as I
  • the output of the direction estimation network is an equal-sized direction marker map that is combined with the non-directional orientation map. Generate the final pattern D I .
  • step 3.1 Decompose the three-dimensional hair sample obtained in step 3.1 into different hair bundle groups, first using a simplified internal expression to represent each hair bundle ⁇ S i ⁇ , that is, using a uniformly divided polyline passing through the center of the hair bundle And the average radius r i is expressed. Then for each hair model, its hair bundles are clustered into different hair bundle groups, and the distance between the hair bundles is defined as:
  • S a and S b are hair bundles
  • n a and n b are the number of fold lines of the hair bundle. Is a broken line, r a , r b is the average radius of the hair bundle.
  • Each hair model is broken down into ten hair bundle groups, and these hair bundle groups are combined to generate more new samples.
  • Figure 3 shows an example of hair sample decomposition and recombination to generate a new sample.
  • the present invention organizes all models in ascending order according to the size of the hair mask area of the front view of the sample. To further improve the matching efficiency, two projection images are further generated for each hair sample H:
  • Hair area mask It is a mask of a two-dimensional projection of a hair sample.
  • the present invention uses Gaussian filtering to smooth the processing of the mask map.
  • Direction map The present invention uses color to represent the projected hair direction, i.e., the XYZ of the direction vector represents the value of the RGB three channels. This maps the two-dimensional projection of the hair sample.
  • the present invention uniformly samples 6 angles in the range of [-p/4, p/4] of yaw and pitch angles. degree. Thus each sample will have 6.6 sets of hair mask maps and patterns. For subsequent matching calculations, all plots are downsampled to 100 ⁇ 100.
  • the input image is aligned into the coordinate system of the sample projection map. Then compare the hair mask area
  • the mask area of the sample is preserved in the range of (0.8
  • the present invention For each sample that is compared by the first step, the present invention further includes a hair mask map and a pattern of the sample. Hair mask and pattern with input image Compare. If the input image is not a front view, the present invention compares the closest view from the pre-computed 6 ⁇ 6 set of sample projections for comparison. The comparison of the hair mask map is based on the distance field of the M I boundary (BALAN, AO, SIGAL, L., BLACK, MJ, DAVIS, JE, AND HAUSSECKER, HW2007. Detailed human shape and pose from images. In Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on, IEEE, 1–8.) Calculate:
  • the hair mask map of the sample and the hair mask map of the input image Indicates the difference in symmetry between the two masks, Is the value of the distance field.
  • the distance of the pattern is defined as the sum of the pixel's direction difference d d .[0,p):
  • the boundary between the sample hair mask map and the mask image of the input image is matched.
  • For each candidate sample H first transform it to the pose of the face in I, and then render the hair mask and pattern (M H , D H ) according to the method of step 3.3.
  • the resolution of the rendered image here is the same as the input graph, not the downsampled thumbnail.
  • 200/2000 points ⁇ P H ⁇ / ⁇ P I ⁇ are uniformly sampled at the boundaries of the mask M H /M I , respectively.
  • For each boundary point P i H /P i I mark its position as Outward normal is marked as
  • the point-to-point correspondence between the calculated boundaries is M( ⁇ P H ⁇ fi ⁇ P I ⁇ ).
  • For each point of the hair mask boundary of the candidate model It is the optimal corresponding point in the input hair mask boundary It is obtained by optimizing the following matching energy equation:
  • E P and E e are energy terms that measure point matching and edge matching.
  • E P wants the position and normal of the point pair to be as close as possible, and the value of the weight l n is 10;
  • E e wants the mapping M to maintain the length of the original boundary as much as possible:
  • E F (W) is the Frobenius paradigm corresponding to the second-order partial derivative of the matrix W.
  • each vertex v of the candidate sample model H is transformed to the target v' by the following optimization function:
  • V H is all vertices of sample H.
  • W(v i ) is the corresponding position of v i , and its XY coordinate is obtained by the above-described global smoothing mapping function W while keeping the Z coordinate unchanged.
  • the optimization function can be solved using the inexact Gaussian Newton method (HUANG, J., SHI, X., LIU, X., ZHOU, K., WEI, L.-Y., TENG, S.-H., BAO, H., GUO, B., AND SHUM, H.-Y.2006.
  • HUANG Gaussian Newton method
  • the final pattern is compared on the full pixel map (the comparison function is identical to step 4.1), and the pattern matching the best model H* is selected to generate the final hair model.
  • H* is then transformed into a three-dimensional directional body expression within the bounding box of the entire model, and then the direction of the H* direction vector and the surface normal of the scalp area is used as a constraint to spread the direction throughout the body.
  • the seeds are evenly sampled on the scalp to generate 10,000 hair bundles.
  • these hair bundles are deformed according to the growth direction estimation map to obtain the final hair model (HU, L., MA, C., LUO, L., AND LI, H.2015.
  • Single-view hair modeling using a hairstyle database ACM Transactions on Graphics (TOG) 34, 4, 125.).
  • Figure 4 shows an example of generating a final hair model from a single image.
  • the inventors implemented an embodiment of the present invention on a machine equipped with an Intel Core i7-3770 central processor, an NVidia GTX 970 graphics processor and 32 GB of memory.
  • the inventors have used all of the parameter values listed in the detailed description to obtain all the experimental results shown in the drawings.
  • the present invention can efficiently generate three-dimensional models of a plurality of hairstyles from a large number of Internet pictures, and these hair-level three-dimensional models closely match the input images. For a typical 800 ⁇ 800 image, the entire process can be completed in less than 1 minute: hair segmentation and orientation estimation is less than 3 seconds, matching and deformation of 3D hair samples takes approximately 20 seconds, and final hair bundle generation is less than 30 second.
  • it takes an average of 1 minute to process a picture; the decomposition and regeneration of the original 3D hair sample takes less than 10 hours; and the training of the neural network takes about 8 hours.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Medical Informatics (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

一种基于单幅人像图像的全自动三维头发建模方法，该方法主要分为四个步骤：头发图像训练数据的生成，基于分层深度神经网络的头发分割和生长方向估计，三维头发样本的生成和组织，以及数据驱动的三维头发建模；该方法可以全自动地鲁棒地生成完整的高质量的三维模型，质量达到当前最先进的基于用户交互的技术的水平。该方法可以用在一系列应用中，如人像发型编辑，发型风格空间的浏览，以及搜索相似发型的互联网图像。

Description

一种基于单幅图像的全自动三维头发建模方法 技术领域
本发明涉及基于单幅图像的三维建模领域，尤其涉及对人像图片的头发进行自动三维建模的方法。
背景技术
基于图像的头发建模是一个创造高质量头发几何的有效途径。基于多视角图像的头发采集技术经常需要复杂的设备配置和较长的处理周期(LUO,L.,LI,H.,AND RUSINKIEWICZ,S.2013.Structure-aware hair capture.ACM Transactions on Graphics(TOG)32,4,76.)(ECHEVARRIA,J.I.,BRADLEY,D.,GUTIERREZ,D.,AND BEELER,T.2014.Capturing and stylizing hair for 3d fabrication.ACM Transactions on Graphics(TOG)33,4,125.)(HU,L.,MA,C.,LUO,L.,AND LI,H.2014.Robust hair capture using simulated examples.ACM Transactions on Graphics(TOG)33,4,126.)(HU,L.,MA,C.,LUO,L.,WEI,L.-Y.,AND LI,H.2014.Capturing braided hairstyles.ACM Transactions on Graphics(TOG)33,6,225.)，所以不适用于普通用户，而且要生成大量三维头发模型的话代价太大。
最近基于单幅图像的头发建模技术取得了令人印象深刻的结果。现有技术使用不同类型的先验知识完成建模，如使用图层边界和遮挡(CHAI,M.,WANG,L.,WENG,Y.,YU,Y.,GUO,B.,AND ZHOU,K.2012.Single-view hair modeling for portrait manipulation.CM Transactions on Graphics(TOG)31,4,116)(CHAI,M.,WANG,L.,WENG,Y.,JIN,X.,AND ZHOU,K.2013.Dynamic hair manipulation in images and videos.ACM Transactions on Graphics(TOG)32,4,75.)，使用三维头发模型数据库(HU,L.,MA,C.,LUO,L.,AND LI,H.2015.Single-view hair modeling using a hairstyle database.ACM Transactions on Graphics(TOG)34,4,125.)，以及使用阴影线索(CHAI,M.,LUO,L.,SUNKAVALLI,K.,CARR,N.,HADAP,S.,747AND ZHOU,K.2015.High-quality hair modeling from a single portrait photo.ACM Transactions on Graphics(TOG)34,6,204.)。但是这些技术都需要不同类型的用户交互，如需要手动将头发从图片中分割出来，或需要用户提供笔画提供头发方向信息，或需要用户画二维发束完成搜索。这些用户交互通常需要5分钟，而得到最终结果需要20分钟左右，这就限制了大规模头发模型的生成。和上述方法不同，本发明是全自动， 而且效率高，可以处理互联网级别的大量图片。
发明内容
本发明的目的在于针对现有技术的不足，提供了一种新的全自动的基于单幅图像的三维头发建模方法，通过分层深度卷积神经网络取得鲁棒、高精度的头发分割结果和方向估计，再利用数据驱动的方法将三维头发样本模型匹配到分割出的头发和方向图，得到最终的以发束表达的头发模型。该方法的结果可媲美当前借助用户交互的方法的结果，具有很高的实用价值。
本发明是通过以下技术方案实现的，一种基于单幅图像的全自动三维头发建模方法，包括以下步骤：
(1)头发训练数据的预处理：标记头发的二维掩码和生长方向，并通过无监督聚类方法得到不同发型的分类。
(2)全自动高精度的头发分割方法：基于步骤(1)的标记数据训练深度神经网络，利用训练得到的分层深度卷积神经网络完成头发的类型识别，头发的分割和头发生长方向的估计；
(3)三维头发样本的生成和组织：通过对原始头发模型的发束分解和再组合生成大量新发型样本，并投影生成二维掩码图和方向图，方便后续匹配；
(4)数据驱动的头发建模：将步骤(3)中的三维头发样本和步骤(2)分割出的头发掩码图和方向图进行匹配、变形，并生成最终模型；
本发明的有益效果是，本发明首次提出了基于单幅图像的全自动三维头发建模的方法，借助深度神经网络完成高精度鲁棒的头发分割和生长方向估计，并借助数据驱动的方法完成高效的头发匹配和建模。本发明取得的效果可媲美当今借助用户交互的方法取得的效果，自动高效，可用于大范围的互联网人像图片的建模。
附图说明
图1是训练数据的标记图；左列：原始图片；中列：头发分割掩码图；右列：基于方向的子区域分割和方向图；
图2是发型识别、头发分割和方向估计的流程图；给定头发区域的估计，首先识别发型的类别，再根据类别选择对应的分割网络和方向估计网络得到分割掩码图和方向图；
图3是三维头发样本的分割和再组合生成新样本的过程图；每行左列：两个原始头发样本；每行右边三列：分解原始发束再组合生成的三个新头发样本；
图4是本发明自动从单幅图像建模出的三维头发结果图；每行从左到右：输入图像，自动分割的头发掩码图和方向估计图，变形后的匹配头发样本，最终3个不同视角下的最终发 束级别的头发模型。
具体实施方式
本发明的核心技术利用深度神经网络完成全自动的头发高精度分割和方向估计，并利用数据驱动的头发匹配方法，完成高质量头发三维建模。该方法主要分为如下四个主要步骤：头发训练数据的预处理、基于深度神经网络的头发分割和方向估计、三维头发样本的生成和组织、数据驱动的三维头发建模。
1.头发训练数据的预处理：标记头发的二维掩码和生长方向，并通过无监督聚类方法得到不同发型的分类；
1.1训练数据标记：
使用两万张人像照片作为训练数据，这些照片具有清晰可见的人脸和头发，并且具有常见的发型和足够的光照亮度。使用PaintSelection(LIU,J.,SUN,J.,AND SHUM,H.-Y.2009.Paint selection.In ACM Transactions on Graphics(ToG),vol.28,ACM,69.)抠图得到头发的二值区域掩码Mh。对于每张照片，将头发区域Mh分成数个具有一致的平滑变化的头发生长方向子区域。对于每个子区域用一个笔画标记发束的生长方向，然后将该方向传播到子区域所有像素，并和在每个像素上计算的不定向朝向图O合在一起产生方向图D。最后，将连续的方向范围[0,2p)离散化为四个区间([0,0.5p)，[0.5p,p)，[p,1.5p)，[1.5p,2p))，然后将这四个标记分配给每个像素得到方向标记图Md。不在头发区域的像素也会有一个标记。图1给出了标记头发掩码图和方向图的例子。
1.2发型分类计算
对于每个标记的图片I，首先使用鲁棒的人脸对齐方法(CAO,X.,WEI,Y.,WEN,F.,AND SUN,J.2014.Face alignment by explicit shape regression.International Journal of Computer Vision 107,2,177–190.)来检测和定位人脸标志，再将I匹配到参考人脸坐标系中的I'，完成大小和正方向的矫正。接着围绕人脸中心的极坐标系统构建了环形分布直方图(划分为nH个区间，nH＝16)。每个区间记录了极角度落在该区间的头发像素的数目。归一化后，该直方图可被看成图片的特征向量。最后，基于这些分布特征向量，使用K-means聚类方法，将训练图片的发型分为四类。两个直方图Ha、Hb的距离使用L1范式的Earth Mover距离(EMD)(LING,H.,AND OKADA,K.2007.An efficient earth mover’s distance algorithm for robust histogram comparison.Pattern Analysis and Machine Intelligence,IEEE Transactions on 29,5,820 840–853.)计算。各聚类中心g是和类内其它成员距离之和最 小的成员。
2.基于深度神经网络的头发分割和方向估计：基于步骤1的标记数据训练深度神经网络，利用训练得到的分层深度卷积神经网络完成头发的类型识别，头发的分割和头发生长方向的估计；算法流程如图2所示。
2.1头发区域的估计
给定一张人像照片，首先使用步骤1.2中的人脸对齐方法检测一系列人脸特征点，将该照片对齐到参考人脸坐标系。接着，对于每一类发型分布，挑选20个典型的头发包围盒，通过旋转和缩放对齐到照片的人脸区域，产生一组候选头发区域。典型的头发包围盒是通过将每一类头发的包围盒预聚类产生。这些候选区域将被裁剪，并独立传给后续识别器进行发型识别。
2.2发型识别，头发分割和方向估计
基于步骤2.1得到的头发区域，进行发型识别。发型识别是使用R-CNN的深度卷积神经网络结构(GIRSHICK,R.,DONAHUE,J.,DARRELL,T.,AND MALIK,J.2014.Rich feature hierarchies for accurate object detection and semantic segmentation.In Computer Vision and Pattern Recognition(CVPR),2014IEEE Conference on,IEEE,580–587.)，并在步骤1.1标记好的头发训练数据上进行进一步学习。得到发型的类别后，进行头发区域的分割和方向估计。头发分割和头发方向估计都是基于公共的深度神经网络VGG16设计的(SIMONYAN,K.,AND ZISSERMAN,A.2014.Very deep convolutional networks for large-scale image recognition.arXivpreprint859arXiv:1409.1556.),该网络是在公共数据集ImageNet上预训练的识别1000种类别的分类网络，本发明在此基础上做了改动使得网络的输出是每个像素的标记(分割器的输出标记数是2，方向估计器的输出标记数是5)。首先，最后两层的2·2的max-pooling层被去除，以提高网络的层分辨率，而且跟随卷积层之后的接受域也分别从3·3和7·7扩展为5·5和25·25(用0填充)。其次，所有的全连接层被换成了卷基层，这样可以让单一识别网络和本发明的逐像素标记的分割器兼容。第三，在训练阶段，损失层在整张图像上计算了输出标记和人工标注标记间的交叉熵之和(由于VGG16中有三个max-pooling层，图像的分辨率降采样了八倍)。最后，在测试阶段，通过双线性差值将输出的标记图升采样到原图片大小，并使用全连接的CRF进行改善。
本发明在训练和测试阶段的图像大小均为512·512。在测试阶段，给定人像照片I，人脸检测器首先将图像对齐到人脸坐标系中并产生围绕人脸的一组头发区域估计。然后发型识别网络测试每个候选区域，选择得分最高的作为类型作为头发的发型类别。对应于此类别的头发分割网络和方向估计网络会依次施加到I上。分割网络的输出是和I相同大小的头发分 割掩码MI(带有一个alpha通道)，而方向估计网络的输出是一个等大小的方向标记图，该图会和不定向朝向图结合在一起生成最终的方向图DI。
3.三维头发样本的生成和组织：通过对原始头发模型的发束分解和再组合生成大量新发型样本，并投影生成二维掩码图和方向图，方便后续匹配；
3.1预处理
搜集300个不同发型的三维模型{H}。所有的模型已经被对齐到同一个参考人头模型，并且由大量的独立薄多边形发束{SH}组成。每个发束代表一缕生长一致的头发，生长方向被编码在参数化纹理坐标中。对于每一个模型做进一步处理以提升模型质量：对于没有连接在头皮上的发束，寻找连接头皮的且和这些发束最近的发束，将它们平滑连接起来形成更长的发束，连接到头皮上；对于过粗的发束(超过人头半径的十分之一)，均匀将这些发束沿着生长方向分成两组发束，知道发束的宽度达到要求。
3.2样本的生成
将步骤3.1得到的三维头发样本分解成不同的发束组，首先使用简化的内部表达来表示每个发束{Si}，即用穿过发束中心的均匀划分的折线
Figure PCTCN2016079613-appb-000001
和平均半径ri表示。接着对于每个头发模型，将它的发束聚类成不同的发束组，发束间的距离定义为：
Figure PCTCN2016079613-appb-000002
其中Sa、Sb是发束，na、nb是发束的折线数目，
Figure PCTCN2016079613-appb-000003
是折线，ra、rb是发束的平均半径。每个头发模型被分解成十个发束组左右，并将这些发束组组合起来生成更多的新样本。图3给出了头发样本分解和再组合生成新样本的例子。
3.3样本的组织
生成新的样本后，本发明按照样本正视图的头发掩码面积的大小，升序将所有模型组织起来。为了进一步提升匹配效率，进一步地为每个头发样本H生成两个投影图：
头发区域掩码图
Figure PCTCN2016079613-appb-000004
它是头发样本的二维投影的掩码图。为了避免杂散发束的影响，本发明使用高斯滤波来平滑处理掩码图。
方向图
Figure PCTCN2016079613-appb-000005
本发明使用颜色来表示投影的头发方向，即用方向向量的XYZ表示RGB三通道的值。这样可绘制出头发样本的二维投影的方向图。
为了处理非正面视角，本发明在偏航角和俯仰角的[-p/4,p/4]范围内均匀采样6个角 度。这样每个样本会有6·6组头发掩码图和方向图。为了后续的匹配计算效率，所有的图都降采样到100·100。
4.数据驱动的头发建模：将步骤3中的三维头发样本和步骤2分割出的头发掩码图和方向图进行匹配、变形，并生成最终模型；
4.1基于图像的三维头发样本匹配
使用步骤2.2得到头发掩码图和方向图来选择一组合适的三维样本。通过两步比较来完成快速的大量数据样本的搜索：
面积比较：首先根据脸部特征点，将输入图像对齐到样本投影图的坐标系中。然后比较输入图像的头发掩码面积|MI|和样本投影的头发掩码面积|MH|。样本的掩码面积在(0.8|MI|,1.25|MI|)范围内得以保留。
图像匹配：对于每一个通过第一步比较的样本，本发明进一步将样本的头发掩码图和方向图
Figure PCTCN2016079613-appb-000006
与输入图像的头发掩码图和方向图
Figure PCTCN2016079613-appb-000007
进行比较。如果输入图像不是正面图，本发明从预计算的6·6组样本投影图中挑选出视角最接近的图进行比较。头发掩码图的比较是基于MI边界的距离场
Figure PCTCN2016079613-appb-000008
(BALAN,A.O.,SIGAL,L.,BLACK,M.J.,DAVIS,J.E.,AND HAUSSECKER,H.W.2007.Detailed human shape and pose from images.In Computer Vision and Pattern Recognition,2007.CVPR’07.IEEE Conference on,IEEE,1–8.)进行计算：
Figure PCTCN2016079613-appb-000009
其中
Figure PCTCN2016079613-appb-000010
分别是样本的头发掩码图和输入图像的头发掩码图，
Figure PCTCN2016079613-appb-000011
表示两个掩码间的对称差，
Figure PCTCN2016079613-appb-000012
是距离场的值。方向图的距离定义为像素的方向差dd.[0,p)的和：
Figure PCTCN2016079613-appb-000013
其中，
Figure PCTCN2016079613-appb-000014
分别是样本的方向图和输入图像的方向图，
Figure PCTCN2016079613-appb-000015
是两个掩码图的重叠区域，
Figure PCTCN2016079613-appb-000016
是重叠区域的像素数目，
Figure PCTCN2016079613-appb-000017
是重叠的像素的方向差。最后保留满足
Figure PCTCN2016079613-appb-000018
和
Figure PCTCN2016079613-appb-000019
的样本作为最终候选样本{H}。
4.2头发变形
首先进行样本头发掩码图和输入图像的掩码图的边界匹配。对于每个候选样本H，首先 将它变换到I中脸的姿势，然后按照步骤3.3的方法渲染得到头发掩码和方向图(MH,DH)。这里渲染图的分辨率和输入图一样，而不是降采样的小图。然后分别在掩码MH/MI的边界分别均匀采样200/2000个点{PH}/{PI}。对于每个边界点Pi H/Pi I，将它的位置标记为
Figure PCTCN2016079613-appb-000020
向外法向标记为
Figure PCTCN2016079613-appb-000021
计算边界间的点对点对应M({PH}fi{PI})。对于候选模型的头发掩码边界的每个点
Figure PCTCN2016079613-appb-000022
它在输入头发掩码边界的最优对应点
Figure PCTCN2016079613-appb-000023
通过优化如下匹配能量方程求得：
Figure PCTCN2016079613-appb-000024
其中EP和Ee是衡量点匹配和边匹配的能量项。EP希望点对的位置和法向尽可能接近，权重ln的值为10；Ee希望映射M尽可能维持原边界的长度：
Figure PCTCN2016079613-appb-000025
Figure PCTCN2016079613-appb-000026
其中
Figure PCTCN2016079613-appb-000027
是候选样本的头发掩码图的边界点位置，
Figure PCTCN2016079613-appb-000028
是它在输入头发掩码边界的最优对应点位置，
Figure PCTCN2016079613-appb-000029
分别是
Figure PCTCN2016079613-appb-000030
和
Figure PCTCN2016079613-appb-000031
的法向，ln是权重，
Figure PCTCN2016079613-appb-000032
分别是
Figure PCTCN2016079613-appb-000033
和
Figure PCTCN2016079613-appb-000034
的相邻的采样点。上述能量方程在隐式马尔科夫模型(HMM)框架下，使用经典的Viterbi算法(FORNEY JR,G.D.1973.The viterbi algorithm.Proceedings of the IEEE 61,3,268–278)求解。
边界匹配后，进一步通过全局平滑映射函数W(MH fi MI)将边界的对应扩散到MH的所有像素。该函数使用了Thin-Plate-Spline(TPS)算法：
Figure PCTCN2016079613-appb-000035
其中
Figure PCTCN2016079613-appb-000036
是
Figure PCTCN2016079613-appb-000037
在输入图像I中的对应位置。EF(W)是对应矩阵W的二阶偏导的Frobenius范式。权重l＝1000。
最后通过下面的优化函数将候选样本模型H的每个顶点v变形到目标v'：
Figure PCTCN2016079613-appb-000038
其中VH是样本H的所有顶点。W(vi)是vi的对应位置，它的XY坐标由上述全局平滑映射函 数W得到，而保持Z坐标不变。D是基于余切函数的离散网格拉普拉斯操作符(DESBRUN,M.,MEYER,M.,SCHRODER,P.,AND BARR,A.H.1999.Implicit fairing of irregular meshes using diffusion and curvature flow.In Proceedings of ACM SIGGRAPH,317–324.)，di是原模型H上的顶点vi的拉普拉斯坐标的大小。权重ls＝1。该优化函数可以使用非精确高斯牛顿方法求解(HUANG,J.,SHI,X.,LIU,X.,ZHOU,K.,WEI,L.-Y.,TENG,S.-H.,BAO,H.,GUO,B.,AND SHUM,H.-Y.2006.Subspace gradient domain mesh deformation.ACM Trans.Graph.25,3(July),1126–1134.)。经过变形后，可以得到与输入图像匹配更好的头发样本{H'}。
4.3最终头发生成
对于步骤4.2变形得到的候选样本{H'}，在全像素图上进行最终方向图的比较(比较函数和步骤4.1一致)，并挑选方向图匹配最好的模型H*来生成最终头发模型。接着将H*在整个模型的包围盒内转化为三维方向体表达，再以H*的方向向量和头皮区域的表面法向作为约束，将方向扩散到整个体内。然后以体方向场为引导，在头皮上均匀采样种子生成10000个发束。最后，将这些发束根据生长方向估计图进行变形得到最终头发模型(HU,L.,MA,C.,LUO,L.,AND LI,H.2015.Single-view hair modeling using a hairstyle database.ACM Transactions on Graphics(TOG)34,4,125.)。图4给出了从单幅图像生成最终头发模型的例子。
实施实例
发明人在一台配备Intel Core i7-3770中央处理器，NVidia GTX970图形处理器及32GB内存的机器上实现了本发明的实施实例。发明人采用所有在具体实施方式中列出的参数值，得到了附图中所示的所有实验结果。本发明可以有效地从大量英特网图片中生成多种发型的三维模型，这些发束级别的三维模型很好地匹配了输入的图像。对于一张典型的800·800图像，整个处理流程可在1分钟之内完成：头发分割和方向估计少于3秒，三维头发样本的匹配和变形大概需要20秒，最终发束生成少于30秒。在准备训练数据方面，处理一张图片需要平均1分钟；原始三维头发样本的分解和再生成需要少于10小时；而神经网络的训练大概需要8小时。

Claims (5)

  1. 一种基于单幅图像的全自动三维头发建模方法，其特征在于，包括以下步骤：
    (1)头发训练数据的预处理：标记头发掩码和头发生长方向图，并通过无监督聚类方法得到不同发型的分类；
    (2)全自动高精度的头发分割和方向估计方法：基于步骤(1)的标记数据训练深度神经网络，利用训练得到的分层深度卷积神经网络完成头发的类型识别，头发的分割和头发生长方向的估计；
    (3)三维头发样本的生成和组织：通过对原始头发模型的发束分解和再组合生成大量新发型样本，并投影生成头发掩码图和方向图，方便后续匹配；
    (4)数据驱动的头发建模：将步骤(3)中的三维头发样本和步骤(2)分割出的头发掩码图和方向图进行匹配、变形，并生成最终模型。
  2. 根据权利要求1所述的基于单幅图像的全自动三维头发建模方法，其特征在于，所述步骤1包括如下子步骤：
    (1.1)从英特网下载大量包含头发的人像图片，标记出头发掩码和头发生长方向图；
    (1.2)利用步骤(1.1)中得到的头发标记数据，计算各种发型的分布特征，进行聚类。
  3. 根据权利要求1所述的基于单幅图像的全自动三维头发建模方法，其特征在于，所述步骤2包括如下子步骤：
    (2.1)对输入图像中的头发区域进行自动估计；
    (2.2)基于步骤(2.1)得到的头发区域，利用深度神经网络R-CNN进行发型类别的识别；
    (2.3)基于步骤(2.2)得到的头发类型选择对应的分割神经网络进行将头发从图片中分割出来，得到头发掩码图；
    (2.4)基于步骤(2.2)得到的头发类型选择对应的方向估计神经网络预测对应头发的方向图。
  4. 根据权利要求1所述的基于单幅图像的全自动三维头发建模方法，其特征在于，所述步骤3包括如下子步骤：
    (3.1)将原始三维头发模型的发束进行分解，再混合分解的发束生成新头发样本；
    (3.2)将步骤(3.1)得到的头发样本投影生成头发掩码图和方向图，方便后续匹配。
  5. 根据权利要求1所述的基于单幅图像的全自动三维头发建模方法，其特征在于，所述步骤4包括如下子步骤：
    (4.1)将步骤(2)中分割出的头发掩码图和生长方向图，与步骤(3)中的样本投影生成的头发掩码图和生长方向图进行匹配，挑选一组合适的样本；
    (4.2)将步骤(4.1)的头发样本进行变形，更好地匹配图像中的头发；
    (4.3)将步骤(4.2)得到的变形样本按照步骤(4.1)的方法进行方向图的匹配，挑选最佳样本生成最终头发模型。
PCT/CN2016/079613 2016-04-19 2016-04-19 一种基于单幅图像的全自动三维头发建模方法 WO2017181332A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2016/079613 WO2017181332A1 (zh) 2016-04-19 2016-04-19 一种基于单幅图像的全自动三维头发建模方法
US16/163,556 US10665013B2 (en) 2016-04-19 2018-10-17 Method for single-image-based fully automatic three-dimensional hair modeling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/079613 WO2017181332A1 (zh) 2016-04-19 2016-04-19 一种基于单幅图像的全自动三维头发建模方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/163,556 Continuation US10665013B2 (en) 2016-04-19 2018-10-17 Method for single-image-based fully automatic three-dimensional hair modeling

Publications (1)

Publication Number Publication Date
WO2017181332A1 true WO2017181332A1 (zh) 2017-10-26

Family

ID=60116535

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079613 WO2017181332A1 (zh) 2016-04-19 2016-04-19 一种基于单幅图像的全自动三维头发建模方法

Country Status (2)

Country Link
US (1) US10665013B2 (zh)
WO (1) WO2017181332A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109002553A (zh) * 2018-08-08 2018-12-14 北京旷视科技有限公司 头发模型的构建方法、装置、电子设备和计算机可读介质
CN109271706A (zh) * 2018-09-14 2019-01-25 厦门美图之家科技有限公司 发型生成方法及装置
CN109408653A (zh) * 2018-09-30 2019-03-01 叠境数字科技（上海）有限公司 基于多特征检索和形变的人体发型生成方法
CN110084826A (zh) * 2018-11-30 2019-08-02 叠境数字科技（上海）有限公司 基于tof相机的头发分割方法
CN110189340A (zh) * 2019-06-03 2019-08-30 北京达佳互联信息技术有限公司 图像分割方法、装置、电子设备及存储介质
CN110599585A (zh) * 2019-08-05 2019-12-20 清华大学 基于深度学习的单图像人体三维重建方法及装置
CN110598504A (zh) * 2018-06-12 2019-12-20 北京市商汤科技开发有限公司 图像识别方法及装置、电子设备和存储介质
CN111881744A (zh) * 2020-06-23 2020-11-03 安徽清新互联信息科技有限公司 一种基于空间位置信息的人脸特征点定位方法及系统
CN113781500A (zh) * 2021-09-10 2021-12-10 中国科学院自动化研究所 舱段图像实例分割的方法、装置、电子设备及存储介质

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3723050A1 (en) * 2019-04-09 2020-10-14 Koninklijke Philips N.V. Modifying an appearance of hair
CN110517352B (zh) * 2019-08-27 2022-06-03 电子科技大学 一种物体的三维重建方法、存储介质、终端及系统
CN110992247A (zh) * 2019-11-25 2020-04-10 杭州趣维科技有限公司 一种人像照片头发拉直特效的实现方法及系统
CN110992374B (zh) * 2019-11-28 2023-09-05 杭州小影创新科技股份有限公司 一种基于深度学习的头发精细化分割方法及系统
EP3885991A1 (en) * 2020-03-26 2021-09-29 Another Brain Autoencoder-based segmentation mask generation in an alpha channel
CN113763228B (zh) * 2020-06-01 2024-03-19 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113822806B (zh) * 2020-06-19 2023-10-03 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备和存储介质
CN112419487B (zh) * 2020-12-02 2023-08-22 网易（杭州）网络有限公司 毛发三维重建方法、装置、电子设备及存储介质
CN112529914B (zh) * 2020-12-18 2021-08-13 北京中科深智科技有限公司 一种实时头发分割方法和系统
US11354860B1 (en) * 2021-01-26 2022-06-07 Qualcomm Incorporated Object reconstruction using media data
CN113129347B (zh) * 2021-04-26 2023-12-12 南京大学 一种自监督单视图三维发丝模型重建方法及系统
CN113538455B (zh) * 2021-06-15 2023-12-12 聚好看科技股份有限公司 三维发型匹配方法及电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5404426A (en) * 1991-05-21 1995-04-04 Hitachi, Ltd. Method of displaying hair style and apparatus for the same
JP2001307135A (ja) * 2000-04-20 2001-11-02 Sony Corp 情報処理装置および方法、並びに記録媒体
KR20110131947A (ko) * 2010-06-01 2011-12-07 삼성전자주식회사 영상 처리 장치 및 방법
CN102419868A (zh) * 2010-09-28 2012-04-18 三星电子株式会社 基于3d头发模板进行3d头发建模的设备和方法
CN102800129A (zh) * 2012-06-20 2012-11-28 浙江大学 一种基于单幅图像的头发建模和肖像编辑方法
CN105844706A (zh) * 2016-04-19 2016-08-10 浙江大学 一种基于单幅图像的全自动三维头发建模方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102103690A (zh) 2011-03-09 2011-06-22 南京邮电大学 一种自动的头发区域分割方法
CN104463172B (zh) 2014-12-09 2017-12-22 重庆中科云丛科技有限公司 基于人脸特征点形状驱动深度模型的人脸特征提取方法
US9679192B2 (en) * 2015-04-24 2017-06-13 Adobe Systems Incorporated 3-dimensional portrait reconstruction from a single photo

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5404426A (en) * 1991-05-21 1995-04-04 Hitachi, Ltd. Method of displaying hair style and apparatus for the same
JP2001307135A (ja) * 2000-04-20 2001-11-02 Sony Corp 情報処理装置および方法、並びに記録媒体
KR20110131947A (ko) * 2010-06-01 2011-12-07 삼성전자주식회사 영상 처리 장치 및 방법
CN102419868A (zh) * 2010-09-28 2012-04-18 三星电子株式会社 基于3d头发模板进行3d头发建模的设备和方法
CN102800129A (zh) * 2012-06-20 2012-11-28 浙江大学 一种基于单幅图像的头发建模和肖像编辑方法
CN105844706A (zh) * 2016-04-19 2016-08-10 浙江大学 一种基于单幅图像的全自动三维头发建模方法

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110598504A (zh) * 2018-06-12 2019-12-20 北京市商汤科技开发有限公司 图像识别方法及装置、电子设备和存储介质
CN110598504B (zh) * 2018-06-12 2023-07-21 北京市商汤科技开发有限公司 图像识别方法及装置、电子设备和存储介质
CN109002553A (zh) * 2018-08-08 2018-12-14 北京旷视科技有限公司 头发模型的构建方法、装置、电子设备和计算机可读介质
CN109271706A (zh) * 2018-09-14 2019-01-25 厦门美图之家科技有限公司 发型生成方法及装置
CN109271706B (zh) * 2018-09-14 2022-08-26 厦门美图之家科技有限公司 发型生成方法及装置
CN109408653B (zh) * 2018-09-30 2022-01-28 叠境数字科技（上海）有限公司 基于多特征检索和形变的人体发型生成方法
CN109408653A (zh) * 2018-09-30 2019-03-01 叠境数字科技（上海）有限公司 基于多特征检索和形变的人体发型生成方法
CN110084826A (zh) * 2018-11-30 2019-08-02 叠境数字科技（上海）有限公司 基于tof相机的头发分割方法
CN110084826B (zh) * 2018-11-30 2023-09-12 叠境数字科技（上海）有限公司 基于tof相机的头发分割方法
CN110189340A (zh) * 2019-06-03 2019-08-30 北京达佳互联信息技术有限公司 图像分割方法、装置、电子设备及存储介质
CN110189340B (zh) * 2019-06-03 2022-01-21 北京达佳互联信息技术有限公司 图像分割方法、装置、电子设备及存储介质
US11288807B2 (en) 2019-06-03 2022-03-29 Beijing Dajia Internet Information Technology Co., Ltd. Method, electronic device and storage medium for segmenting image
CN110599585A (zh) * 2019-08-05 2019-12-20 清华大学 基于深度学习的单图像人体三维重建方法及装置
CN111881744A (zh) * 2020-06-23 2020-11-03 安徽清新互联信息科技有限公司 一种基于空间位置信息的人脸特征点定位方法及系统
CN113781500A (zh) * 2021-09-10 2021-12-10 中国科学院自动化研究所 舱段图像实例分割的方法、装置、电子设备及存储介质
CN113781500B (zh) * 2021-09-10 2024-04-05 中国科学院自动化研究所 舱段图像实例分割的方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
US20190051048A1 (en) 2019-02-14
US10665013B2 (en) 2020-05-26

Similar Documents

Publication Publication Date Title
WO2017181332A1 (zh) 一种基于单幅图像的全自动三维头发建模方法
CN105844706B (zh) 一种基于单幅图像的全自动三维头发建模方法
US10891511B1 (en) Human hairstyle generation method based on multi-feature retrieval and deformation
Sarode et al. Pcrnet: Point cloud registration network using pointnet encoding
Chai et al. Autohair: Fully automatic hair modeling from a single image
US9792725B2 (en) Method for image and video virtual hairstyle modeling
Li et al. Robust flow-guided neural prediction for sketch-based freeform surface modeling
Roveri et al. A network architecture for point cloud classification via automatic depth images generation
Liao et al. Automatic caricature generation by analyzing facial features
Liu et al. NeRF-In: Free-form NeRF inpainting with RGB-D priors
Chen et al. Autosweep: Recovering 3d editable objects from a single photograph
Li et al. ColorSketch: A drawing assistant for generating color sketches from photos
CN111524226A (zh) 讽刺肖像画的关键点检测与三维重建方法
Hu et al. Geometric feature enhanced line segment extraction from large-scale point clouds with hierarchical topological optimization
Abdulwahab et al. Adversarial learning for depth and viewpoint estimation from a single image
Shi et al. Geometric granularity aware pixel-to-mesh
Maninchedda et al. Semantic 3d reconstruction of heads
He et al. A computational fresco sketch generation framework
Pang et al. Developing an image-based 3D model editing method
Ji et al. An extraction method for roof point cloud of ancient building using deep learning framework
Zeng et al. 3D plants reconstruction based on point cloud
Wei et al. BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models
CN113033656B (zh) 一种基于生成对抗网络的交互式孔探数据扩展方法
Ma et al. Computing 2D Skeleton via Generalized Electric Potential
Shen et al. Two-stage Cytopathological Image Synthesis for Augmenting Cervical Abnormality Screening

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16898922

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16898922

Country of ref document: EP

Kind code of ref document: A1