WO2016050729A1 - Retouche faciale à l'aide d'une déformation affine par morceaux et codage épars - Google Patents

Retouche faciale à l'aide d'une déformation affine par morceaux et codage épars Download PDF

Info

Publication number
WO2016050729A1
WO2016050729A1 PCT/EP2015/072354 EP2015072354W WO2016050729A1 WO 2016050729 A1 WO2016050729 A1 WO 2016050729A1 EP 2015072354 W EP2015072354 W EP 2015072354W WO 2016050729 A1 WO2016050729 A1 WO 2016050729A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
module
mask
face image
occlusion
Prior art date
Application number
PCT/EP2015/072354
Other languages
English (en)
Inventor
Joaquin ZEPEDA SALVATIERRA
Patrick Perez
Xavier BURGOS
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Publication of WO2016050729A1 publication Critical patent/WO2016050729A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2136—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on sparsity criteria, e.g. with an overcomplete basis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present invention relates to the reconstruction of lost or deteriorated parts of images or videos and, in particular, to reconstruction of facial expressions of images or videos.
  • Inpainting is the process of reconstructing lost or deteriorated parts of images and videos.
  • inpainting refers to the process of reconstructing regions of a face that were hidden due to typical occlusions such as sunglasses, hair, etc. From this initial problem formulation one can derive a wide number of similar tasks (detailed below) such as facial transfer, facial hallucination or facial expression transfer, etc.
  • An aspect of the present disclosure involves applying sparse coding to efficiently recover large regions of a face.
  • Sparse coding methods have been successfully applied to a large number of image processing problems, including denoising, inpainting, compression, classification and face recognition.
  • the aim of sparse coding is to represent each signal vector using a linear combination of a few column vectors, called atoms, from a rectangular matrix called the dictionary.
  • a good dictionary will contain atoms including spatial patterns that occur commonly in natural images.
  • Many off-the-shelf dictionary matrices exist, such as the DCT dictionary, but better results can be obtained by learning the dictionary from a set of training images.
  • the dictionary matrix D required in equation (2) needs to be chosen carefully for the task at hand.
  • a good dictionary will contain atoms that represent commonly occurring spatial patterns.
  • Inpainting based on sparse coding works as follows: let A represent the indices of the available pixels of y. Letting y (respectively D ) denote the sub-vector (sub-matrix) obtained by retaining the coefficients (rows) at positions JL, an approximation of the whole image block can be obtained from Dx°(y ,D ).
  • a goal is to locate the position of a sparse set of pre-defined P 2D key- point landmark locations encoding shape S (commonly including, for example, the corners of the eyes, mouth, and nose):
  • Sparse coding has been successfully applied to create face-tailored image compression schemes.
  • An example is the work of Bryt and Elad which also employs a piecewise-affine warping of the face to normalize physiognomy and size.
  • the application targeted by Bryt and Elad is compression, not face inpainting and their method uses the standard block-by-block rasterization approach, not a whole-image rasterization approach.
  • Yuille, Hallinan and Cohen also use sparse coding for face restoration, but in the context of face super-resolution and not the face-inpainting problem. Furthermore, the method of Yuille, Hallinan and Cohen does not consider piecewise-affine face alignment as described in the present disclosure and the sparse coding stage applied subsequently uses a standard block-by-block sparse-coding approachusing dictionaries of approximately 100,000 patches of size 5x5 taken from many face images. In contrast, the principles of the present disclosure involve learning the dictionary for the reconstruction task.
  • Facial expression transfer has been studied with fully unoccluded faces with the goal of transferring expressions across individuals, or from a video stream into a 3D animated model by estimating 3D facial landmarks.
  • the principles of the present disclosure allows recovery of the original expression in large occluded regions of the face.
  • the proposed method applies sparse coding to inpainting of face images, particularly when large spatial regions of the face are missing.
  • An aspect comprises applying sparse coding to the entire face image following geometrical normalization via piecewise-affine warping. This allows exploitation of subtle spatial dependencies to inpaint in an expression-coherent manner, as it is the case that expressions are manifested in all parts of the face (for example, both the eyes and the mouth take a particular form when one smiles).
  • the proposed method has a wide range of applications. Examples include recovering full facial expression portrayed by a subject even when large regions of his/her face are hidden, useful for video-conferencing or network social communication. Other examples include security (e.g. , removal of face masks, sunglasses etc.) and video editing (removal of glasses or jewelry, removal of face-covering hair dos). Security is of particular interest to law enforcement and anti-terrorism.
  • HMDs head mounted displays
  • Oculus http://www.oculusvr.com/.
  • a method and apparatus for performing face occlusion removal are described including receiving a face image and an occlusion mask, the occlusion mask indicating missing pixels, receiving training images, performing face alignment on the received training images and the face image and the occlusion mask, receiving a mask, receiving a learned dictionary and reconstructing the face image using the mask and the learned dictionary.
  • Fig. 1 is an overview of the proposed approach for expression-aware inpainting through sparse coding.
  • Fig. 2 is the portion of Fig. 1 that deals with face alignment.
  • Fig. 3 shows two different image rasterization methods.
  • Fig. 4 shows the dictionary learning portion of Fig. 1.
  • Fig. 5 is a flowchart of an exemplary implementation of the proposed method shown in Fig. 1.
  • Fig. 6 is a flowchart of an exemplary implementation of the offline component of the face alignment step (act) 505 of Fig. 5.
  • Fig. 7 is a flowchart of an exemplary implementation of the online component of the face alignment step (act) 505 of Fig. 5.
  • Fig. 8 is a flowchart of an exemplary implementation of the face reconstruction (inpainting) portion of the proposed method.
  • Fig. 9 is a block diagram of an exemplary apparatus for face occlusion removal. It should be understood that the drawing(s) are for purposes of illustrating the concepts of the disclosure and is not necessarily the only possible configuration for illustrating the disclosure.
  • any switches shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software.
  • the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared.
  • explicit use of the term "processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage. Other hardware, conventional and/or custom, may also be included.
  • any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
  • any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function.
  • the disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
  • FIG. 1 An overview of the proposed approach can be seen in Fig. 1. It is composed of three main steps: 1) Face alignment (pre-processing) 2) Dictionary offline learning and 3) Face reconstruction through sparse coding.
  • the face alignment step is shown in a dark grey box with white letters. That is, face alignment is face landmark based warping.
  • the offline dictionary learning is shown by a grey box with black lettering.
  • the remaining boxes are the steps required for face reconstruction through sparse coding (inpainting). Each of the steps is detailed below.
  • Faces captured in uncontrolled conditions can present a heterogeneity of sizes and positions in the image due to 1) use of different cameras (each with a different field of view, pixel resolution, etc.) 2) the distance of the subject from the camera and 3) the subject's physiognomy.
  • the proposed method pre-processes images to align the observed face with a standard face, well centered and of a predefined fixed scale. This process is illustrated in Fig. 2.
  • the first step is to estimate the shape of the face S, encoded as a sparse set of predefined P 2D key-point landmark locations. This can be achieved using any state-of-the- art algorithm such as proposed by Burgos-Artizzu, Perona and Dollar.
  • the face is successfully warped onto the average shape, removing variations due to differences in pixel resolutions, camera projections and to different physiognomies.
  • the resulting set of training vectors ⁇ ⁇ is used to learn a dictionary by minimizing equation (3).
  • the dictionary learning portion of the proposed method is illustrated on the top of Fig. 4.
  • the image is first pre-processed using the face alignment method above. Then, letting fl. denote the indices of available pixels inside the shape-normalized face, and let M denote the indices of the occluded pixels (for an illustration of these masks, see the bottom of Fig. 4). If the occlusion mask is specified in the image before shape normalization, one only needs to apply the shape normalization function computed in the first step to the occluded pixel positions. The pixels indicated by are then concatenated to build the signal vector y that is decomposed via sparse coding using a dictionary D including only the of rows of D corresponding to
  • the resulting sparse code vector x is used to obtain an approximation of the pixels in M using D ⁇ x. This estimate is substituted in place of the occlusion in the shape normalized image, and the composite image is subsequently de-normalized to map it back to the original signal shape.
  • the occlusion mask M required in the above proposed method can be manually input by the user.
  • an automatic occlusion detection system that works as follows is proposed: A large training set of two parts is required. The first part are shape- normalized images without occlusions. The second part includes occluded shape- normalized images with known M. A feature vector (e.g., the well-known SIFT feature proposed by Lowe) is extracted from each pixel of every image. For each pixel, a binary classifier is learned using the occluded and non-occluded features as a training set.
  • Fig. 5 is a flowchart of an exemplary implementation of the proposed method shown in Fig. 1.
  • a face image and occlusion mask M are accepted (received, input).
  • face alignment is performed.
  • Mask A warped
  • the learned dictionary D is accepted (received, input).
  • face reconstruction using sparse coding (inpainting) is performed.
  • Fig. 6 is a flowchart of an exemplary implementation of the offline portion of the face alignment step (act) 505 of Fig. 5.
  • training images with or without occlusion are accepted (received, input).
  • cascaded regression landmark estimation is performed on the training images.
  • the average face shape is calculated (determined, computed).
  • Delaunay triangulation is performed. Delaunay triangulation for a set P of points in a plane is a triangulation DT(P) such that no point in P is inside the circumcircle of any triangle in DT(P).
  • the second component of the face alignment step (act) of 505 of Fig. 5 is an online component.
  • Fig. 7 is a flowchart of an exemplary implementation of the online component of the face alignment step (act) 505 of Fig. 5.
  • cascaded regression landmark estimation is performed on the face image with occlusion that was accepted (received, input) at 505.
  • the results of the Delaunay triangulation are accepted (received, input).
  • piece-wise affine transform estimation is performed.
  • the piece- wise affine transform estimation yields a warped face image of standard shape and size.
  • an affine transformation is a function between affine spaces which preserves points, straight lines and planes. Also, sets of parallel lines remain parallel after an affine transformation.
  • An affine transformation does not necessarily preserve angles between lines or distances between points, though it does preserve ratios of distances between points lying on a straight line.
  • Fig. 8 is a flowchart of an exemplary implementation of the face reconstruction (inpainting) portion of the proposed method.
  • vector A of available pixels is extracted from the warped face image of standard shape and size.
  • sparse coding using DA A rows of D
  • the result is sparse code vector x.
  • the missing pixels are reconstructed and DMX (matrix of reconstructed pixels) is substituted into positions M of the warped face image.
  • the inpainted (reconstructed face image) is unwarped.
  • the result of the unwarping module is a reconstructed face (a face image with inpainted occlusion).
  • Fig. 9 is a block diagram of an exemplary apparatus for face occlusion removal.
  • the apparatus in which the proposed method is performed may be any suitable processor.
  • a suitable processor will also include memory (storage), at least one communications interface, antennas if wireless communications are necessary or available, an internal communications means (such as but not limited to a bus, token ring etc.), at least one display device.
  • Such components are standard and not shown in Fig. 9 so as to not clutter Fig. 9.
  • the memory (storage) may include but is not limited to disks, CDs, any form of RAM, optical disks etc.
  • the at least one communications interface acts to accept (receive, input) the face image and occlusion mask, Mask A, the learned dictionary (if the learned dictionary processing is performed offline in a standalone processor).
  • the at least one communications interface also outputs the reconstructed (inpainted) face image.
  • That output may be to a printer (for hard copy) to a removable storage device, to a display device or by a network link to another computer system for further processing or face matching.
  • Any or all of the processors herein may be computer systems or may be partially or entirely implemented in application specific integrated circuits (ASICs), filed programmable gate arrays (FPGAs), reduced instructions set computers (RISCs) or any other form that a processor may take.
  • ASICs application specific integrated circuits
  • FPGAs filed programmable gate arrays
  • RISCs reduced instructions set computers
  • the learned dictionary portion of the proposed method may be performed in the same apparatus (processor) or in a standalone processor or a co-processor of the apparatus having the face alignment module and the face reconstruction module.
  • the face alignment module has two components.
  • the offline component accepts (receives) training images with or without occlusion.
  • the offline component of the face alignment module may be performed within the face occlusion removal apparatus or in a standalone processor or in a co-processor.
  • the offline component of the face alignment module then performs cascaded regression landmark estimation on the training images.
  • the average face shape is then calculated (determined, computed) in the offline component of the face alignment module. Delaunay triangulation is then performed in the offline component of the face alignment module.
  • the online component of the face alignment module then accepts (receives) a face image and occlusion mask.
  • the online component of the face alignment module then performs cascaded regression landmark estimation on the face image.
  • the online component of the face alignment module then performs piece-wise affine transform estimation using the results of the Delaunay triangulation to yield a warped face image of standard shape and size.
  • the warped face image of standard shape and size is provided to the face reconstruction module, which includes an extraction module, a sparse coding module, a substitution module and a unwarping module.
  • the extraction module also accepts Mask A (warped) specifying position of n ⁇ m available pixels.
  • the extraction module extracts vector A of available pixels from the warped face image of standard shape and size.
  • the results of the extraction module are provided to the sparse coding module.
  • the Mask A (warped) specifying position of n ⁇ m available pixels is also provided to the sparse coding module.
  • the sparse coding module also accepts the learned dictionary. As shown in Fig. 9 the sparse coding module accepts the learned dictionary from the learned dictionary module shown in a dashed outline to indicate that it may be performed within the inpainting (face reconstruction) apparatus and the learned dictionary is shown as input to the sparse coding module with a solid line (arrow) to indicate that the learned dictionary is provided from a standalone processor.
  • the sparse coding module uses DA (A rows of D), the learned dictionary and the vector of available pixels to generate (compute, determine, calculate) a sparse vector x. The results of the sparse coding module are provided to the substitution module.
  • the substitution module reconstructs the missing pixels (indicated by the occlusion mask) and substitutes DMX (matrix of reconstructed pixels) into positions M of the warped face image.
  • the results of the substitution module are provided to the unwarping module, which unwarps the inpainted face image.
  • the result of the unwarping module is a reconstructed face (a face image with inpainted occlusion).
  • the dictionary learning method described above can be specialized for the specific task addresed by the proposed method by considering the following learning problem in place of equation (3):
  • Equation (6) needs to be solved individually for each mask M.
  • random masks can be used that are varied for each sample in the training set.
  • the resulting dictionary is sub-optimal for any specific mask, but performs well on average for any mask.
  • each strip will provide an inpainting prediction for a subset of the missing pixels M. If the strips are not disjoint, the average of the available predicted pixel values is taken for each pixel.
  • the proposed method is applicable to a picture or video containing occluded faces for which it is desirable to reconstruct.
  • the proposed method attempts to preserve the true expression of the subject, where even if the eyes were originally occluded when the person smiles one can see changes in the expression of his/her eyes. This is in clear contrast which classical "static" reconstructions which are constant regardless of facial expression.
  • the present invention may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof.
  • Special purpose processors may include application specific integrated circuits (ASICs), reduced instruction set computers (RISCs) and/or field programmable gate arrays (FPGAs).
  • ASICs application specific integrated circuits
  • RISCs reduced instruction set computers
  • FPGAs field programmable gate arrays
  • the present invention is implemented as a combination of hardware and software.
  • the software is preferably implemented as an application program tangibly embodied on a program storage device.
  • the application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
  • the machine is implemented on a computer platform having hardware such as one or more central processing units (CPU), a random access memory (RAM), and input/output (I/O) interface(s).
  • CPU central processing units
  • RAM random access memory
  • I/O input/output
  • the computer platform also includes an operating system and microinstruction code.
  • the various processes and functions described herein may either be part of the microinstruction code or part of the application program (or a combination thereof), which is executed via the operating system.
  • various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.
  • the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces.
  • general-purpose devices which may include a processor, memory and input/output interfaces.
  • the phrase "coupled" is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software based components.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé et un appareil permettant d'effectuer une élimination d'occlusion faciale, tel qu'illustré sur les figures 5 et 6, consistant à recevoir une image faciale et un masque d'occlusion, le masque d'occlusion indiquant des pixels manquants (505), à recevoir des images d'apprentissage (605), à effectuer un alignement facial sur les images d'apprentissage reçues, l'image faciale et le masque d'occlusion (510), à recevoir un masque (515), à recevoir un dictionnaire d'apprentissage (520) et à reconstruire l'image faciale à l'aide du masque et du dictionnaire d'apprentissage (525).
PCT/EP2015/072354 2014-09-30 2015-09-29 Retouche faciale à l'aide d'une déformation affine par morceaux et codage épars WO2016050729A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP14306545 2014-09-30
EP14306545.6 2014-09-30

Publications (1)

Publication Number Publication Date
WO2016050729A1 true WO2016050729A1 (fr) 2016-04-07

Family

ID=51786907

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2015/072354 WO2016050729A1 (fr) 2014-09-30 2015-09-29 Retouche faciale à l'aide d'une déformation affine par morceaux et codage épars

Country Status (1)

Country Link
WO (1) WO2016050729A1 (fr)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105930878A (zh) * 2016-06-24 2016-09-07 山东大学 一种基于差分切片能量图和稀疏编码的微表情识别方法
CN106204667A (zh) * 2016-07-01 2016-12-07 山东大学 一种解决图像超分辨率重建中的相似度保留问题的稀疏编码方法
US20170024864A1 (en) * 2015-07-20 2017-01-26 Tata Consultancy Services Limited System and method for image inpainting
WO2018002533A1 (fr) * 2016-06-30 2018-01-04 Fittingbox Procédé d'occultation d'un objet dans une image ou une vidéo et procédé de réalité augmentée associé
FR3053509A1 (fr) * 2016-06-30 2018-01-05 Fittingbox Procede d’occultation d’un objet dans une image ou une video et procede de realite augmentee associe
KR20180109634A (ko) * 2017-03-28 2018-10-08 삼성전자주식회사 얼굴 인증 방법 및 장치
CN108664782A (zh) * 2017-03-28 2018-10-16 三星电子株式会社 面部验证方法和设备
CN109671034A (zh) * 2018-12-26 2019-04-23 维沃移动通信有限公司 一种图像处理方法及终端设备
CN109961407A (zh) * 2019-02-12 2019-07-02 北京交通大学 基于人脸相似性的人脸图像修复方法
US20190205616A1 (en) * 2017-12-29 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for detecting face occlusion
CN110263756A (zh) * 2019-06-28 2019-09-20 东北大学 一种基于联合多任务学习的人脸超分辨率重建系统
CN111127308A (zh) * 2019-12-08 2020-05-08 复旦大学 用于局部遮挡下单样本人脸识别的镜像特征重排修复方法
CN111639602A (zh) * 2020-05-29 2020-09-08 华中科技大学 一种行人遮挡及朝向检测方法
CN112001865A (zh) * 2020-09-02 2020-11-27 广东工业大学 一种人脸识别方法、装置和设备
US20200388017A1 (en) * 2019-06-04 2020-12-10 The Boeing Company System, apparatus and method for facilitating inspection of a target object
CN113496468A (zh) * 2020-03-20 2021-10-12 北京航空航天大学 深度图像的修复方法、装置和存储介质

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ALESSANDRO COLOMBO ET AL: "Detection and Restoration of Occlusions for 3D Face Recognition", PROCEEDINGS / 2006 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME 2006 : JULY 9 - 12, 2006, HILTON, TORONTO, TORONTO, ONTARIO, CANADA, IEEE SERVICE CENTER, PISCATAWAY, NJ, 1 July 2006 (2006-07-01), pages 1541 - 1544, XP031033142, ISBN: 978-1-4244-0366-0 *
JINGU HEO ET AL: "Face Pose Correction With Eyeglasses and Occlusions Removal", BIOMETRICS SYMPOSIUM, 2007, IEEE, PI, 1 September 2007 (2007-09-01), pages 1 - 6, XP031202452, ISBN: 978-1-4244-1548-9 *
WRIGHT J ET AL: "Robust Face Recognition via Sparse Representation", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 31, no. 2, 1 February 2009 (2009-02-01), pages 210 - 227, XP011226073, ISSN: 0162-8828, DOI: 10.1109/TPAMI.2008.79 *
WU C ET AL: "AUTOMATIC EYEGLASSES REMOVAL FROM FACE IMAGES", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 26, no. 3, 1 March 2004 (2004-03-01), pages 322 - 336, XP001190677, ISSN: 0162-8828, DOI: 10.1109/TPAMI.2004.1262319 *

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170024864A1 (en) * 2015-07-20 2017-01-26 Tata Consultancy Services Limited System and method for image inpainting
US9886746B2 (en) * 2015-07-20 2018-02-06 Tata Consultancy Services Limited System and method for image inpainting
CN105930878A (zh) * 2016-06-24 2016-09-07 山东大学 一种基于差分切片能量图和稀疏编码的微表情识别方法
CN105930878B (zh) * 2016-06-24 2020-01-14 山东大学 一种基于差分切片能量图和稀疏编码的微表情识别方法
WO2018002533A1 (fr) * 2016-06-30 2018-01-04 Fittingbox Procédé d'occultation d'un objet dans une image ou une vidéo et procédé de réalité augmentée associé
FR3053509A1 (fr) * 2016-06-30 2018-01-05 Fittingbox Procede d’occultation d’un objet dans une image ou une video et procede de realite augmentee associe
US9892561B2 (en) 2016-06-30 2018-02-13 Fittingbox Method of hiding an object in an image or video and associated augmented reality process
CN106204667B (zh) * 2016-07-01 2019-07-30 山东大学 一种解决图像超分辨率重建中的相似度保留问题的稀疏编码方法
CN106204667A (zh) * 2016-07-01 2016-12-07 山东大学 一种解决图像超分辨率重建中的相似度保留问题的稀疏编码方法
CN108664782A (zh) * 2017-03-28 2018-10-16 三星电子株式会社 面部验证方法和设备
CN108664782B (zh) * 2017-03-28 2023-09-12 三星电子株式会社 面部验证方法和设备
US20220044009A1 (en) * 2017-03-28 2022-02-10 Samsung Electronics Co., Ltd. Face verifying method and apparatus
KR20180109634A (ko) * 2017-03-28 2018-10-08 삼성전자주식회사 얼굴 인증 방법 및 장치
US10387714B2 (en) * 2017-03-28 2019-08-20 Samsung Electronics Co., Ltd. Face verifying method and apparatus
KR102359558B1 (ko) 2017-03-28 2022-02-09 삼성전자주식회사 얼굴 인증 방법 및 장치
US11163982B2 (en) * 2017-03-28 2021-11-02 Samsung Electronics Co., Ltd. Face verifying method and apparatus
US20190205616A1 (en) * 2017-12-29 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for detecting face occlusion
CN109671034B (zh) * 2018-12-26 2021-03-26 维沃移动通信有限公司 一种图像处理方法及终端设备
CN109671034A (zh) * 2018-12-26 2019-04-23 维沃移动通信有限公司 一种图像处理方法及终端设备
CN109961407B (zh) * 2019-02-12 2021-01-26 北京交通大学 基于人脸相似性的人脸图像修复方法
CN109961407A (zh) * 2019-02-12 2019-07-02 北京交通大学 基于人脸相似性的人脸图像修复方法
US20200388017A1 (en) * 2019-06-04 2020-12-10 The Boeing Company System, apparatus and method for facilitating inspection of a target object
US11544839B2 (en) * 2019-06-04 2023-01-03 The Boeing Company System, apparatus and method for facilitating inspection of a target object
CN110263756A (zh) * 2019-06-28 2019-09-20 东北大学 一种基于联合多任务学习的人脸超分辨率重建系统
CN111127308A (zh) * 2019-12-08 2020-05-08 复旦大学 用于局部遮挡下单样本人脸识别的镜像特征重排修复方法
CN111127308B (zh) * 2019-12-08 2023-06-30 复旦大学 用于局部遮挡下单样本人脸识别的镜像特征重排修复方法
CN113496468A (zh) * 2020-03-20 2021-10-12 北京航空航天大学 深度图像的修复方法、装置和存储介质
CN111639602A (zh) * 2020-05-29 2020-09-08 华中科技大学 一种行人遮挡及朝向检测方法
CN111639602B (zh) * 2020-05-29 2022-04-12 华中科技大学 一种行人遮挡及朝向检测方法
CN112001865A (zh) * 2020-09-02 2020-11-27 广东工业大学 一种人脸识别方法、装置和设备

Similar Documents

Publication Publication Date Title
WO2016050729A1 (fr) Retouche faciale à l'aide d'une déformation affine par morceaux et codage épars
Purohit et al. Bringing alive blurred moments
Shi et al. Just noticeable defocus blur detection and estimation
Yang et al. Face hallucination via sparse coding
Hacohen et al. Deblurring by example using dense correspondence
Punnappurath et al. Face recognition across non-uniform motion blur, illumination, and pose
Abiantun et al. SSR2: Sparse signal recovery for single-image super-resolution on faces with extreme low resolutions
CN112052831B (zh) 人脸检测的方法、装置和计算机存储介质
CN111275626A (zh) 一种基于模糊度的视频去模糊方法、装置及设备
Tang et al. Greedy regression in sparse coding space for single-image super-resolution
Tuzel et al. Global-local face upsampling network
US20230051960A1 (en) Coding scheme for video data using down-sampling/up-sampling and non-linear filter for depth map
CN116803079A (zh) 视频和相关特征的可分级译码
Pons et al. Variational stereovision and 3D scene flow estimation with statistical similarity measures
Li et al. A maximum a posteriori estimation framework for robust high dynamic range video synthesis
Rajput et al. Noise robust face hallucination via outlier regularized least square and neighbor representation
Galteri et al. Deep 3d morphable model refinement via progressive growing of conditional generative adversarial networks
Akbari et al. Joint sparse learning with nonlocal and local image priors for image error concealment
Hu et al. Graph-based joint denoising and super-resolution of generalized piecewise smooth images
Song et al. Multistage curvature-guided network for progressive single image reflection removal
Li et al. Robust blind motion deblurring using near-infrared flash image
Li et al. Robust foreground segmentation based on two effective background models
Valente et al. Perspective distortion modeling, learning and compensation
JP2019096146A (ja) 画像識別装置、画像識別方法、コンピュータプログラム、及び記憶媒体
Zhou et al. Polarization-aware low-light image enhancement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15770911

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15770911

Country of ref document: EP

Kind code of ref document: A1