WO2020037962A1 - Facial image correction method and apparatus, and storage medium - Google Patents

Facial image correction method and apparatus, and storage medium Download PDF

Info

Publication number
WO2020037962A1
WO2020037962A1 PCT/CN2019/075929 CN2019075929W WO2020037962A1 WO 2020037962 A1 WO2020037962 A1 WO 2020037962A1 CN 2019075929 W CN2019075929 W CN 2019075929W WO 2020037962 A1 WO2020037962 A1 WO 2020037962A1
Authority
WO
WIPO (PCT)
Prior art keywords
facial
correction
species
facial image
sample
Prior art date
Application number
PCT/CN2019/075929
Other languages
French (fr)
Chinese (zh)
Inventor
陈日伟
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2020037962A1 publication Critical patent/WO2020037962A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face

Definitions

  • the present disclosure relates to the field of artificial intelligence technology, and in particular, to a method, a device, and a storage medium for facial image correction.
  • a first aspect of the present disclosure relates to a method for correcting a facial image, which specifically includes the steps of establishing a multi-type sample bank to establish a multi-species sample bank, wherein the sample ID of each species stores the sample ID of the corresponding species and the face of the species Image; multi-class recognition model training step, using a machine learning algorithm to learn samples in multiple types of sample libraries for multiple said face images of a species with the same sample ID to obtain a multi-class recognition model for different species; obtaining multiple faces An image step to obtain multiple face images of the object to be identified; a multi-class recognition model to parse multiple face images; to use the multi-class recognition model to parse multiple face images of the object to be identified to obtain multiple facial features; and to correct the recognition step, A plurality of facial features on the facial image of the object to be recognized are corrected and identified by using a correction model.
  • the disclosure can realize the simultaneous acquisition of dynamic and static facial images for multiple creatures at the same time, especially for specific objects with unfixed postures, such as animals and babies. Even if a positive posture avatar cannot be collected, accurate facial recognition can be achieved. Add special effects to the face.
  • the optional embodiment further includes a special effect rendering step, which uses special effect tools to perform special effect rendering on the corrected and recognized face image.
  • the step of parsing a plurality of facial images by a multi-class recognition model in an optional embodiment includes: using a deep convolutional neural network to parse a plurality of the facial images of an object to be identified, respectively, to obtain a plurality of facial features.
  • an optional embodiment further includes a pre-processing step of pre-processing a plurality of the facial images of the acquired object to be identified.
  • a second aspect of the present disclosure relates to a facial image correction device, including: a multi-type sample library establishing module to establish a multi-species sample library, wherein the sample library of each species stores a sample ID of a corresponding species and a face image of the species ; Classification recognition model training module, for a plurality of said facial images of a species with the same sample ID, respectively, using a machine learning algorithm to learn samples from a multi-type sample library to obtain a multi-class recognition model for different species; obtaining multiple facial images
  • the module obtains multiple facial images of the object to be identified; the multiple classification recognition model parses multiple facial image modules; uses the multiple classification recognition model to parse multiple facial images of the object to be identified separately to obtain multiple facial features; corrects the identification module, uses corrections
  • the model corrects and recognizes multiple facial features on the facial image of the object to be recognized.
  • the device further includes a special effect rendering module for performing special effect rendering on the face image corrected and identified by using a special effect tool.
  • the multi-class recognition model in the device parses multiple facial image modules, and is configured to use a deep convolutional neural network to parse multiple multiple facial images of an object to be identified respectively. Multiple facial features.
  • the device further includes a pre-processing module that pre-processes a plurality of the facial images of the acquired object to be identified.
  • a third aspect of the present disclosure relates to a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements any of the foregoing when the program is executed. Steps of face image correction method.
  • a fourth aspect of the present disclosure relates to a computer-readable storage medium having stored thereon a computer program, wherein when the program is executed by a processor, the steps of any one of the facial image correction methods described above are implemented.
  • FIG. 1 is a schematic flowchart of a first embodiment of a facial image correction method according to the present disclosure
  • FIG. 2 is a block diagram of a first embodiment of a facial image correction apparatus according to the present disclosure
  • FIG. 3 is a schematic flowchart of a second embodiment of a facial image correction method according to the present disclosure
  • FIG. 4 is a block diagram of a second embodiment of a facial image correction apparatus according to the present disclosure.
  • FIG. 5 is a schematic flowchart of a third embodiment of a facial image correction method according to the present disclosure.
  • FIG. 6 is a block diagram of a third embodiment of a facial image correction apparatus according to the present disclosure.
  • FIG. 7 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a hardware structure of a human-computer interaction device according to an embodiment of the present disclosure
  • FIG. 9 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure.
  • a first aspect of the present disclosure relates to a facial image correction method, including:
  • a multi-type sample bank establishing step is performed to establish a multi-species sample bank, in which a sample ID of each species and a face image of the species are stored in the sample bank of each species.
  • the face sample database LFPW, AFLW, BioID, ICCV13, MVFW, and olivettifaces
  • the cat face sample database can randomly collect a sufficient number (such as 200) of cat breeds of various breeds as the cat face sample database
  • the dog face sample database can Randomly collect a sufficient number (such as 200) of dog faces of various breeds as a dog face sample library.
  • a multi-class recognition model training step is performed on multiple face images of a species with the same sample ID by using a machine learning algorithm to learn samples in a multi-type sample library to obtain multi-class recognition models for different species.
  • the multi-class recognition model includes forming a face recognition group for a human face, a cat face recognition group for a cat, and a dog face recognition group for a dog. It can also go further and form a European face recognition group for Europeans and a Persian cat face recognition group for Persian cats.
  • step 103 a plurality of facial image acquisition steps are performed to acquire a plurality of facial images of the object to be identified.
  • the picture format is not limited.
  • the specific method of acquisition may be real-time acquisition from a camera or input from an image library. It should be noted that, here, for example, there may be multiple facial pictures taken at regular intervals (such as every 1 s), and the object to be identified may also be multiple creatures at the same time, for example, two cats, or one person and one dog. .
  • the multi-class recognition model parses multiple facial images, and uses the multi-class recognition model to parse multiple facial images of the object to be recognized separately to obtain multiple facial features.
  • the multi-classifier can be one or more of various facial recognition detection algorithms, such as geometric feature-based algorithms, local feature algorithms, eigenface algorithms, elastic model-based algorithms, and neural network algorithms. For example, a facial sample image matching the facial image of the to-be-recognized object is searched in the face image sample library according to the feature vector of the facial image of the to-be-recognized object, and the facial features of the facial image of the to-be-recognized object are determined according to the facial sample image. . By calculating the vector distance between the feature vector of the face image of the object to be recognized and the feature vector of the face sample image, the face sample image with the smallest or smaller vector distance is used as the face sample image that matches the face image of the object to be recognized.
  • the facial features of the facial sample image are the facial features of the facial image of the object to be identified.
  • a plurality of facial images of an object to be recognized may be separately analyzed by using a deep convolutional neural network to obtain a plurality of facial features.
  • Step 105 a correction recognition step, using a correction model to perform correction recognition on a plurality of facial features on a facial image of a subject to be recognized.
  • a specific implementation manner may be, for example, calculating a rotation amount of an object to be identified according to a functional relationship between a depth value of the facial feature and the features, and fitting a front image of the object to be identified based on obtaining multiple facial images of the object to be identified. To match the multi-type sample library again to obtain recognition results.
  • the object to be identified includes multiple organisms, the identification of multiple organisms is performed simultaneously.
  • the facial image correction method of the present disclosure further includes: step 106, a special effect rendering step, using a special effect tool to perform special effect rendering on the face image that is identified and corrected.
  • a special effect rendering step using a special effect tool to perform special effect rendering on the face image that is identified and corrected.
  • Use various functional rendering tools to perform various special effects rendering on the identified facial images such as stickers (with garlands, glasses, hair coloring, beauty, dressing), deformation, stretching, liquefaction, and so on.
  • the present disclosure can achieve the simultaneous acquisition of dynamic and static facial images for multiple creatures at the same time, especially for specific objects with fixed postures, such as animals and babies. Since they do not cooperate with facial image acquisition, it is difficult to capture a still positive facial image.
  • the present disclosure realizes that even if a facial image of a positive posture cannot be collected, that is, it can be understood as a person's avatar in a narrow sense, accurate facial image recognition can be realized, and a special technical effect is added to its face.
  • a second aspect of the embodiment of the present disclosure relates to a face image correction apparatus.
  • the face image correction apparatus of the embodiment of the present invention is described below with reference to FIG. 2.
  • a multi-type sample bank building module 201 establishes a multi-species sample bank, in which a sample ID of a corresponding species and a face image of the species are stored in the sample bank of each species.
  • the classification recognition model training module 202 uses machine learning algorithms to respectively learn samples from multiple types of sample libraries for multiple facial images of species with the same sample ID to obtain multi-class recognition models for different species.
  • the multiple facial image module 203 acquires multiple facial images of an object to be identified.
  • the multi-class recognition model analyzes multiple face image modules 204, and uses the multi-class recognition model to parse multiple face images of the object to be recognized separately to obtain multi-face features.
  • the correction recognition module 205 performs correction recognition on a plurality of facial features on a facial image of a subject to be identified by using a correction model.
  • the facial image correction device further includes: a special effect rendering module 206 for performing special effect rendering on the face image corrected and identified by using a special effect tool.
  • the quality of the image directly affects the design of the recognition algorithm and the accuracy of the effect. Therefore, in addition to being able to optimize the algorithm, the preprocessing technology occupies a very important factor in the entire project.
  • the obtained image to be identified may be pre-processed first to eliminate irrelevant images in the image.
  • Information recover useful real information, enhance the detectability of the information and simplify the data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition.
  • the obtained image to be identified is pre-processed first.
  • the pre-processing process generally includes steps such as digitization, geometric transformation, normalization, smoothing, restoration, and enhancement, which are not repeated here.
  • the facial image correction device of this embodiment is described below with reference to FIG. 4. First of all, it should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and details are not described herein again.
  • the facial image correction device further includes a pre-processing module 203 ′, which pre-processes the obtained image to be identified first.
  • step 105 may specifically include:
  • a parameter acquisition step uses multiple facial features to calculate and obtain parameters required for the correction function.
  • the image depth of the above-mentioned characteristic points of the mouth, nose tip, eyes, and eyebrows in each face image is calculated separately from each facial feature in the obtained multiple facial images; and in each face image, the mouth, nose tip, eyes,
  • the number of pixels of the above-mentioned feature points of the eyebrows, the left or right rotation angle of the face is calculated based on the mutual position relationship of the feature points, and the three parameters of the image depth, the number of pixels, and the rotation angle are corrected to finally obtain a frontal facial image.
  • a function correction step completes a correction operation for a multi-angle facial image by using the parameters in the obtained correction function.
  • the three parameters of the acquired image depth, the number of pixels, and the rotation angle and the parameter ranges of the image depth, the number of pixels, and the rotation angle of the front face image learned in advance are adjusted and corrected.
  • Get the correction function includes a vector group of facial feature points of the corrected facial image, a vector group of facial feature points obtained by obtaining facial images of different species, and a relationship with the corrected facial image in the correction function.
  • the matching of the facial feature points completes the operation of correcting and identifying multiple facial features in a three-dimensional reconstruction state on the facial image of the object to be recognized by modifying the model.
  • the present disclosure adopts a neuron position-sensitive matching mode.
  • the main purpose of the neuron position-sensitive matching pattern is to obtain the deep convolution features at different neuron positions through a training database as the weights for 3D facial image recognition.
  • the weight is combined with the traditional sparse representation classifier, that is, based on the neuron position-sensitive matching mode, the sparse representation model is used to calculate multiple facial features.
  • the parameters required for the correction function are obtained to achieve a three-dimensional face comparison.
  • the facial image correction device of this embodiment is described below with reference to FIG. 6. First of all, it should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and details are not described herein again.
  • the correction recognition module 205 of the facial image correction device specifically includes:
  • the parameter obtaining unit 2051 calculates parameters required for the correction function by using multiple facial features, such as a rotation angle, an image depth, and a number of pixels.
  • the function correction unit 2052 completes a correction operation for a multi-angle face image by using the parameters in the obtained correction function.
  • the face image correction method and apparatus of the present disclosure can be implemented on a terminal device.
  • the terminal device may be implemented in various forms, and the terminal device in the present disclosure may include, but is not limited to, such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (Portable multimedia player), a navigation device, a vehicle-mounted terminal device, a vehicle-mounted display terminal, a vehicle-mounted electronic rear-view mirror, and the like, and a mobile terminal device such as a digital TV, a desktop computer, and the like.
  • a mobile phone such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (Portable multimedia player), a navigation device, a vehicle-mounted terminal device, a vehicle-mounted display terminal, a vehicle-mounted electronic rear-view mirror, and the like, and a
  • the terminal device may include a wireless communication unit 1, an A / V (audio / video) input unit 2, a user input unit 3, a sensing unit 4, an output unit 5, a memory 6, and an interface unit 7. , Controller 8 and power supply unit 9 and so on.
  • the A / V (audio / video) input unit 2 includes, but is not limited to, a camera, a front camera, a rear camera, and various audio and video input devices.
  • the various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof.
  • the implementations described herein can be implemented by using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( (FPGA), processor, controller, microcontroller, microprocessor, electronic unit designed to perform the functions described herein, and in some cases, such an implementation may be in the controller Implementation.
  • ASICs application-specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable gate arrays
  • processor controller, microcontroller, microprocessor, electronic unit designed to perform the functions described herein, and in some cases, such an implementation may be in the controller Implementation.
  • an implementation such as a process or function may be implemented with a separate software module allowing at least one function or operation
  • a facial image correction device 80 provided by an embodiment of the third aspect of the present disclosure includes a memory 801, a processor 802, and a program stored on the memory and executable on the processor. When the processor executes the program, any one of the foregoing is specific. Steps of a method for adding special effects to a subject's face.
  • the memory is used to store non-transitory computer-readable instructions.
  • the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory.
  • the volatile memory may include, for example, a random access memory (RAM) and / or a cache memory.
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • the processor may be a central processing unit (CPU) or other form of processing unit having data processing capability and / or instruction execution capability, and may control other components in the human-machine interaction device to execute Expected function.
  • the processor is configured to execute computer-readable instructions stored in the memory, so that the facial image correction apparatus executes the foregoing facial image correction method.
  • the face image correction device 80 includes a memory 801 and a processor 802.
  • the components in the face image correction device 80 are interconnected by a bus system and / or other forms of connection mechanisms (not shown).
  • the memory 801 is configured to store non-transitory computer-readable instructions.
  • the memory 801 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory.
  • the volatile memory may include, for example, a random access memory (RAM) and / or a cache memory.
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • the processor 802 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the face image correction apparatus 80 to perform a desired function.
  • the processor 802 is configured to execute computer-readable instructions stored in the memory 801, so that the facial image correction apparatus 80 executes the method for adding special effects to the face of a specific object.
  • the face image correction device is the same as the embodiment described above for the method of adding special effects to the face of a specific object, and the repeated description will be omitted here.
  • a computer-readable storage medium 900 provided by an embodiment of the fourth aspect of the present disclosure stores a computer program thereon.
  • the program is executed by a processor, any one of the above-mentioned methods for adding special effects to the face of a specific object is stored. Method steps.
  • the computer-readable storage medium may include, but is not limited to, any type of disk, including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), static random access memory (SRAM), and electrically erasable memory.
  • the computer-readable storage medium 900 has non-transitory computer-readable instructions 901 stored thereon.
  • a method for adding special effects to the face of a specific subject according to the embodiment of the present disclosure described with reference to the above is performed.
  • connection may be a fixed connection, a detachable connection, or an integral connection.
  • Connected can be directly connected or indirectly connected through an intermediate medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

Provided is a method for correcting a facial image, comprising: establishing multiple sample libraries of species, wherein a facial image of a sample ID species of a corresponding species is stored in each sample library; respectively learning samples in a multi-type sample library by using a machine learning algorithm with regard to a plurality of facial images of the species with the same sample ID to obtain a multi-classification recognition model with regard to different species; acquiring a plurality of facial images of an object to be recognized; respectively parsing the plurality of facial images of the object to be recognized by using the multi-classification recognition model to obtain a plurality of facial features; and performing corrective recognition on the plurality of facial features on the facial images of the object to be recognized by using a modification model. According to the present disclosure, fast simultaneous collection of dynamic and static facial images with regard to a plurality of organisms can be realized, and the technical effects of accurate facial recognition and adding special effects to faces of the organisms can also be realized. The present disclosure also relates to a facial image correction apparatus.

Description

面部图像校正方法、装置及存储介质Face image correction method, device and storage medium
本申请要求于2018年8月24日提交中国专利局、申请号为201810975874.7、申请名称为“面部图像校正方法、装置及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority from a Chinese patent application filed on August 24, 2018 with the Chinese Patent Office, application number 201810975874.7, and application name "Face Image Correction Method, Device, and Storage Medium", the entire contents of which are incorporated herein by reference. Applying.
技术领域Technical field
本公开涉及人工智能技术领域,具体而言,涉及一种面部图像校正方法、装置及存储介质。The present disclosure relates to the field of artificial intelligence technology, and in particular, to a method, a device, and a storage medium for facial image correction.
背景技术Background technique
本公开对于背景技术的描述属于与本公开相关的相关技术,仅仅是用于说明和便于理解本公开的发明内容,不应理解为申请人明确认为或推定申请人认为是本公开在首次提出申请的申请日的现有技术。The description of the background technology of the present disclosure belongs to the related technology related to the present disclosure, and is only used to explain and facilitate the understanding of the inventive content of the present disclosure. Prior art at the filing date.
随着科学技术的快速发展,越来越多的电子多媒体技术应用到人们的日常生活中,人们的娱乐休闲方式的也来越多,其中,拍短视频的音乐创意短视频社交软件就是其中一种,在拍摄或者编辑短视频时,可对人的面部增加特效,增加娱乐效果。但是目前短视频社交软件无法对摄像头采集不固定姿态的特定对象,如动物、婴儿的面部增加特效,因而无法满足用户更多的需求。With the rapid development of science and technology, more and more electronic multimedia technologies have been applied to people's daily lives, and people have more and more ways of entertainment and leisure. Among them, music creative short video social software that shoots short videos is one of them. This method can add special effects to the face of a person when shooting or editing a short video to increase the entertainment effect. However, the current short video social software cannot collect special objects with non-fixed poses on the camera, such as the faces of animals and babies, to increase special effects, so it cannot meet the needs of users.
发明内容Summary of the Invention
本公开的第一个方面涉及一种面部图像校正方法,具体包括:多类型样本库建立步骤,建立多物种样本库,其中,每个物种的样本库中存储相应物种的样本ID、物种的面部图像;多分类识别模型训练步骤,针对同一样本ID的物种的多个所述面部图像利用机器学习算法对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模 型;获取多个面部图像步骤,获取待识别对象的多个面部图像;多分类识别模型解析多个面部图像步骤,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多个面部特征;校正识别步骤,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。A first aspect of the present disclosure relates to a method for correcting a facial image, which specifically includes the steps of establishing a multi-type sample bank to establish a multi-species sample bank, wherein the sample ID of each species stores the sample ID of the corresponding species and the face of the species Image; multi-class recognition model training step, using a machine learning algorithm to learn samples in multiple types of sample libraries for multiple said face images of a species with the same sample ID to obtain a multi-class recognition model for different species; obtaining multiple faces An image step to obtain multiple face images of the object to be identified; a multi-class recognition model to parse multiple face images; to use the multi-class recognition model to parse multiple face images of the object to be identified to obtain multiple facial features; and to correct the recognition step, A plurality of facial features on the facial image of the object to be recognized are corrected and identified by using a correction model.
本公开可以实现快速针对多个生物同时采集动态和静态的面部图像,尤其针对不固定姿态的特定对象,如动物、婴儿,即便无法采集到正面的姿态头像,也能实现精准面部识别,为其面部增加特效。The disclosure can realize the simultaneous acquisition of dynamic and static facial images for multiple creatures at the same time, especially for specific objects with unfixed postures, such as animals and babies. Even if a positive posture avatar cannot be collected, accurate facial recognition can be achieved. Add special effects to the face.
根据本公开,可选的实施方式中还包括:特效渲染步骤,使用特效工具在修正识别的面部图像上,进行特效渲染。According to the present disclosure, the optional embodiment further includes a special effect rendering step, which uses special effect tools to perform special effect rendering on the corrected and recognized face image.
根据本公开,可选的实施方式中的多分类识别模型解析多个面部图像步骤包括:利用深度卷积神经网络对待识别对象的多个所述面部图像分别进行解析得到多个面部特征。According to the present disclosure, the step of parsing a plurality of facial images by a multi-class recognition model in an optional embodiment includes: using a deep convolutional neural network to parse a plurality of the facial images of an object to be identified, respectively, to obtain a plurality of facial features.
根据本公开,可选的实施方式中还包括预处理步骤,将获取的待识别对象的多个所述面部图像进行预处理。According to the present disclosure, an optional embodiment further includes a pre-processing step of pre-processing a plurality of the facial images of the acquired object to be identified.
本公开的第二个方面涉及一种面部图像校正装置,包括:多类型样本库建立模块,建立多物种样本库,其中,每个物种的样本库中存储相应物种的样本ID、物种的面部图像;分类识别模型训练模块,针对同一样本ID的物种的多个所述面部图像利用机器学习算法分别对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模型;获取多个面部图像模块,获取待识别对象的多个面部图像;多分类识别模型解析多个面部图像模块,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多面部特征;校正识别模块,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。A second aspect of the present disclosure relates to a facial image correction device, including: a multi-type sample library establishing module to establish a multi-species sample library, wherein the sample library of each species stores a sample ID of a corresponding species and a face image of the species ; Classification recognition model training module, for a plurality of said facial images of a species with the same sample ID, respectively, using a machine learning algorithm to learn samples from a multi-type sample library to obtain a multi-class recognition model for different species; obtaining multiple facial images The module obtains multiple facial images of the object to be identified; the multiple classification recognition model parses multiple facial image modules; uses the multiple classification recognition model to parse multiple facial images of the object to be identified separately to obtain multiple facial features; corrects the identification module, uses corrections The model corrects and recognizes multiple facial features on the facial image of the object to be recognized.
根据本公开,可选的实施方式中,该装置还包括:特效渲染模块,用于使用特效工具在修正识别的面部图像上,进行特效渲染。According to the present disclosure, in an optional implementation manner, the device further includes a special effect rendering module for performing special effect rendering on the face image corrected and identified by using a special effect tool.
根据本公开,可选的实施方式中,该装置中的所述多分类识别模型解析多个面部图像模块,用于利用深度卷积神经网络对待识别对象的多个所述面部图像分别进行解析得到多个面部特征。According to the present disclosure, in an optional implementation manner, the multi-class recognition model in the device parses multiple facial image modules, and is configured to use a deep convolutional neural network to parse multiple multiple facial images of an object to be identified respectively. Multiple facial features.
根据本公开,可选的实施方式中,该装置还包括:预处理模块,将获取的待识别对象的多个所述面部图像进行预处理。According to an optional embodiment of the present disclosure, the device further includes a pre-processing module that pre-processes a plurality of the facial images of the acquired object to be identified.
本公开的第三个方面涉及一种计算机设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,其中,所述处理器执行所述程序时实现上述任一种面部图像校正方法的步骤。A third aspect of the present disclosure relates to a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements any of the foregoing when the program is executed. Steps of face image correction method.
本公开的第四个方面涉及一种计算机可读存储介质,其上存储有计算机程序,其中,该程序被处理器执行所述程序时实现上述任一种面部图像校正方法的步骤。A fourth aspect of the present disclosure relates to a computer-readable storage medium having stored thereon a computer program, wherein when the program is executed by a processor, the steps of any one of the facial image correction methods described above are implemented.
本公开的附加方面和优点将在下面的描述部分中变得明显,或通过本公开的实践了解到。Additional aspects and advantages of the disclosure will become apparent in the following description, or be learned through practice of the disclosure.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
本公开的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解,其中:The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
图1是本公开一种面部图像校正方法的第一实施例流程示意图;1 is a schematic flowchart of a first embodiment of a facial image correction method according to the present disclosure;
图2是本公开一种面部图像校正装置的第一实施例框图;2 is a block diagram of a first embodiment of a facial image correction apparatus according to the present disclosure;
图3是本公开一种面部图像校正方法的第二实施例流程示意图;3 is a schematic flowchart of a second embodiment of a facial image correction method according to the present disclosure;
图4是本公开一种面部图像校正装置的第二实施例框图;4 is a block diagram of a second embodiment of a facial image correction apparatus according to the present disclosure;
图5是本公开一种面部图像校正方法的第三实施例流程示意图;5 is a schematic flowchart of a third embodiment of a facial image correction method according to the present disclosure;
图6是本公开一种面部图像校正装置的第三实施例框图;6 is a block diagram of a third embodiment of a facial image correction apparatus according to the present disclosure;
图7为本公开实施例的终端设备的硬件结构示意图;7 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present disclosure;
图8是本公开实施例的人机交互装置的硬件结构示意图;8 is a schematic diagram of a hardware structure of a human-computer interaction device according to an embodiment of the present disclosure;
图9是本公开实施例的计算机可读存储介质的示意图。FIG. 9 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure.
具体实施方式detailed description
为了能够更清楚地理解本公开的上述目的、特征和优点,下面结合附图和具体实施方式对本公开进行进一步的详细描述。虽然每个实施例代表了发明的单一组合,但是本公开不同实施例可以替换,或者合并组合,因此本公开也可认为包含所记载的相同和 /或不同实施例的所有可能组合。因而,如果一个实施例包含A、B、C,另一个实施例包含B和D的组合,那么本公开也应视为包括含有A、B、C、D的一个或多个所有其他可能的组合的实施例,尽管该实施例可能并未在以下内容中有明确的文字记载。In order to more clearly understand the foregoing objectives, features, and advantages of the present disclosure, the present disclosure is described in further detail below with reference to the accompanying drawings and specific embodiments. Although each embodiment represents a single combination of inventions, different embodiments of the present disclosure may be substituted or combined, so the disclosure may also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment contains A, B, C and another embodiment contains a combination of B and D, then this disclosure should also be considered to include one or more of all other possible combinations containing A, B, C, D Embodiment, although this embodiment may not be clearly written in the following content.
在下面的描述中阐述了很多具体细节以便于充分理解本公开,但是,本公开还可以采用其他不同于在此描述的其他方式来实施,因此,本公开的保护范围并不受下面公开的具体实施例的限制。In the following description, many specific details are set forth in order to fully understand the present disclosure. However, the present disclosure can also be implemented in other ways than those described herein. Therefore, the scope of protection of the present disclosure is not limited by the specifics disclosed below. Limitations of Examples.
实施例1Example 1
如图1所示,本公开的第一个方面,涉及一种面部图像校正方法,包括:As shown in FIG. 1, a first aspect of the present disclosure relates to a facial image correction method, including:
步骤101,多类型样本库建立步骤,建立多物种样本库,其中,每个物种的样本库中存储相应物种的样本ID、物种的面部图像。例如,人脸样本库LFPW、AFLW、BioID、ICCV13、MVFW、olivettifaces;猫脸样本库,可以随机采集足够数量(如200张)各品种的猫脸作为猫脸样本库;狗脸样本库,可以随机采集足够数量(如200张)各品种的狗脸作为狗脸样本库。In step 101, a multi-type sample bank establishing step is performed to establish a multi-species sample bank, in which a sample ID of each species and a face image of the species are stored in the sample bank of each species. For example, the face sample database LFPW, AFLW, BioID, ICCV13, MVFW, and olivettifaces; the cat face sample database can randomly collect a sufficient number (such as 200) of cat breeds of various breeds as the cat face sample database; the dog face sample database can Randomly collect a sufficient number (such as 200) of dog faces of various breeds as a dog face sample library.
步骤102,多分类识别模型训练步骤,针对同一样本ID的物种的多个面部图像利用机器学习算法分别对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模型。In step 102, a multi-class recognition model training step is performed on multiple face images of a species with the same sample ID by using a machine learning algorithm to learn samples in a multi-type sample library to obtain multi-class recognition models for different species.
根据示例实施例,学习过程中,对同一学习样本以组进行机器算法学习,以得到针对该物种的识别模型。例如,多分类识别模型中含针对人脸形成人脸识别组,针对猫形成猫脸识别组,针对狗形成狗脸识别组。也可以更进一步,对欧洲人形成欧洲人脸识别组,对波斯猫形成波斯猫脸识别组。According to an exemplary embodiment, during the learning process, machine algorithm learning is performed on the same learning sample in groups to obtain a recognition model for the species. For example, the multi-class recognition model includes forming a face recognition group for a human face, a cat face recognition group for a cat, and a dog face recognition group for a dog. It can also go further and form a European face recognition group for Europeans and a Persian cat face recognition group for Persian cats.
步骤103,获取多个面部图像步骤,获取待识别对象的多个面部图像。In step 103, a plurality of facial image acquisition steps are performed to acquire a plurality of facial images of the object to be identified.
获取待识别对象的多张面部图片,图片格式不限。获取的具体方式,可以是,从摄像头实时采集,也可以是从图片库输入。需要说明的是,这里,可以是例如固定间隔时间(如每隔1s)拍摄的多副面部图片,所述待识别对象也可以同时是多个生物,例如,两只猫,或者一人一狗等。Obtain multiple facial pictures of the object to be identified. The picture format is not limited. The specific method of acquisition may be real-time acquisition from a camera or input from an image library. It should be noted that, here, for example, there may be multiple facial pictures taken at regular intervals (such as every 1 s), and the object to be identified may also be multiple creatures at the same time, for example, two cats, or one person and one dog. .
步骤104,多分类识别模型解析多个面部图像步骤,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多个面部特征。In step 104, the multi-class recognition model parses multiple facial images, and uses the multi-class recognition model to parse multiple facial images of the object to be recognized separately to obtain multiple facial features.
多分类识别器可以是各种面部识别检测算法,如基于几何特征的算法、局部特征的算法、特征脸算法、基于弹性模型的算法、神经网络算法中的一种或几种。如,根据待识别对象面部图像的特征向量在所述人脸图像样本库中查找与该待识别对象面部图像相匹配的面部样本图像,根据该面部样本图像确定待识别对象的面部图像的面部特征。通过计算待识别对象的面部图像的特征向量与面部样本图像的特征向量之间的向量距离,将向量距离最小或小于阈值的面部样本图像作为与待识别对象面部图像相匹配的面部样本图像。该面部样本图像的面部特征即为待识别对象面部图像的面部特征。The multi-classifier can be one or more of various facial recognition detection algorithms, such as geometric feature-based algorithms, local feature algorithms, eigenface algorithms, elastic model-based algorithms, and neural network algorithms. For example, a facial sample image matching the facial image of the to-be-recognized object is searched in the face image sample library according to the feature vector of the facial image of the to-be-recognized object, and the facial features of the facial image of the to-be-recognized object are determined according to the facial sample image. . By calculating the vector distance between the feature vector of the face image of the object to be recognized and the feature vector of the face sample image, the face sample image with the smallest or smaller vector distance is used as the face sample image that matches the face image of the object to be recognized. The facial features of the facial sample image are the facial features of the facial image of the object to be identified.
上述方式中,可选利用深度卷积神经网络对待识别对象的多个面部图像分别进行解析得到多个面部特征。In the above manner, a plurality of facial images of an object to be recognized may be separately analyzed by using a deep convolutional neural network to obtain a plurality of facial features.
步骤105,校正识别步骤,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。Step 105: a correction recognition step, using a correction model to perform correction recognition on a plurality of facial features on a facial image of a subject to be recognized.
在上一步骤的基础上,本公开继续进行校正识别步骤的有益效果为,即使未获取与多类型样本库非常匹配的图像,也可以获得很好的识别结果。具体实现方式可以是,例如,根据面部特征的深度值和特征之间的函数关系,计算待识别对象的旋转量,根据获取待识别对象的多个面部图像拟合出该待识别对象的正面图像,从而与多类型样本库再次匹配获得识别结果。On the basis of the previous step, the beneficial effect of continuing the correction and recognition step of the present disclosure is that a good recognition result can be obtained even if an image that closely matches the multi-type sample library is not obtained. A specific implementation manner may be, for example, calculating a rotation amount of an object to be identified according to a functional relationship between a depth value of the facial feature and the features, and fitting a front image of the object to be identified based on obtaining multiple facial images of the object to be identified. To match the multi-type sample library again to obtain recognition results.
需要说明的是,如上文所述,若待识别对象包括多个生物,那么多个生物的识别是同时进行的。It should be noted that, as described above, if the object to be identified includes multiple organisms, the identification of multiple organisms is performed simultaneously.
此外,还需要说明的是,本公开的面部图像校正方法还包括:步骤106,特效渲染步骤,使用特效工具在修正识别的面部图像上进行特效渲染。使用各种功能的渲染工具对识别出的面部图像进行各种特效渲染,例如,贴纸(带花环,带眼镜,染发,美颜、换装)、变形、拉伸、液化等等。In addition, it should also be noted that the facial image correction method of the present disclosure further includes: step 106, a special effect rendering step, using a special effect tool to perform special effect rendering on the face image that is identified and corrected. Use various functional rendering tools to perform various special effects rendering on the identified facial images, such as stickers (with garlands, glasses, hair coloring, beauty, dressing), deformation, stretching, liquefaction, and so on.
本公开可以实现快速针对多个生物同时采集动态和静态的面部图像,尤其针对不固定姿态的特定对象,如动物、婴儿,由于他们不配合面部图像采集,难以捕捉到静止的正面的面部图像,而本公开实现了即便无法采集到正面姿态的面部图像,即可以狭义理解为人物头像,也能实现精准面部图像的识别,并为其面部增加特效的技术效果。The present disclosure can achieve the simultaneous acquisition of dynamic and static facial images for multiple creatures at the same time, especially for specific objects with fixed postures, such as animals and babies. Since they do not cooperate with facial image acquisition, it is difficult to capture a still positive facial image. The present disclosure realizes that even if a facial image of a positive posture cannot be collected, that is, it can be understood as a person's avatar in a narrow sense, accurate facial image recognition can be realized, and a special technical effect is added to its face.
本公开实施例的第二个方面涉及一种面部图像校正装置,下面参考附图2描述本发明实施例的面部图像校正装置。首先需要说明的是,前述对方法实施例的解释说明也适用于该实施例的装置,此处不再赘述其细节。A second aspect of the embodiment of the present disclosure relates to a face image correction apparatus. The face image correction apparatus of the embodiment of the present invention is described below with reference to FIG. 2. First of all, it should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and details are not described herein again.
多类型样本库建立模块201,建立多物种样本库,其中,每个物种的样本库中存储相应物种的样本ID、物种的面部图像。A multi-type sample bank building module 201 establishes a multi-species sample bank, in which a sample ID of a corresponding species and a face image of the species are stored in the sample bank of each species.
分类识别模型训练模块202,针对同一样本ID的物种的多个面部图像利用机器学习算法分别对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模型。The classification recognition model training module 202 uses machine learning algorithms to respectively learn samples from multiple types of sample libraries for multiple facial images of species with the same sample ID to obtain multi-class recognition models for different species.
获取多个面部图像模块203,获取待识别对象的多个面部图像。The multiple facial image module 203 acquires multiple facial images of an object to be identified.
多分类识别模型解析多个面部图像模块204,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多面部特征。The multi-class recognition model analyzes multiple face image modules 204, and uses the multi-class recognition model to parse multiple face images of the object to be recognized separately to obtain multi-face features.
校正识别模块205,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。The correction recognition module 205 performs correction recognition on a plurality of facial features on a facial image of a subject to be identified by using a correction model.
此外,还需要说明的是,本公开涉及的面部图像校正装置,还包括:特效渲染模块206,用于使用特效工具在修正识别的面部图像上,进行特效渲染。In addition, it should be noted that the facial image correction device according to the present disclosure further includes: a special effect rendering module 206 for performing special effect rendering on the face image corrected and identified by using a special effect tool.
实施例二Example two
图像识别中,图像质量的好坏直接影响识别算法的设计与效果精度,因此除了能在算法上的优化外,预处理技术在整个项目中占有很重要的因素。In image recognition, the quality of the image directly affects the design of the recognition algorithm and the accuracy of the effect. Therefore, in addition to being able to optimize the algorithm, the preprocessing technology occupies a very important factor in the entire project.
本实施例考虑到图像实时采集过程中,通常会有光线明暗、阴影、复杂背景等因素,因此,可选实施例中,可以将获得的待识别图像先进行预处理,以消除图像中无关的信息,恢复有用的真实信息,增强有关信息的可检测性和最大限度地简化数据,从而改进特征抽取、图像分割、匹配和识别的可靠性。This embodiment takes into account factors such as light and darkness, shadows, and complex backgrounds during the real-time image acquisition process. Therefore, in an optional embodiment, the obtained image to be identified may be pre-processed first to eliminate irrelevant images in the image. Information, recover useful real information, enhance the detectability of the information and simplify the data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition.
如图3所示,步骤103',将获得的待识别图像先进行预处理。预处理过程一般有数字化、几何变换、归一化、平滑、复原和增强等步骤,在此不再赘述。As shown in FIG. 3, in step 103 ', the obtained image to be identified is pre-processed first. The pre-processing process generally includes steps such as digitization, geometric transformation, normalization, smoothing, restoration, and enhancement, which are not repeated here.
下面参考附图4描述本实施例的面部图像校正装置。首先需要说明的是,前述对方法实施例的解释说明也适用于该实施例的装置,此处不再赘述其细节。The facial image correction device of this embodiment is described below with reference to FIG. 4. First of all, it should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and details are not described herein again.
如图4所示,该面部图像校正装置还包括预处理模块203’,将获得的待识别图像先进行预处理。As shown in FIG. 4, the facial image correction device further includes a pre-processing module 203 ′, which pre-processes the obtained image to be identified first.
实施例三Example three
本实施例是可选实施例,具体采用了从利用多面部特征计算得到校正函数所需的各个参数,然后通过多图像的各面部特征点之间的映射关系建立映射函数,对多角度面部图像进行校正的方式来调整非正面图像识别所带来的误差。参照图5,步骤105具体可以包括:This embodiment is an optional embodiment, specifically adopting various parameters required to obtain the correction function from the calculation of using multiple facial features, and then establishing a mapping function through the mapping relationship between the facial feature points of the multiple images, for multi-angle facial images Correction is performed to adjust the errors caused by non-frontal image recognition. Referring to FIG. 5, step 105 may specifically include:
步骤1051,参数获取步骤,利用多个面部特征计算得到校正函数所需的参数。In step 1051, a parameter acquisition step uses multiple facial features to calculate and obtain parameters required for the correction function.
例如,通过获取的多张面部图像中每个面部特征分别计算得到每副面部图像中,嘴、鼻尖、眼睛、眉毛上述特征点的图像深度;以及每副面部图像中,嘴、鼻尖、眼睛、眉毛上述特征点的像素数目,通过特征点的相互位置关系计算面部左或右的旋转角度,通过获取的图像深度、像素数目以及旋转角度这三个参数进行校正,最终获取正面的面部图像。For example, the image depth of the above-mentioned characteristic points of the mouth, nose tip, eyes, and eyebrows in each face image is calculated separately from each facial feature in the obtained multiple facial images; and in each face image, the mouth, nose tip, eyes, The number of pixels of the above-mentioned feature points of the eyebrows, the left or right rotation angle of the face is calculated based on the mutual position relationship of the feature points, and the three parameters of the image depth, the number of pixels, and the rotation angle are corrected to finally obtain a frontal facial image.
步骤1052,函数校正步骤,通过获取的校正函数中的参数完成针对多角度的面部图像的校正操作。In step 1052, a function correction step completes a correction operation for a multi-angle facial image by using the parameters in the obtained correction function.
需要说明的是,将获取的图像深度、像素数目以及旋转角度的三个参数与预先学习好的正面的面部图像的图像深度、像素数目以及旋转角度的参数范围进行调整校正操作,与此同时,获取校正函数。需要说明的是,校正函数中包括校正的面部图像的面部特征点向量组,通过获取的不同物种的面部图像的面部特征点的向量组的获取,以及与校正函数中已校正后的面部图像的面部特征点进行匹配完成通过修正模型对待识别对象的面部图像上在三维重建状态下的多个面部特征进行校正识别的操作。此外,可以理解的是,在面部校正步骤中,考虑到不同面部区域对三维面部识别具有不同重要性,本公开采用了基于神经元位置敏感的匹配模式。基于神经元位置敏感的匹配模式的主要目的是通过一个训练数据库获得不同神经元位置处的深度卷积特征,作为三维面部图像识别时的权重。在基于神经元位置敏感的匹配模式下,在识别过程中,将该权重与传统的稀疏表示分类器结合,即基于神经元位置敏感的匹配模式下的稀疏表示模型,对利用多个 面部特征计算得到校正函数所需的参数,实现三维面对的比对。It should be noted that the three parameters of the acquired image depth, the number of pixels, and the rotation angle and the parameter ranges of the image depth, the number of pixels, and the rotation angle of the front face image learned in advance are adjusted and corrected. At the same time, Get the correction function. It should be noted that the correction function includes a vector group of facial feature points of the corrected facial image, a vector group of facial feature points obtained by obtaining facial images of different species, and a relationship with the corrected facial image in the correction function. The matching of the facial feature points completes the operation of correcting and identifying multiple facial features in a three-dimensional reconstruction state on the facial image of the object to be recognized by modifying the model. In addition, it can be understood that in the face correction step, considering that different face regions have different importance for three-dimensional face recognition, the present disclosure adopts a neuron position-sensitive matching mode. The main purpose of the neuron position-sensitive matching pattern is to obtain the deep convolution features at different neuron positions through a training database as the weights for 3D facial image recognition. In the neuron position-sensitive matching mode, in the recognition process, the weight is combined with the traditional sparse representation classifier, that is, based on the neuron position-sensitive matching mode, the sparse representation model is used to calculate multiple facial features. The parameters required for the correction function are obtained to achieve a three-dimensional face comparison.
下面参考附图6描述本实施例的面部图像校正装置。首先需要说明的是,前述对方法实施例的解释说明也适用于该实施例的装置,此处不再赘述其细节。The facial image correction device of this embodiment is described below with reference to FIG. 6. First of all, it should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and details are not described herein again.
如图6所示,该面部图像校正装置的校正识别模块205具体包括:As shown in FIG. 6, the correction recognition module 205 of the facial image correction device specifically includes:
参数获取单元2051,利用多个面部特征计算得到校正函数所需的参数,例如如旋转角度,图像深度以及像素数目等。The parameter obtaining unit 2051 calculates parameters required for the correction function by using multiple facial features, such as a rotation angle, an image depth, and a number of pixels.
函数校正单元2052,通过获取的校正函数中的参数完成针对多角度的面部图像的校正操作。The function correction unit 2052 completes a correction operation for a multi-angle face image by using the parameters in the obtained correction function.
此外,如图7所示,本公开的面部图像校正方法和装置可以在终端设备上实现。终端设备可以以各种形式来实施,本公开中的终端设备可以包括但不限于诸如移动电话、智能电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、导航装置、车载终端设备、车载显示终端、车载电子后视镜等等的移动终端设备以及诸如数字TV、台式计算机等等的固定终端设备。In addition, as shown in FIG. 7, the face image correction method and apparatus of the present disclosure can be implemented on a terminal device. The terminal device may be implemented in various forms, and the terminal device in the present disclosure may include, but is not limited to, such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (Portable multimedia player), a navigation device, a vehicle-mounted terminal device, a vehicle-mounted display terminal, a vehicle-mounted electronic rear-view mirror, and the like, and a mobile terminal device such as a digital TV, a desktop computer, and the like.
在本公开的一个实施例中,终端设备可以包括无线通信单元1、A/V(音频/视频)输入单元2、用户输入单元3、感测单元4、输出单元5、存储器6、接口单元7、控制器8和电源单元9等等。其中,A/V(音频/视频)输入单元2包括但不限于,摄像头、前置摄像头,后置摄像头,各类音视频输入设备。本领域的技术人员应该理解,上述实施例列出的终端设备所包括的组件,不止上述所述的种类,可以包括更少或者更多的组件。In one embodiment of the present disclosure, the terminal device may include a wireless communication unit 1, an A / V (audio / video) input unit 2, a user input unit 3, a sensing unit 4, an output unit 5, a memory 6, and an interface unit 7. , Controller 8 and power supply unit 9 and so on. The A / V (audio / video) input unit 2 includes, but is not limited to, a camera, a front camera, a rear camera, and various audio and video input devices. Those skilled in the art should understand that the components included in the terminal device listed in the foregoing embodiments are not limited to the types described above, and may include fewer or more components.
本领域的技术人员应该理解,这里描述的各种实施方式可以以使用例如计算机软件、硬件或其任何组合的计算机可读介质来实施。对于硬件实施,这里描述的实施方式可以通过使用特定用途集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理装置(DSPD)、可编程逻辑装置(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器、被设计为执行这里描述的功能的电子单元中的至少一种来实施,在一些情况下,这样的实施方式可以在控制器中实施。对于软件实施,诸如过程或功能的实施方式可以与允许执行至少一种功能或操作的单独的软件模块来实施。软件代码可以由以 任何适当的编程语言编写的软件应用程序(或程序)来实施,软件代码可以存储在存储器中并且由控制器执行。Those skilled in the art will understand that the various embodiments described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or any combination thereof. For hardware implementation, the implementations described herein can be implemented by using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( (FPGA), processor, controller, microcontroller, microprocessor, electronic unit designed to perform the functions described herein, and in some cases, such an implementation may be in the controller Implementation. For software implementation, an implementation such as a process or function may be implemented with a separate software module allowing at least one function or operation to be performed. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in a memory and executed by a controller.
本公开第三方面的实施例提供的面部图像校正装置80,包括存储器801、处理器802及存储在存储器上并可在处理器上运行的程序,处理器执行程序时实现上述任一项为特定对象的面部增加特效的方法的步骤。A facial image correction device 80 provided by an embodiment of the third aspect of the present disclosure includes a memory 801, a processor 802, and a program stored on the memory and executable on the processor. When the processor executes the program, any one of the foregoing is specific. Steps of a method for adding special effects to a subject's face.
在本公开的一个实施例中,存储器用于存储非暂时性计算机可读指令。根据示例实施例,存储器可以包括一个或多个计算机程序产品,计算机程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在本公开的一个实施例中,处理器可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元,并且可以控制人机交互装置中的其它组件以执行期望的功能。在本公开的一个实施例中,处理器用于运行存储器中存储的计算机可读指令,使得面部图像校正装置执行上述面部图像校正方法。In one embodiment of the present disclosure, the memory is used to store non-transitory computer-readable instructions. According to example embodiments, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like. In one embodiment of the present disclosure, the processor may be a central processing unit (CPU) or other form of processing unit having data processing capability and / or instruction execution capability, and may control other components in the human-machine interaction device to execute Expected function. In an embodiment of the present disclosure, the processor is configured to execute computer-readable instructions stored in the memory, so that the facial image correction apparatus executes the foregoing facial image correction method.
在本公开的一个实施例中,如图8所示,面部图像校正装置80包括存储器801和处理器802。面部图像校正装置80中的各组件通过总线系统和/或其它形式的连接机构(未示出)互连。In one embodiment of the present disclosure, as shown in FIG. 8, the face image correction device 80 includes a memory 801 and a processor 802. The components in the face image correction device 80 are interconnected by a bus system and / or other forms of connection mechanisms (not shown).
存储器801用于存储非暂时性计算机可读指令。根据示例实施例,存储器801可以包括一个或多个计算机程序产品,计算机程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。The memory 801 is configured to store non-transitory computer-readable instructions. According to example embodiments, the memory 801 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
处理器802可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元,并且可以控制面部图像校正装置80中的其它组件以执行期望的功能。在本公开的一个实施例中,处理器802用于运行存储器801中存储的计算机可读指令,使得面部图像校正装置80执行上述为特定对象的面部增加特效的方法。面部图像校正装置与上述为特定对象的面部增加特效的方法描述的实施例相同,在此将省 略其重复描述。The processor 802 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the face image correction apparatus 80 to perform a desired function. In an embodiment of the present disclosure, the processor 802 is configured to execute computer-readable instructions stored in the memory 801, so that the facial image correction apparatus 80 executes the method for adding special effects to the face of a specific object. The face image correction device is the same as the embodiment described above for the method of adding special effects to the face of a specific object, and the repeated description will be omitted here.
本公开第四方面的实施例提供的计算机可读存储介质900,如图9所示,其上存储有计算机程序,该程序被处理器执行时实现上述任一项为特定对象的面部增加特效的方法的步骤。其中,计算机可读存储介质可以包括但不限于任何类型的盘,包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等等)、静态随机访问存储器(SRAM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、软盘、光盘、DVD、CD-ROM、微型驱动器以及磁光盘、ROM、RAM、EPROM、EEPROM、DRAM、VRAM、闪速存储器设备、磁卡或光卡、纳米系统(包括分子存储器IC),或适合于存储指令和/或数据的任何类型的媒介或设备。在本公开的一个实施例中,计算机可读存储介质900其上存储有非暂时性计算机可读指令901。当非暂时性计算机可读指令901由处理器运行时,执行参照上述描述的根据本公开实施例的为特定对象的面部增加特效的方法。As shown in FIG. 9, a computer-readable storage medium 900 provided by an embodiment of the fourth aspect of the present disclosure stores a computer program thereon. When the program is executed by a processor, any one of the above-mentioned methods for adding special effects to the face of a specific object is stored. Method steps. The computer-readable storage medium may include, but is not limited to, any type of disk, including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), static random access memory (SRAM), and electrically erasable memory. In addition to programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, floppy disks, optical disks, DVDs, CD-ROMs, micro-drives and magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, Flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data. In one embodiment of the present disclosure, the computer-readable storage medium 900 has non-transitory computer-readable instructions 901 stored thereon. When the non-transitory computer-readable instructions 901 are executed by the processor, a method for adding special effects to the face of a specific subject according to the embodiment of the present disclosure described with reference to the above is performed.
在本公开中,术语“安装”、“相连”、“连接”、“固定”等术语均应做广义理解,例如,“连接”可以是固定连接,也可以是可拆卸连接,或一体地连接;“相连”可以是直接相连,也可以通过中间媒介间接相连。对于本领域的普通技术人员而言,可以根据具体情况理解上述术语在本公开中的具体含义。In the present disclosure, the terms "installation", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, "connected" may be a fixed connection, a detachable connection, or an integral connection. ; "Connected" can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific situations.
在本说明书的描述中,术语“一个实施例”、“一些实施例”、“具体实施例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或特点包含于本公开的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不一定指的是相同的实施例或实例。而且,描述的具体特征、结构、材料或特点可以在任何的一个或多个实施例或示例中以合适的方式结合。In the description of this specification, the descriptions of the terms “one embodiment”, “some embodiments”, “specific embodiments” and the like mean that specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in the present disclosure In at least one embodiment or example. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
以上所述仅为本公开的可选实施例而已,并不用于限制本公开,对于本领域的技术人员来说,本公开可以有各种更改和变化。凡在本公开的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本公开的保护范围之内。The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modification, equivalent replacement, or improvement made within the spirit and principle of this disclosure shall be included in the protection scope of this disclosure.

Claims (12)

  1. 一种面部图像校正方法,包括:A facial image correction method includes:
    多类型样本库建立步骤,建立多物种样本库,其中,每个物种的样本库中存储相应物种的样本ID、物种的面部图像;The multi-type sample bank establishment step establishes a multi-species sample bank, in which the sample ID of each species stores the sample ID of the corresponding species and the face image of the species;
    多分类识别模型训练步骤,针对同一样本ID的物种的多个所述面部图像利用机器学习算法分别对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模型;A multi-class recognition model training step, in which multiple face images of a species with the same sample ID are used to learn samples in a multi-type sample library by using a machine learning algorithm to obtain a multi-class recognition model for different species;
    获取多个面部图像步骤,获取待识别对象的多个面部图像;Steps of obtaining multiple facial images to obtain multiple facial images of an object to be identified;
    多分类识别模型解析多个面部图像步骤,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多个面部特征;The multi-class recognition model parses multiple facial image steps, and uses the multi-class recognition model to parse multiple facial images of the object to be recognized separately to obtain multiple facial features;
    校正识别步骤,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。The correction recognition step performs correction recognition on a plurality of facial features on a facial image of a subject to be identified using a correction model.
  2. 根据权利要求1所述的面部图像校正方法,其中,还包括:The method according to claim 1, further comprising:
    特效渲染步骤,使用特效工具在校正的面部图像上,进行特效渲染。The special effect rendering step uses special effects tools to perform special effect rendering on the corrected face image.
  3. 根据权利要求1所述的面部图像校正方法,其中,所述多分类识别模型解析多个面部图像步骤包括:利用深度卷积神经网络对待识别对象的多个所述面部图像分别进行解析得到多个面部特征。The method for facial image correction according to claim 1, wherein the step of parsing a plurality of facial images by the multi-class recognition model comprises: using a deep convolutional neural network to parse a plurality of the facial images of an object to be identified to obtain a plurality of Facial features.
  4. 根据权利要求1所述的面部图像校正方法,其中,该方法还包括:The method for correcting a facial image according to claim 1, further comprising:
    预处理步骤,将获取的待识别对象的多个所述面部图像进行预处理。The pre-processing step is to pre-process multiple acquired facial images of the object to be identified.
  5. 根据权利要求1所述的面部图像校正方法,其中,该方法的校正识别步骤包括:The method for correcting a facial image according to claim 1, wherein the correction recognition step of the method comprises:
    参数获取步骤,利用多个面部特征计算得到校正函数所需的参数;A parameter acquisition step, which uses multiple facial features to calculate parameters required for the correction function;
    函数校正步骤,通过获取的所述校正函数中的所述参数完成针对多角度的所述面部图像的校正操作。The function correction step completes a correction operation for the face image at multiple angles by using the parameters in the correction function obtained.
  6. 一种面部图像校正装置,包括:A facial image correction device includes:
    多类型样本库建立模块,建立多物种样本库,其中,每个物种的样本库中存储相 应物种的样本ID、物种的面部图像;A multi-type sample library building module to establish a multi-species sample library, in which the sample ID of each species and the face image of the species are stored in the sample library of each species;
    分类识别模型训练模块,针对同一样本ID的物种的多个所述面部图像利用机器学习算法分别对多类型样本库中样本进行学习,得到针对不同物种的多分类识别模型;A classification recognition model training module, which uses a machine learning algorithm to learn samples from multiple types of sample libraries for multiple said facial images of a species with the same sample ID to obtain a multi-class recognition model for different species;
    获取多个面部图像模块,获取待识别对象的多个面部图像;Acquiring multiple facial image modules to acquire multiple facial images of an object to be identified;
    多分类识别模型解析多个面部图像模块,利用多分类识别模型对待识别对象的多个面部图像分别进行解析得到多面部特征;The multi-class recognition model parses multiple facial image modules, and uses the multi-class recognition model to parse multiple facial images of the object to be recognized separately to obtain multi-face features;
    校正识别模块,利用修正模型对待识别对象的面部图像上的多个面部特征进行校正识别。The correction recognition module performs correction recognition on a plurality of facial features on a facial image of an object to be identified using a correction model.
  7. 根据权利要求6所述的面部图像校正装置,其中,还包括:The facial image correction device according to claim 6, further comprising:
    特效渲染模块,用于使用特效工具在修正识别的面部图像上,进行特效渲染。The special effect rendering module is used to perform special effect rendering on the face image corrected and recognized by using the special effect tool.
  8. 根据权利要求6所述的面部图像校正装置,其中,该装置中的所述多分类识别模型解析多个面部图像模块,用于利用深度卷积神经网络对待识别对象的多个所述面部图像分别进行解析得到多个面部特征。The facial image correction device according to claim 6, wherein the multi-class recognition model in the device analyzes a plurality of facial image modules for respectively using a deep convolutional neural network to a plurality of the facial images of an object to be identified. Analyze to get multiple facial features.
  9. 根据权利要求6所述的面部图像校正装置,其中,该装置还包括:The facial image correction apparatus according to claim 6, further comprising:
    预处理模块,将获取的待识别对象的多个所述面部图像进行预处理。The pre-processing module pre-processes the plurality of facial images of the acquired object to be identified.
  10. 根据权利要求6所述的面部图像校正装置,其中,校正识别模块进一步包括:The facial image correction device according to claim 6, wherein the correction recognition module further comprises:
    参数获取模块,用于利用多个面部特征计算得到校正函数所需的参数;A parameter acquisition module for calculating parameters required for the correction function by using multiple facial features;
    函数校正模块,用于通过获取的所述校正函数中的所述参数完成针对多角度的所述面部图像的校正操作。A function correction module is configured to complete a correction operation for the facial image at multiple angles by using the obtained parameters in the correction function.
  11. 一种计算机设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,其中,所述处理器执行所述程序时实现所述权利要求1-5中任一项所述的方法的步骤。A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements any of the claims 1-5 when the processor executes the program. The steps of the method described.
  12. 一种计算机可读存储介质,其上存储有计算机程序,其中,该程序被处理器执行时实现如所述权利要求1-5中任一项所述的方法的步骤。A computer-readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the steps of the method according to any one of claims 1-5.
PCT/CN2019/075929 2018-08-24 2019-02-22 Facial image correction method and apparatus, and storage medium WO2020037962A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810975874.7A CN109241890B (en) 2018-08-24 2018-08-24 Face image correction method, apparatus and storage medium
CN201810975874.7 2018-08-24

Publications (1)

Publication Number Publication Date
WO2020037962A1 true WO2020037962A1 (en) 2020-02-27

Family

ID=65069471

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/075929 WO2020037962A1 (en) 2018-08-24 2019-02-22 Facial image correction method and apparatus, and storage medium

Country Status (2)

Country Link
CN (1) CN109241890B (en)
WO (1) WO2020037962A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112215742A (en) * 2020-09-15 2021-01-12 杭州缦图摄影有限公司 Automatic liquefaction implementation method based on displacement field

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109241890B (en) * 2018-08-24 2020-01-14 北京字节跳动网络技术有限公司 Face image correction method, apparatus and storage medium
CN110110811A (en) * 2019-05-17 2019-08-09 北京字节跳动网络技术有限公司 Method and apparatus for training pattern, the method and apparatus for predictive information
CN111160359A (en) * 2019-12-23 2020-05-15 潍坊科技学院 Digital image processing method
CN115471416A (en) * 2022-08-29 2022-12-13 湖北星纪时代科技有限公司 Object recognition method, storage medium, and apparatus

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050147280A1 (en) * 2000-12-01 2005-07-07 Microsoft Corporation System and method for face recognition using synthesized images
CN104036247A (en) * 2014-06-11 2014-09-10 杭州巨峰科技有限公司 Facial feature based face racial classification method
CN107609459A (en) * 2016-12-15 2018-01-19 平安科技(深圳)有限公司 A kind of face identification method and device based on deep learning
CN108182714A (en) * 2018-01-02 2018-06-19 腾讯科技(深圳)有限公司 Image processing method and device, storage medium
CN109241890A (en) * 2018-08-24 2019-01-18 北京字节跳动网络技术有限公司 Face image correction method, apparatus and storage medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4946730B2 (en) * 2007-08-27 2012-06-06 ソニー株式会社 Face image processing apparatus, face image processing method, and computer program
CN103514442B (en) * 2013-09-26 2017-02-08 华南理工大学 Video sequence face identification method based on AAM model
CN103605965A (en) * 2013-11-25 2014-02-26 苏州大学 Multi-pose face recognition method and device
CN104978550B (en) * 2014-04-08 2018-09-18 上海骏聿数码科技有限公司 Face identification method based on extensive face database and system
CN105404854A (en) * 2015-10-29 2016-03-16 深圳怡化电脑股份有限公司 Methods and devices for obtaining frontal human face images
CN105847734A (en) * 2016-03-30 2016-08-10 宁波三博电子科技有限公司 Face recognition-based video communication method and system
CN107844744A (en) * 2017-10-09 2018-03-27 平安科技(深圳)有限公司 With reference to the face identification method, device and storage medium of depth information

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050147280A1 (en) * 2000-12-01 2005-07-07 Microsoft Corporation System and method for face recognition using synthesized images
CN104036247A (en) * 2014-06-11 2014-09-10 杭州巨峰科技有限公司 Facial feature based face racial classification method
CN107609459A (en) * 2016-12-15 2018-01-19 平安科技(深圳)有限公司 A kind of face identification method and device based on deep learning
CN108182714A (en) * 2018-01-02 2018-06-19 腾讯科技(深圳)有限公司 Image processing method and device, storage medium
CN109241890A (en) * 2018-08-24 2019-01-18 北京字节跳动网络技术有限公司 Face image correction method, apparatus and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112215742A (en) * 2020-09-15 2021-01-12 杭州缦图摄影有限公司 Automatic liquefaction implementation method based on displacement field

Also Published As

Publication number Publication date
CN109241890B (en) 2020-01-14
CN109241890A (en) 2019-01-18

Similar Documents

Publication Publication Date Title
WO2020037962A1 (en) Facial image correction method and apparatus, and storage medium
US11151363B2 (en) Expression recognition method, apparatus, electronic device, and storage medium
US11481869B2 (en) Cross-domain image translation
US10832039B2 (en) Facial expression detection method, device and system, facial expression driving method, device and system, and storage medium
US10354362B2 (en) Methods and software for detecting objects in images using a multiscale fast region-based convolutional neural network
US9978003B2 (en) Utilizing deep learning for automatic digital image segmentation and stylization
Radenovic et al. Deep shape matching
US9275273B2 (en) Method and system for localizing parts of an object in an image for computer vision applications
US20210174072A1 (en) Microexpression-based image recognition method and apparatus, and related device
WO2017193906A1 (en) Image processing method and processing system
WO2016110030A1 (en) Retrieval system and method for face image
CN108961149B (en) Image processing method, device and system and storage medium
WO2020037963A1 (en) Facial image identifying method, device and storage medium
US9443325B2 (en) Image processing apparatus, image processing method, and computer program
WO2022052530A1 (en) Method and apparatus for training face correction model, electronic device, and storage medium
US11574392B2 (en) Automatically merging people and objects from multiple digital images to generate a composite digital image
WO2019042080A1 (en) Image data processing system and method
CN107944398A (en) Based on depth characteristic association list diagram image set face identification method, device and medium
WO2021127916A1 (en) Facial emotion recognition method, smart device and computer-readabel storage medium
EP4435660A1 (en) Target detection method and apparatus
CN108268863A (en) A kind of image processing method, device and computer storage media
CN109087240B (en) Image processing method, image processing apparatus, and storage medium
Zhang et al. Landmark‐Guided Local Deep Neural Networks for Age and Gender Classification
Zhang et al. Hierarchical facial landmark localization via cascaded random binary patterns
US20220027666A1 (en) Automatic image annotation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19851030

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 27.05.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19851030

Country of ref document: EP

Kind code of ref document: A1