WO2016101883A1 - Method for face beautification in real-time video and electronic equipment - Google Patents

Method for face beautification in real-time video and electronic equipment Download PDF

Info

Publication number
WO2016101883A1
WO2016101883A1 PCT/CN2015/098355 CN2015098355W WO2016101883A1 WO 2016101883 A1 WO2016101883 A1 WO 2016101883A1 CN 2015098355 W CN2015098355 W CN 2015098355W WO 2016101883 A1 WO2016101883 A1 WO 2016101883A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image area
current picture
beautified
module
Prior art date
Application number
PCT/CN2015/098355
Other languages
French (fr)
Chinese (zh)
Inventor
武俊敏
冯加伟
薛涵凛
Original Assignee
掌赢信息科技(上海)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to CN201410840316.1A priority Critical patent/CN104574306A/en
Priority to CN201410840316.1 priority
Application filed by 掌赢信息科技(上海)有限公司 filed Critical 掌赢信息科技(上海)有限公司
Publication of WO2016101883A1 publication Critical patent/WO2016101883A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration by the use of local operators

Abstract

Disclosed in the present invention are a method for face beautification in a real-time video and electronic equipment, which belong to the field of image processing. The method comprises: acquiring a face image area from a current picture of a real-time video; beautifying the face image area to acquire a beautified face image area; integrating the beautified face image area with the face image area to generate an integrated face image area, and displaying the current picture at least including the integrated face image area. By beautifying a face in a real-time video and integrating the beautified face area with the previous face area, the picture quality of the real-time video is improved and user experience is enhanced as well.

Description

一种即时视频中的人脸美化方法和电子设备Facial beautification method and electronic device in instant video 技术领域Technical field
本发明涉及图像处理领域,特别涉及一种即时视频中的人脸美化方法和电子设备。The present invention relates to the field of image processing, and in particular, to a face beautification method and an electronic device in an instant video.
背景技术Background technique
随着即时视频应用在移动终端上的普及,使得越来越多的用户通过即时视频应用来实现与他人之间的交互,然而,由于移动终端设备的局限性以及交互场景摄像环境的影响,导致即时视频的质量较差,尤其是即时视频中的人脸画面质量较差,所以,需要提供一种即时视频中的人脸美化方法。With the popularity of instant video applications on mobile terminals, more and more users realize interaction with others through instant video applications. However, due to the limitations of mobile terminal devices and the influence of interactive scene camera environments, The quality of instant video is poor, especially the quality of face images in instant video is poor. Therefore, it is necessary to provide a method of face beautification in instant video.
现有技术提供一种即时视频中的人脸美化方法,该方法具体包括:获取即时视频中的当前帧图像,对当前帧图像中的人脸进行美化,并利用美化后的当前帧图像替换原有的当前帧图像,对当前帧的下一帧图像继续执行上述步骤,从而实现即时视频中的人脸美化。The prior art provides a method for facial beautification in an instant video. The method specifically includes: acquiring a current frame image in an instant video, beautifying a face in the current frame image, and replacing the original frame image with the beautification Some current frame images continue to perform the above steps on the next frame image of the current frame, thereby realizing face beautification in the instant video.
由于现有技术提供的方法算法复杂,对设备的硬件要求较高,而如智能手机和平板电脑等移动终端的硬件无法满足该方法的硬件要求,使得移动终端无法使用现有技术提供的方法进行即时视频中的人脸美化,或者,在移动终端使用现有技术提供的方法时,大量占用设备的处理资源和存储资源,影响设备的使用,从而降低了用户体验效果。Because the method provided by the prior art is complicated, the hardware requirements of the device are high, and the hardware of the mobile terminal such as the smart phone and the tablet cannot meet the hardware requirements of the method, so that the mobile terminal cannot use the method provided by the prior art. The face of the instant video is beautified, or when the mobile terminal uses the method provided by the prior art, the processing resources and storage resources of the device are occupied in a large amount, which affects the use of the device, thereby reducing the user experience.
发明内容Summary of the invention
为了在移动终端上实现即时视频中的人脸美化,增加用户的体验效果,本发明实施例提供了一种即时视频中的人脸美化方法和设备。所述技术方案如下:In order to implement the face beautification in the instant video on the mobile terminal and increase the experience of the user, the embodiment of the invention provides a method and device for facializing in the instant video. The technical solution is as follows:
第一方面,提供了一种即时视频中的人脸美化方法,所述方法包括: In a first aspect, a method for beautifying a face in an instant video is provided, the method comprising:
从即时视频的当前画面中获取人脸图像区域;Obtaining a face image area from a current picture of the instant video;
对所述人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the face image area to obtain a beautified face image area;
将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括所述融合后的人脸图像区域的当前画面。The beautified face image area is merged with the face image area to generate a merged face image area, and a current picture including at least the merged face image area is displayed.
结合第一方面,在第一种可能的实现方式中,所述从即时视频的当前画面中获取人脸图像区域包括:With reference to the first aspect, in a first possible implementation, the acquiring a face image area from a current picture of the instant video includes:
获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域。The face image region in the current picture is acquired from the grayscale image according to the face frame size, the face pixel distance, and the search zoom factor.
结合第一方面的第一种可能的实现方式,在第二种可能的实现方式中,所述获取所述当前画面中的人脸图像区域的灰度图像之前,所述方法还包括:In conjunction with the first possible implementation of the first aspect, in a second possible implementation, before the acquiring the grayscale image of the face image region in the current picture, the method further includes:
对所述当前画面进行压缩,在完成压缩后,执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。Compressing the current picture, and after completing the compression, performing the step of acquiring a grayscale image of the face image area in the current picture.
结合第一方面的第一种或第一方面的第二种可能的实现方式,在第三种可能的实现方式中,所述获取所述当前画面中的人脸图像区域的灰度图像之后,所述方法还包括:With reference to the first aspect of the first aspect or the second possible implementation manner of the first aspect, in a third possible implementation manner, after acquiring the grayscale image of the face image region in the current picture, The method further includes:
对所述人脸图像区域的灰度图像进行直方图均衡化处理。A histogram equalization process is performed on the grayscale image of the face image region.
结合第一方面的第一种至第一方面的第三种任一可能的实现方式,在第四种可能的实现方式中,所述获取当前画面中的人脸图像区域之后,所述方法还包括:With reference to the first aspect of the first aspect to any of the third possible implementation manners of the first aspect, in the fourth possible implementation manner, after the acquiring the facial image region in the current picture, the method further include:
对所述人脸图像区域进行缩放,获取所述人脸图像区域在未压缩的当前画面中的坐标。The face image area is scaled, and the coordinates of the face image area in the uncompressed current picture are acquired.
结合第一方面至第一方面的第四种任一可能的实现方式,在第五种可能的实现方式中,对所述人脸图像区域进行美化包括:In combination with the first aspect to any of the possible implementations of the fourth aspect, in a fifth possible implementation, the beautifying the face image area includes:
对所述灰度图像中的人脸细节进行检测; Detecting face details in the grayscale image;
将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除至少包括眼睛、嘴和鼻子之外的人脸部分;Separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the face mask image including at least an eye, a mouth And the face part outside the nose;
对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The face mask image is beautified to obtain a beautified face mask image.
结合第一方面的第五种可能的实现方式,在第六种可能的实现方式中,所述对所述人脸面具图像进行美化包括:In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation, the beautifying the facial mask image includes:
对所述人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。And integrating the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
结合第一方面的第六种可能的实现方式,在第七种可能的实现方式中,所述获取美化后的人脸图像区域包括:In conjunction with the sixth possible implementation of the first aspect, in the seventh possible implementation, the obtaining the beautified face image area includes:
根据所述人脸图像区域在未压缩的当前画面中的坐标,将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。And merging the face detail image with the beautified face mask image according to the coordinates of the face image region in the uncompressed current picture to obtain the beautified face image region.
第二方面,提供了一种电子设备,所述电子设备包括:In a second aspect, an electronic device is provided, the electronic device comprising:
获取模块,用于从即时视频的当前画面中获取人脸图像区域;An obtaining module, configured to obtain a face image area from a current picture of the instant video;
美化模块,用于对所述人脸图像区域进行美化,获取美化后的人脸图像区域;a beautification module, configured to beautify the face image area, and obtain a beautified face image area;
融合模块,将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域;a fusion module, the beautified face image area is merged with the face image area to generate a merged face image area;
显示模块,用于显示至少包括所述融合后的人脸图像区域的当前画面;a display module, configured to display a current picture including at least the merged face image area;
存储模块,所述存储模块用于存储所述人脸图像区域、美化后的人脸图像区域以及所述融合后的人脸图像区域。 And a storage module, configured to store the face image area, the beautified face image area, and the merged face image area.
结合第二方面,在第一种可能的实现方式中,所述获取模块具体用于:With reference to the second aspect, in a first possible implementation manner, the acquiring module is specifically configured to:
获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域;Obtaining a face image region in the current picture from the grayscale image according to a face frame size, a face pixel distance, and a search scaling factor;
所述存储模块还用于存储所述灰度图像。The storage module is further configured to store the grayscale image.
结合第二方面的第一种可能的实现方式,在第二种可能的实现方式中,所述设备还包括:With reference to the first possible implementation of the second aspect, in a second possible implementation, the device further includes:
压缩模块,用于对所述当前画面进行压缩,在完成压缩后,触发所述获取模块执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。And a compression module, configured to compress the current picture, and after the compression is completed, trigger the acquiring module to perform the step of acquiring the gray image of the face image area in the current picture.
结合第二方面的第一种或第二方面的第二种可能的实现方式,在第三种可能的实现方式中,所述设备还包括:With reference to the second possible implementation of the first aspect or the second aspect, in a third possible implementation, the device further includes:
处理模块,用于对所述人脸图像区域的灰度图像进行直方图均衡化处理。And a processing module, configured to perform a histogram equalization process on the grayscale image of the face image region.
结合第二方面的第一种至第二方面的第三种任一可能的实现方式,在第四种可能的实现方式中,所述压缩模块还用于对所述人脸图像区域进行缩放,所述获取模块还用于获取所述人脸图像区域在未压缩的当前画面中的坐标;With reference to any one of the first aspect of the second aspect to the third possible implementation of the second aspect, in a fourth possible implementation, the compression module is further configured to: The acquiring module is further configured to acquire coordinates of the face image area in an uncompressed current picture;
所述存储模块还用于存储所述坐标。The storage module is further configured to store the coordinates.
结合第二方面至第二方面的第四种任一可能的实现方式,在第五种可能的实现方式中,With reference to any one of the second aspect to the fourth possible implementation of the second aspect, in a fifth possible implementation manner,
所述设备还包括分离模块,用于将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除眼睛、嘴和鼻子之外的人脸部分;The apparatus further includes a separation module for separating a face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the person The face mask image includes a face portion other than the eyes, mouth, and nose;
所述美化模块用于对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The beautification module is configured to beautify the face mask image to obtain a beautified face mask image.
结合第二方面的第五种可能的实现方式,在第六种可能的实现方式中,In conjunction with the fifth possible implementation of the second aspect, in a sixth possible implementation,
所述美化模块具体用于:The beautification module is specifically used to:
对所述人脸面具图像进行高斯滤波,获取反差图像; Performing Gaussian filtering on the face mask image to obtain a contrast image;
对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
所述融合模块用于融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。The fusion module is configured to fuse the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
结合第二方面的第六种可能的实现方式,在第七种可能的实现方式中,所述融合模块用于:With reference to the sixth possible implementation of the second aspect, in a seventh possible implementation, the merging module is used to:
将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
第三方面,提供了一种电子设备,包括视频输入模块、视频输出模块、发送模块、接收模块、存储器以及分别与所述视频输入模块、所述视频输出模块、所述发射模块、所述接收模块和所述存储器连接的处理器,其中,所述存储器中存储一组程序代码,所述处理器用于调用所述存储器中存储的程序代码,执行以下操作:In a third aspect, an electronic device is provided, including a video input module, a video output module, a sending module, a receiving module, a memory, and the video input module, the video output module, the transmitting module, and the receiving, respectively And a processor coupled to the memory, wherein the memory stores a set of program code, the processor is configured to invoke program code stored in the memory, and perform the following operations:
从即时视频的当前画面中获取人脸图像区域;Obtaining a face image area from a current picture of the instant video;
对所述人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the face image area to obtain a beautified face image area;
将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括所述融合后的人脸图像区域的当前画面。The beautified face image area is merged with the face image area to generate a merged face image area, and a current picture including at least the merged face image area is displayed.
结合第三方面,在第一种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:In conjunction with the third aspect, in a first possible implementation, the processor is further configured to invoke program code stored in the memory, and perform the following operations:
获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域。The face image region in the current picture is acquired from the grayscale image according to the face frame size, the face pixel distance, and the search zoom factor.
结合第三方面的第一种可能的实现方式,在第二种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作: In conjunction with the first possible implementation of the third aspect, in a second possible implementation, the processor is further configured to invoke program code stored in the memory, and perform the following operations:
对所述当前画面进行压缩,在完成压缩后,执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。Compressing the current picture, and after completing the compression, performing the step of acquiring a grayscale image of the face image area in the current picture.
结合第三方面的第一种或第三方面的第二种可能的实现方式,在第三种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:In conjunction with the second possible implementation of the first aspect or the third aspect, in a third possible implementation, the processor is further configured to invoke the program code stored in the memory, and execute the following operating:
对所述人脸图像区域的灰度图像进行直方图均衡化处理。A histogram equalization process is performed on the grayscale image of the face image region.
结合第三方面的第一种至第三方面的第三种任一可能的实现方式,在第四种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:With reference to any of the third to third possible implementations of the third aspect, in a fourth possible implementation, the processor is further configured to invoke program code stored in the memory, Do the following:
对所述人脸图像区域进行缩放,获取所述人脸图像区域在未压缩的当前画面中的坐标。The face image area is scaled, and the coordinates of the face image area in the uncompressed current picture are acquired.
结合第三方面至第三方面的第四种任一可能的实现方式,在第五种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:With reference to any of the third aspect to the fourth possible implementation of the third aspect, in a fifth possible implementation, the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
对所述灰度图像中的人脸细节进行检测;Detecting face details in the grayscale image;
将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除至少包括眼睛、嘴和鼻子之外的人脸部分;Separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the face mask image including at least an eye, a mouth And the face part outside the nose;
对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The face mask image is beautified to obtain a beautified face mask image.
结合第三方面的第五种可能的实现方式,在第六种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:In conjunction with the fifth possible implementation of the third aspect, in a sixth possible implementation, the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
对所述人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像; Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。And integrating the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
结合第三方面的第六种可能的实现方式,在第七种可能的实现方式中,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:In conjunction with the sixth possible implementation of the third aspect, in a seventh possible implementation, the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
本发明实施例提供了一种即时视频中的人脸美化方法和电子设备,该方法包括从即时视频的当前画面中获取人脸图像区域;对所述人脸图像区域进行美化,获取美化后的人脸图像区域;将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括所述融合后的人脸图像区域的当前画面。通过在移动终端上实现即时视频中的人脸美化,增加了用户的体验效果。Embodiments of the present invention provide a method and an electronic device for facial glazing in an instant video, the method comprising: obtaining a face image region from a current picture of an instant video; beautifying the face image region to obtain a beautified a face image area; merging the beautified face image area with the face image area to generate a merged face image area, and displaying a current picture including at least the merged face image area . By realizing the beautification of the face in the instant video on the mobile terminal, the user's experience is increased.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.
图1是本发明实施例提供的一种交互系统示意图;FIG. 1 is a schematic diagram of an interaction system according to an embodiment of the present invention;
图2是本发明实施例提供的一种交互系统示意图;2 is a schematic diagram of an interaction system according to an embodiment of the present invention;
图3是本发明实施例提供的一种即时视频中的人脸美化方法流程图;3 is a flowchart of a face beautification method in an instant video according to an embodiment of the present invention;
图4是本发明实施例提供的一种即时视频中的人脸美化方法流程图;4 is a flowchart of a method for facializing a face in an instant video according to an embodiment of the present invention;
图5是本发明实施例提供的一种即时视频中的人脸美化方法流程图;FIG. 5 is a flowchart of a method for facializing a face in an instant video according to an embodiment of the present invention; FIG.
图6是本发明实施例提供的一种即时视频中的人脸美化方法流程图;6 is a flowchart of a face beautification method in an instant video according to an embodiment of the present invention;
图7是本发明实施例提供的一种电子设备结构示意图;FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
图8是本发明实施例提供的一种电子设备结构示意图。 FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
具体实施方式detailed description
为使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明实施方式作进一步地详细描述。The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
本发明实施例提供了一种即时视频中的人脸美化方法,该方法应用于一种包括至少两个移动终端和服务器的交互系统中,该交互系统可以参照图1所示,其中,移动终端1为即时视频发送方,移动终端2为即时视频接收方,移动终端1所发送的即时视频经由服务器转发至移动终端2;其中,该移动终端可以是智能手机(Smart Phone),可以是平板电脑(Tablet Personal Computer),还可以是其他移动终端,本发明实施例对具体的移动终端不加以限定。移动终端至少包括视频输入模块和视频显示模块,视频输入模块可以包括摄像头,视频显示模块可以包括显示屏,且移动终端中至少可以运行一即时视频程序,该即时视频程序控制移动终端的视频输入模块和视频显示模块进行即时视频。An embodiment of the present invention provides a method for facial landscaping in an instant video. The method is applied to an interactive system including at least two mobile terminals and a server. The interactive system may be as shown in FIG. 1 , wherein the mobile terminal 1 is an instant video sender, the mobile terminal 2 is an instant video receiver, and the instant video sent by the mobile terminal 1 is forwarded to the mobile terminal 2 via the server; wherein the mobile terminal can be a smart phone or a tablet (Tablet Personal Computer), which may also be other mobile terminals, and the specific mobile terminal is not limited in the embodiment of the present invention. The mobile terminal includes at least a video input module and a video display module. The video input module can include a camera. The video display module can include a display screen, and at least one instant video program can be run in the mobile terminal, and the instant video program controls the video input module of the mobile terminal. Instant video with the video display module.
特别的,本发明实施例所提供的方法的执行主体,即电子设备,可以是移动终端1、移动终端2和服务器中的任意一个,若该方法的执行主体是移动终端1,则对该即时视频中的人脸进行美化,然后将美化后的即时视频发送至服务器,再由服务器发送至通讯设备2,由通讯设备2中的视频显示模块显示该美化后的即时视频;若该方法的执行主体是服务器,则移动终端1在通过自身的视频输入模块输入即时视频后,将该即时视频发送至服务器,由服务器对即时视频中的人脸进行美化,再将该美化后的人脸发送至移动终端2,由通讯设备2中的视频显示模块显示该美化后的即时视频;若该方法的执行主体是移动终端2,移动终端1在通过自身的视频输入模块输入即时视频后,将该即时视频发送至服务器,服务器将该即时视频发送至移动终端2,移动终端2对即时视频中的人脸进行美化,再通过自身的视频显示模块显示该美化后的即时视频。本发明实施例对该交互系统中的该方法的具体的执行主体不加以限定。In particular, the execution body of the method provided by the embodiment of the present invention, that is, the electronic device, may be any one of the mobile terminal 1, the mobile terminal 2, and the server. If the execution subject of the method is the mobile terminal 1, the instant is The face in the video is beautified, and then the beautified instant video is sent to the server, and then sent by the server to the communication device 2, and the beautified instant video is displayed by the video display module in the communication device 2; if the method is executed After the main body is a server, the mobile terminal 1 sends the instant video to the server after inputting the instant video through the video input module of the user, and the server visualizes the face in the instant video, and then sends the beautified face to the server. The mobile terminal 2 displays the beautified instant video by the video display module in the communication device 2; if the execution subject of the method is the mobile terminal 2, the mobile terminal 1 inputs the instant video through its own video input module. The video is sent to the server, and the server sends the instant video to the mobile terminal 2, and the mobile terminal 2 is in the live video. Face beautification, and through its own video display module to display real-time video after the landscaping. The specific implementation body of the method in the interaction system is not limited in the embodiment of the present invention.
除此之外,本发明实施例所提供的方法还可以应用于一种只包括移动终端1 和移动终端2的交互系统中,该交互系统可以参照图2所示,其中,移动终端1为即时视频发送方,移动终端2为即时视频接收方,且移动终端至少包括视频输入模块和视频显示模块,视频输入模块可以包括摄像头,视频显示模块可以包括显示屏,且移动终端中至少可以运行一即时视频程序,该即时视频程序控制移动终端的视频输入模块和视频显示模块进行即时视频。In addition, the method provided by the embodiment of the present invention can also be applied to a mobile terminal 1 only. In the interaction system with the mobile terminal 2, the interactive system can be referred to FIG. 2, wherein the mobile terminal 1 is an instant video sender, the mobile terminal 2 is an instant video receiver, and the mobile terminal includes at least a video input module and a video display. The module, the video input module may include a camera, and the video display module may include a display screen, and at least one instant video program may be run in the mobile terminal, and the instant video program controls the video input module and the video display module of the mobile terminal to perform real-time video.
特别的,本发明实施例所提供的方法的执行主体,即电子设备,可以是移动终端1和移动终端2中的任意一个,若该方法的执行主体是移动终端1,则该移动终端1在通过自身的视频输入模块输入即时视频之后,对该即时视频中的人脸进行美化,然后将美化后的即时视频发送至通讯设备2,由通讯设备2中的视频显示模块显示该美化后的即时视频;若该方法的执行主体是移动终端2,移动终端1在通过自身的视频输入模块输入即时视频后,将该即时视频发送至移动终端2,移动终端2对即时视频中的人脸进行美化,再通过自身的视频显示模块显示该美化后的即时视频。本发明实施例对该交互系统中的该方法的具体的执行主体不加以限定。In particular, the execution body of the method provided by the embodiment of the present invention, that is, the electronic device, may be any one of the mobile terminal 1 and the mobile terminal 2. If the execution subject of the method is the mobile terminal 1, the mobile terminal 1 is After inputting the live video through the video input module of the user, the face in the instant video is beautified, and then the beautified instant video is sent to the communication device 2, and the beautified instant is displayed by the video display module in the communication device 2 If the execution body of the method is the mobile terminal 2, the mobile terminal 1 sends the live video to the mobile terminal 2 after inputting the live video through its own video input module, and the mobile terminal 2 beautifies the face in the instant video. And then display the beautified instant video through its own video display module. The specific implementation body of the method in the interaction system is not limited in the embodiment of the present invention.
实施例一Embodiment 1
本发明实施例提供了一种即时视频中的人脸美化方法,该方法应用于上述任一交互系统中,参见图3所示,该方法流程包括:An embodiment of the present invention provides a method for facial landscaping in an instant video. The method is applied to any of the foregoing interaction systems. Referring to FIG. 3, the method includes:
301、从即时视频的当前画面中获取人脸图像区域;301: Obtain a face image area from a current picture of the instant video;
具体的,该步骤可以包括:Specifically, the step may include:
获取当前画面的灰度图像;Obtaining a grayscale image of the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从灰度图像中,获取当前画面中的人脸图像区域。The face image area in the current picture is acquired from the gray image according to the face frame size, the face pixel distance, and the search zoom factor.
可选的,获取当前画面中的人脸图像区域的灰度图像之前,还可以对当前画面进行压缩,在完成压缩后,再执行获取当前画面中的人脸图像区域的灰度图像的步骤。Optionally, before acquiring the grayscale image of the face image area in the current picture, the current picture may be compressed, and after the compression is completed, the step of acquiring the gray image of the face image area in the current picture is performed.
可选的,获取当前画面中的人脸图像区域的灰度图像之后,还可以对人脸 图像区域的灰度图像进行直方图均衡化处理。Optionally, after obtaining the gray image of the face image area in the current picture, the face can also be The grayscale image of the image area is subjected to histogram equalization processing.
可选的,获取当前画面中的人脸图像区域之后,还可以对人脸图像区域进行缩放,获取人脸图像区域在未压缩的当前画面中的坐标。Optionally, after obtaining the face image area in the current picture, the face image area may also be scaled to obtain coordinates of the face image area in the uncompressed current picture.
302、对人脸图像区域进行美化,获取美化后的人脸图像区域;302. Beautifying the face image area, and obtaining the beautified face image area;
具体的,将灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,人脸细节图像至少包括眼睛、嘴和鼻子,人脸面具图像包括除眼睛、嘴和鼻子之外的人脸部分;Specifically, the face in the grayscale image is separated into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, and the face mask image includes the eyes, the mouth, and the nose. Face part
对人脸面具图像进行美化,获取美化后的人脸面具图像。Beautify the face mask image and obtain the beautified face mask image.
其中,对人脸面具图像进行美化包括:Among them, the beautification of the face mask image includes:
对人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
对反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
融合低通滤波图像、人脸面具图像以及高通滤波图像,获取美化后的人脸面具图像。A low-pass filtered image, a face mask image, and a high-pass filtered image are combined to obtain a beautified face mask image.
303、将美化后的人脸图像区域与人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括融合后的人脸图像区域的当前画面。303. fused the beautified face image area and the face image area to generate a merged face image area, and display a current picture including at least the merged face image area.
具体的,根据人脸图像区域在未压缩的当前画面中的坐标,将人脸细节图像与美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。Specifically, according to the coordinates of the face image area in the uncompressed current picture, the face detail image is merged with the beautified face mask image to obtain the beautified face image area.
本发明实施例提供了一种即时视频中的人脸美化方法通过从即时视频的当前画面中获取人脸图像区域,对人脸图像区域进行美化,获取美化后的人脸图像区域,并将美化后的人脸图像区域与人脸图像区域进行融合,生成融合后的人脸图像区域,显示至少包括融合后的人脸图像区域的当前画面。从而通过对即时视频中的人脸进行美化,并与之前的人脸区域进行融合,提高了即时视频的画面质量,增加了用户的体验效果。The embodiment of the present invention provides a method for facial beautification in instant video by obtaining a face image region from a current picture of an instant video, beautifying the face image region, and obtaining a beautified face image region, and beautifying the face image region. The subsequent face image area is merged with the face image area to generate a merged face image area, and the current picture including at least the merged face image area is displayed. Therefore, by landscaping the face in the instant video and merging with the previous face area, the picture quality of the instant video is improved, and the user experience is increased.
实施例二 Embodiment 2
本发明实施例提供了一种即时视频中的人脸美化方法,参见图4所示,该方法流程包括:An embodiment of the present invention provides a method for facializing a face in an instant video. Referring to FIG. 4, the method includes:
401、获取当前画面中的人脸图像区域的灰度图像。401. Obtain a grayscale image of a face image area in the current picture.
具体的,获取当前画面的灰度通道,通过该灰度通道获取当前画面中的人脸图像区域的灰度图像,并储存该灰度图像,其中,本发明实施例对具体的获取当前画面的灰度通道以及该灰度通道获取当前画面中的人脸图像区域的灰度图像的方式不加以限定。Specifically, the gray-scale channel of the current picture is obtained, and the gray-scale image of the face image area in the current picture is obtained by the gray-scale channel, and the gray-scale image is stored, where the embodiment of the present invention specifically acquires the current picture. The manner in which the grayscale channel and the grayscale channel acquire the grayscale image of the face image region in the current screen is not limited.
由于相比于彩色图像,灰度图像的数据量较小,所以通过获取当前画面中的人脸图像区域的灰度图像并进行美化,可以减少人脸美化过程中的数据处理量,节省存储内存,提高人脸美化效率。Since the amount of data of the grayscale image is small compared to the color image, by acquiring the grayscale image of the face image region in the current image and beautifying, the amount of data processing during the face beautification process can be reduced, and the storage memory is saved. To improve the efficiency of face beautification.
402、根据人脸框大小、人脸像素距离和搜索缩放因子,从灰度图像中获取当前画面中的人脸图像区域。402. Obtain a face image area in the current picture from the gray image according to the face frame size, the face pixel distance, and the search scaling factor.
具体的,人脸框大小可以用min objectSize表示,且该min objectSize可以为至少长度和宽度的矩阵,该矩阵可以为:[长度,宽度]T,长度和宽度分别可以为正有理数;人脸像素距离可以用min neighbor表示,且该min neighbor可以为自然数;搜索缩放因子可以用ScaleFactor表示,且该ScaleFactor可以为正有理数,其中,人脸框大小用于指示该灰度图像的长度和宽度;人脸像素距离用于指示在该灰度图像中最近的人脸画面的个数,若灰度图像中只有一张人脸,则人脸像素距离为1,若灰度图像中有两张人脸,则人脸像素距离为2,并以此类推;搜索缩放因子用于指示在进行人脸检测时对灰度图像的缩放比例。Specifically, the size of the face frame can be represented by min objectSize, and the min objectSize can be a matrix of at least length and width. The matrix can be: [length, width] T , length and width can be positive rational numbers, respectively; face pixels The distance can be represented by min neighbor, and the min neighbor can be a natural number; the search scaling factor can be represented by ScaleFactor, and the ScaleFactor can be a positive rational number, wherein the face frame size is used to indicate the length and width of the grayscale image; The face pixel distance is used to indicate the number of face images that are closest in the grayscale image. If there is only one face in the grayscale image, the face pixel distance is 1 if there are two faces in the grayscale image. , the face pixel distance is 2, and so on; the search zoom factor is used to indicate the scaling of the grayscale image when performing face detection.
优选的,在实际应用中,可以设置min objectSize=[40,40]T,min neighbor=2,ScaleFactor=1.2。Preferably, in practical applications, min objectSize=[40,40] T , min neighbor=2, ScaleFactor=1.2 can be set.
通过人脸框大小、人脸像素距离和搜索缩放因子从灰度图像中获取当前画面中的人脸图像区域,使得获取的人脸图像区域至少包括当前用户的人脸,可以提高人脸图像区域获取的准确性,从而实现了人脸美化的可靠性。同时,在 获取人脸图像区域的过程中,通过搜索缩放因子对该灰度图像进行缩放,可以减少获取人脸图像区域的过程中的数据处理量,降低了对设备的硬件要求,使得可以在移动终端上使用该方法进行即时视频中的人脸美化,并且可以提高移动终端上人脸美化效率。The face image area in the current picture is obtained from the gray image by the face frame size, the face pixel distance, and the search zoom factor, so that the acquired face image area includes at least the current user's face, and the face image area can be improved. The accuracy of the acquisition, thus achieving the reliability of face beautification. At the same time, at In the process of acquiring the face image region, the grayscale image is scaled by searching the zoom factor, which can reduce the data processing amount in the process of acquiring the face image region, and reduce the hardware requirement on the device, so that the mobile terminal can be This method is used for face beautification in instant video, and the face beautification efficiency on the mobile terminal can be improved.
值得注意的是,步骤401至步骤402是从即时视频的当前画面中获取人脸图像区域的过程,除了上述过程之外,还可以通过其他方式从即时视频的当前画面中获取人脸图像区域,本发明实施例对具体的获取过程不加以限定。It should be noted that steps 401 to 402 are processes for acquiring a face image region from the current picture of the live video. In addition to the above process, the face image region may be acquired from the current picture of the live video by other means. The specific acquisition process is not limited in the embodiment of the present invention.
403、对人脸图像区域进行缩放,获取人脸图像区域在未压缩的当前画面中的坐标。403. Scale the face image area to obtain coordinates of the face image area in the uncompressed current picture.
具体的,根据缩放比例计算公式对灰度图像进行缩放,缩放比例计算公式可以为:Specifically, the grayscale image is scaled according to the scaling formula, and the scaling formula can be:
Figure PCTCN2015098355-appb-000001
Figure PCTCN2015098355-appb-000001
其中,width表示该人脸图像区域的宽度,height表示该人脸图像区域的高度。Wherein, width represents the width of the face image area, and height represents the height of the face image area.
在完成缩放后,获取该人脸图像区域在未压缩的画面图像中的坐标,记为[x,y],并且[x,y]∈R2,并存储该坐标。After the zoom is completed, the coordinates of the face image area in the uncompressed picture image are obtained, denoted as [x, y], and [x, y] ∈ R 2 , and the coordinates are stored.
可选的,该步骤还可以包括对输出的人脸图像区域进行缩放,使该人脸图像区域在一个矩形范围内,其中,矩形范围是为了减少人脸美化过程中的运算量。具体的缩放过程可以为:Optionally, the step may further include scaling the output face image area such that the face image area is within a rectangular range, wherein the rectangular range is to reduce the amount of calculation in the face beautification process. The specific scaling process can be:
设置缩放比例为[a1,a2]∈R2,其中,a1表示宽度,a2表示高度,且a1和a2分别为正有理数,优选的,在实际应用中,可以设置a1的值为1.1,a2的值为1.05;Set the scaling to [a1, a2] ∈ R 2 , where a1 represents the width, a2 represents the height, and a1 and a2 are positive rational numbers, respectively. Preferably, in practical applications, the value of a1 can be set to 1.1, a2 The value is 1.05;
根据该缩放比例进行缩放。Scale according to the scaling.
在步骤403之后,还需要存储该灰度图像,具体的存储过程可以为:After step 403, the grayscale image needs to be stored, and the specific storage process may be:
根据人脸坐标[x,y,width,heigth]T分配新的内存,内存的大小与人脸的高宽一致,将上述检测出的人脸图像区域复制到新分配的内存中;Allocating new memory according to the face coordinate [x, y, width, heigth] T , the size of the memory is consistent with the height and width of the face, and copying the detected face image area to the newly allocated memory;
可选的,如果有多个人脸,重复该步骤,并存储新的人脸队列,该人脸队 列是指人脸的个数,当有一个人脸,队列的长度为1,假设有n个人脸,则队列的长度为n。Optionally, if there are multiple faces, repeat the step and store a new face queue, the face team The column refers to the number of faces. When there is a face, the length of the queue is 1, and if there are n faces, the length of the queue is n.
404、将灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,人脸细节图像至少包括眼睛、嘴和鼻子,人脸面具图像包括除包括眼睛、嘴和鼻子之外的人脸部分;404. Separating a face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, and the face mask image includes the eyes, the mouth, and the nose. Face part
具体的,该过程可以为:Specifically, the process can be:
a、计算该灰度图像的直方图;a. calculating a histogram of the grayscale image;
b、找到该直方图的近似谷点,并设置其相应的灰度级别为全局阀值;b. Find an approximate valley point of the histogram and set its corresponding gray level to a global threshold;
c、根据该全局阀值对人脸图像进行分段;c. segmenting the face image according to the global threshold;
其中,获取该人脸图像的眉毛、眼睛、鼻子和嘴巴在直方图中的取值,该取值在该直方图的近似谷点的左边,获取该人脸图像的皮肤在直方图中的取值,该取值在该近似谷点的右边,通过该直方图的该近似谷点,可以将人脸图像的眉毛、眼睛、鼻子和嘴巴的人脸细节和人脸皮肤进行分类。The value of the eyebrow, the eyes, the nose, and the mouth of the face image in the histogram is obtained, and the value is taken to the left of the approximate valley of the histogram, and the skin of the face image is obtained in the histogram. A value, which is to the right of the approximate valley point, through the approximate valley point of the histogram, the face details of the face image, the eye, the nose, and the face and the face skin can be classified.
值得注意的是,经过该步骤的分离,获得的人脸细节包括眉毛、眼睛、鼻子和嘴巴是需要保留原始图像的,获得该人脸的皮肤是要进行美化的,It is worth noting that after the separation of the steps, the obtained facial details including the eyebrows, eyes, nose and mouth are required to retain the original image, and the skin obtained for the face is to beautify,
其中,人脸面具图像包括除眼睛、嘴和鼻子之外的人脸部分。Among them, the face mask image includes a face portion other than the eyes, the mouth, and the nose.
d、获取二进制图像,其非零像素用于指明该二进制图像的细节;d, obtaining a binary image whose non-zero pixels are used to indicate the details of the binary image;
具体的,根据步骤c中获得直方图的近似谷点,将该谷点右边的人脸面具对应的位置的取值为0,将该谷点左边的人脸面具对应位置的取值为1或与0不同的数值,此处不加以限定;Specifically, according to the approximate valley point of the histogram obtained in step c, the position corresponding to the face mask on the right side of the valley point is 0, and the corresponding position of the face mask on the left side of the valley point is 1 or Values different from 0 are not limited here;
其中,在图像中眼睛、嘴巴等细节的亮度是要比皮肤暗的,皮肤是比较亮的,所以,变化的谷点就是皮肤和眼睛嘴巴等细节的分类的选择点。Among them, in the image, the brightness of the eyes, mouth and other details is darker than the skin, and the skin is brighter. Therefore, the changing valley point is the selection point of the classification of details such as skin and eyes and mouth.
e、存储该人脸细节和该二进制人脸面具图像。e. Store the face detail and the binary face mask image.
通过获取该灰度图像的直方图,获取该直方图的近似谷点,并将该近似谷点作为进行全局阈值分割的分割阈值,使得将人脸图像中的人脸细节和人脸面具分离,从而保留该人脸细节,同时获得了人脸面具,提高了移动终端上人脸 美化的效率。Obtaining an approximate valley point of the histogram by acquiring a histogram of the gray image, and using the approximate valley point as a segmentation threshold for performing global threshold segmentation, so that the face detail and the face mask in the face image are separated. Thereby retaining the face details, and at the same time obtaining a face mask, improving the face on the mobile terminal Beautify the efficiency.
405、对人脸面具图像进行美化,获取美化后的人脸面具图像。405. Beautify the face mask image and obtain the beautified face mask image.
具体的,该过程可以为:Specifically, the process can be:
a、获取人脸面具图像的三原色通道;a, obtaining the three primary color channels of the face mask image;
其中,三原色通道包括红色通道、黄色通道和蓝色通道,例如,获取该人脸面具的蓝色通道,也可以为红色通道和黄色通道,此处不加以限定,下面以蓝色通道为例,说明人脸面具美化的过程,The three primary color channels include a red channel, a yellow channel, and a blue channel. For example, the blue channel and the yellow channel of the face mask are obtained, which are not limited. Explain the process of beautifying the face mask,
b、对人脸面具图像进行高斯滤波,获取反差图像;b. Perform Gaussian filtering on the face mask image to obtain a contrast image;
其中,示例性的,可以通过蓝色通道对该人脸面具进行高斯滤波,生成反差图像,在高斯滤波过程中,设置滤波半径为[b1,b2]T,并通过以下公式进行高斯滤波:For example, the face mask can be Gaussian filtered by a blue channel to generate a contrast image. In the Gaussian filtering process, the filter radius is set to [b1, b2] T , and Gaussian filtering is performed by the following formula:
contrast Im age=GaussianFilter-Original+A1Contrast Im age=GaussianFilter-Original+A1
在该公式中,GaussianFilter为高斯滤波后的图像,Original为原始图像,A1为位移,且A1为正整数。In this formula, the GaussianFilter is a Gaussian filtered image, Original is the original image, A1 is the displacement, and A1 is a positive integer.
其中,b1和b2分别正整数,优选的,在实际应用中,可以设置b1=b2=31,A1=127;Wherein b1 and b2 are positive integers respectively, preferably, in practical applications, b1=b2=31, A1=127;
c、对反差图像进行高斯滤波,获取低通滤波图像;c. Perform Gaussian filtering on the contrast image to obtain a low-pass filtered image;
其中,可以利用高斯滤波对反差图像进行处理,滤波半径为[c1,c2]T,c1和c2分别为正整数,优选的,在实际应用中,可以设置c1=c2=11;The Gaussian filter can be used to process the contrast image, the filter radius is [c1, c2] T , and c1 and c2 are positive integers respectively. Preferably, in practical applications, c1=c2=11 can be set;
根据反差图像对原始图像进行处理,反差大的保留原始图,反差小的保留高斯滤波后的反差图像。The original image is processed according to the contrast image, the contrast is large, and the original image is preserved, and the contrast is small, and the Gaussian filtered contrast image is retained.
d、对低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;d. performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain the adjusted low-pass filtered image;
具体的,可以利用一下公式进行对比度调整: Specifically, you can use the formula to adjust the contrast:
Figure PCTCN2015098355-appb-000002
Figure PCTCN2015098355-appb-000002
其中
Figure PCTCN2015098355-appb-000003
d1、d2和d3分别为正有理数,优选的,在实际应用中,可以设置d1=2,d2=255,d3=128;
among them
Figure PCTCN2015098355-appb-000003
D1, d2 and d3 are positive rational numbers, respectively, preferably, in practical applications, d1=2, d2=255, d3=128;
除此之外,还可以使用拉格朗日插值法对图像的亮度进行调整,参数为:In addition, Lagrange interpolation can also be used to adjust the brightness of the image. The parameters are:
Figure PCTCN2015098355-appb-000004
Figure PCTCN2015098355-appb-000004
其中,e1、e2、e3和e4分别为正有理数,优选的,是实际应用中,可以设置e1=0.76,e2=e4=255,e3=0.85Where e1, e2, e3, and e4 are positive rational numbers, respectively. Preferably, in practical applications, e1=0.76, e2=e4=255, and e3=0.85 can be set.
获取并存储该低通滤波图。Acquire and store the low pass filter map.
e、对调整后的低通滤波图像进行高通滤波,获取高通滤波图像;e. performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
其中,复制第一人脸美化图像区域到新的内存,使用拉普拉斯滤波对图像进行滤波,得到高通细节,获取高通滤波图,并存储该高通滤波图;Copying the first face to beautify the image area to a new memory, filtering the image using Laplacian filtering, obtaining a high-pass detail, obtaining a high-pass filter map, and storing the high-pass filter map;
此处对滤波的方式不加以限定,也可以使用canny等其他的高通图的滤波方式。The filtering method is not limited here, and other high-pass filtering methods such as canny may be used.
获取并存储该高通滤波图。Acquire and store the high pass filter map.
f、融合低通滤波图像、人脸面具图像以及高通滤波图像,获取美化后的人脸面具图像。f. Combine the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
具体的,对高通滤波图像、人脸面具图像、低通滤波图像进行融合,计算公式为:Specifically, the high-pass filtered image, the face mask image, and the low-pass filtered image are fused, and the calculation formula is:
Figure PCTCN2015098355-appb-000005
Figure PCTCN2015098355-appb-000005
其中,FaceArea为人脸图像区域,Lowpass为低通滤波图,Highpass为高通滤波图,且f1为正整数,优选的,在实际应用中,可以设置f1=255。Among them, FaceArea is the face image area, Lowpass is the low-pass filter, Highpass is the high-pass filter, and f1 is a positive integer. Preferably, in practical applications, f1=255 can be set.
g、融合低通滤波图像、人脸面具图像以及高通滤波图像,获取美化后的人脸面具图像。 g. Combine the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
其中,步骤404至步骤405是对人脸图像区域进行美化的过程,除上述过程外,还可以通过其他方式对人脸图像区域进行美化,本发明实施例对具体的美化方式不加以限定。Step 404 to step 405 are a process of beautifying the face image area. In addition to the above process, the face image area can be beautified by other means. The specific beautification mode is not limited in the embodiment of the present invention.
406、根据人脸图像区域在未压缩的当前画面中的坐标,将人脸细节图像与美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。406. Combine the face detail image with the beautified face mask image according to the coordinates of the face image area in the uncompressed current picture, and obtain the beautified face image area.
具体的,在人脸图像区域在未压缩的当前画面中的坐标的基础上,的人脸细节图像与美化后的人脸面具图进行融合,计算公式为:Specifically, based on the coordinates of the face image area in the uncompressed current picture, the face detail image is merged with the beautified face mask image, and the calculation formula is:
如果人脸面具图mask=255,
Figure PCTCN2015098355-appb-000006
If the face mask map mask=255,
Figure PCTCN2015098355-appb-000006
否则,
Figure PCTCN2015098355-appb-000007
otherwise,
Figure PCTCN2015098355-appb-000007
其中,FaceArea为步骤中获得的人脸区域,dealedface为步骤S8中获得的图像,Output为美化后的人脸图像区域,
Figure PCTCN2015098355-appb-000008
Figure PCTCN2015098355-appb-000009
为0和1之间的有理数,优选的,在实际应用中,可以设置
Figure PCTCN2015098355-appb-000010
Wherein, FaceArea is the face area obtained in the step, the deleted face is the image obtained in step S8, and Output is the face image area after the beautification,
Figure PCTCN2015098355-appb-000008
with
Figure PCTCN2015098355-appb-000009
a rational number between 0 and 1, preferably, in practical applications, can be set
Figure PCTCN2015098355-appb-000010
可选的,在步骤401之前,还可以执行步骤407,参照图5所示,Optionally, before step 401, step 407 may also be performed, as shown in FIG. 5,
407、对当前画面进行压缩。407. Compress the current picture.
具体的,对当前画面的人脸图像区域进行取样,并将该区域进行缩放,缩放计算公式为:Specifically, the face image area of the current picture is sampled, and the area is scaled, and the scaling calculation formula is:
Figure PCTCN2015098355-appb-000011
Figure PCTCN2015098355-appb-000011
其中,width表示该当前画面的宽度,height表示该人脸图像区域的高度。Where width represents the width of the current picture and height represents the height of the face image area.
该步骤中取样的方式为图像的直接下采样或者使用双线性插值法,或者其他方式,此处不加以限定。The manner of sampling in this step is direct downsampling of the image or using bilinear interpolation, or other means, which is not limited herein.
通过将该当前画面的人脸图像区域进行缩放,使得该当前画面的人脸图像区域归一化,从而可以控制算法的复杂度,使之不会因为图像的分辨率的变大而变大,从而减少了运算量,使得可以在移动终端上实现即时视频中的人脸美化,从而提高了人脸美化的效率。By scaling the face image area of the current picture, the face image area of the current picture is normalized, so that the complexity of the algorithm can be controlled so that it does not become larger because the resolution of the image becomes larger. Thereby, the amount of calculation is reduced, so that the face beautification in the instant video can be realized on the mobile terminal, thereby improving the efficiency of the face beautification.
可选的,在步骤401之前,还可以执行步骤408,参照图6所示, Optionally, before step 401, step 408 may also be performed, as shown in FIG.
408、对人脸图像区域的灰度图像进行直方图均衡化处理。408. Perform histogram equalization processing on the grayscale image of the face image region.
具体的,对人脸图像区域的灰度图像进行直方图均衡化处理,获取该直方图均衡化处理,并储存该直方图均衡化数据。Specifically, a histogram equalization process is performed on the grayscale image of the face image region, the histogram equalization process is acquired, and the histogram equalization data is stored.
通过将该人脸图像区域的灰度图像进行直方图均衡化处理,可以排除光照对人脸检测的干扰,提高了移动终端上人脸美化的可靠性。By performing histogram equalization processing on the grayscale image of the face image region, the interference of illumination on the face detection can be eliminated, and the reliability of the face beautification on the mobile terminal is improved.
由于本发明实施例所提供的即时视频中的人脸美化方法,通过对即时视频中的人脸图像进行缩放,减少了人脸美化过程中的数据处理量,同时控制了算法的复杂度,不仅使得移动终端的硬件可以满足人脸美化时的硬件要求,还避免了在人脸美化过程中对设备的处理资源和存储资源的大量占用,提高了用户体验效果。Due to the face beautification method in the instant video provided by the embodiment of the present invention, by scaling the face image in the instant video, the amount of data processing in the face beautification process is reduced, and the complexity of the algorithm is controlled, not only The hardware of the mobile terminal can meet the hardware requirements of the face beautification, and also avoids a large occupation of the processing resources and storage resources of the device in the face beautification process, thereby improving the user experience.
本发明实施例提供了一种即时视频中的人脸美化方法,通过在移动终端上实现即时视频中的人脸美化,增加了用户的体验效果;同时,通过人脸图像区域的灰度图像进行直方图均衡化处理,提高了移动终端上人脸美化的可靠性,且通过控制算法的复杂度,提高了移动终端上人脸美化的效率。The embodiment of the invention provides a method for facializing the face in the instant video, which realizes the facial effect beautification in the instant video on the mobile terminal, thereby increasing the user experience; and simultaneously performing the grayscale image of the face image region. The histogram equalization processing improves the reliability of the face beautification on the mobile terminal, and improves the efficiency of the face beautification on the mobile terminal by controlling the complexity of the algorithm.
实施例三Embodiment 3
本发明实施例提供一种电子设备7,参照图7所示,该电子设备7包括:An embodiment of the present invention provides an electronic device 7. Referring to FIG. 7, the electronic device 7 includes:
获取模块71,用于从即时视频的当前画面中获取人脸图像区域;An obtaining module 71, configured to acquire a face image area from a current picture of the instant video;
美化模块72,用于对人脸图像区域进行美化,获取美化后的人脸图像区域;The beautification module 72 is configured to beautify the face image area to obtain the beautified face image area;
融合模块73,将美化后的人脸图像区域与人脸图像区域进行融合,生成融合后的人脸图像区域;The fusion module 73 fuses the beautified face image area and the face image area to generate a merged face image area;
显示模块74,用于显示至少包括融合后的人脸图像区域的当前画面;The display module 74 is configured to display a current picture including at least the merged face image area;
存储模块75,存储模块用于存储人脸图像区域、美化后的人脸图像区域以及融合后的人脸图像区域。The storage module 75 is configured to store a face image area, a beautified face image area, and a merged face image area.
可选的,获取模块71具体用于:Optionally, the obtaining module 71 is specifically configured to:
获取当前画面中的人脸图像区域的灰度图像; Obtaining a grayscale image of a face image area in the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从灰度图像中获取当前画面中的人脸图像区域;Obtaining a face image area in the current picture from the gray image according to the face frame size, the face pixel distance, and the search scaling factor;
存储模块还用于存储灰度图像。The storage module is also used to store grayscale images.
可选的,电子设备7还包括:Optionally, the electronic device 7 further includes:
压缩模块,用于对当前画面进行压缩,在完成压缩后,触发获取模块执行获取当前画面中的人脸图像区域的灰度图像的步骤。The compression module is configured to compress the current picture, and after the compression is completed, trigger the acquiring module to perform the step of acquiring the gray image of the face image area in the current picture.
可选的,电子设备7还包括:Optionally, the electronic device 7 further includes:
处理模块,用于对人脸图像区域的灰度图像进行直方图均衡化处理。The processing module is configured to perform histogram equalization processing on the grayscale image of the face image region.
可选的,Optional,
压缩模块还用于对人脸图像区域进行缩放,获取模块还用于获取人脸图像区域在未压缩的当前画面中的坐标;The compression module is further configured to scale the face image area, and the obtaining module is further configured to acquire coordinates of the face image area in the uncompressed current picture;
存储模块还用于存储坐标。The storage module is also used to store coordinates.
可选的,Optional,
电子设备7还包括检测模块,用于对灰度图像中的人脸细节进行检测;The electronic device 7 further includes a detecting module for detecting face details in the grayscale image;
电子设备7还包括分离模块,用于将灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,人脸细节图像至少包括眼睛、嘴和鼻子,人脸面具图像包括除眼睛、嘴和鼻子之外的人脸部分;The electronic device 7 further includes a separation module for separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth and a nose, and the face mask image includes an eye removal a face part other than the mouth and nose;
美化模块72用于对人脸面具图像进行美化,获取美化后的人脸面具图像。The beautification module 72 is used to beautify the face mask image and obtain the beautified face mask image.
可选的,美化模块具体用于:Optionally, the beautification module is specifically used to:
对人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
对反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
融合模块73用于融合低通滤波图像、人脸面具图像以及高通滤波图像,获取美化后的人脸面具图像。The fusion module 73 is configured to fuse the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
可选的,融合模块73用于: Optionally, the fusion module 73 is configured to:
将人脸细节图像与美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
本发明实施例提供了一种电子设备,该电子设备通过在移动终端上实现即时视频中的人脸美化,增加了用户的体验效果。The embodiment of the invention provides an electronic device, which enhances the user experience by realizing face beautification in the instant video on the mobile terminal.
实施例四Embodiment 4
本发明实施例提供一种电子设备,参照图8所示,该电子设备8包括视频输入模块81、视频输出模块82、发送模块83、接收模块84、存储器85以及分别与视频输入模块、视频输出模块、发送模块、接收模块和存储器连接的处理器86,其中,存储器85中存储一组程序代码,处理器86用于调用存储器85中存储的程序代码,执行以下操作:An embodiment of the present invention provides an electronic device. Referring to FIG. 8 , the electronic device 8 includes a video input module 81 , a video output module 82 , a sending module 83 , a receiving module 84 , a memory 85 , and a video input module and a video output respectively. A module 86, a transmitting module, a receiving module, and a memory connected processor, wherein the memory 85 stores a set of program codes, and the processor 86 is configured to call the program code stored in the memory 85 to perform the following operations:
从即时视频的当前画面中获取人脸图像区域;Obtaining a face image area from a current picture of the instant video;
对人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the face image area to obtain the beautified face image area;
将美化后的人脸图像区域与人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括融合后的人脸图像区域的当前画面。The beautified face image area and the face image area are merged to generate a merged face image area, and the current picture including at least the merged face image area is displayed.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
获取当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
根据人脸框大小、人脸像素距离和搜索缩放因子,从灰度图像中获取当前画面中的人脸图像区域。The face image area in the current picture is acquired from the gray image according to the face frame size, the face pixel distance, and the search zoom factor.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
对当前画面进行压缩,在完成压缩后,执行获取当前画面中的人脸图像区域的灰度图像的步骤。The current picture is compressed, and after the compression is completed, a step of acquiring a grayscale image of the face image area in the current picture is performed.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
对人脸图像区域的灰度图像进行直方图均衡化处理。 A histogram equalization process is performed on the grayscale image of the face image region.
可选的,处理器还用于调用存储器中存储的程序代码,执行以下操作:Optionally, the processor is further configured to call the program code stored in the memory, and perform the following operations:
对人脸图像区域进行缩放,获取人脸图像区域在未压缩的当前画面中的坐标。The face image area is scaled to obtain the coordinates of the face image area in the uncompressed current picture.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
对灰度图像中的人脸细节进行检测;Detecting face details in grayscale images;
将灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,人脸细节图像至少包括眼睛、嘴和鼻子,人脸面具图像包括除至少包括眼睛、嘴和鼻子之外的人脸部分;Separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, and the face mask image includes a person other than at least including the eyes, the mouth, and the nose. Face part
对人脸面具图像进行美化,获取美化后的人脸面具图像。Beautify the face mask image and obtain the beautified face mask image.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
对人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
对反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
对低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
对调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
融合低通滤波图像、人脸面具图像以及高通滤波图像,获取美化后的人脸面具图像。A low-pass filtered image, a face mask image, and a high-pass filtered image are combined to obtain a beautified face mask image.
可选的,处理器86还用于调用存储器85中存储的程序代码,执行以下操作:Optionally, the processor 86 is further configured to call the program code stored in the memory 85, and perform the following operations:
将人脸细节图像与美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
本发明实施例提供了一种电子设备,该电子设备通过在移动终端上实现即时视频中的人脸美化,增加了用户的体验效果。The embodiment of the invention provides an electronic device, which enhances the user experience by realizing face beautification in the instant video on the mobile terminal.
实施例五Embodiment 5
本发明实施例提供的一种即时视频中的人脸美化系统,该系统包括: A facial landscaping system in an instant video provided by an embodiment of the present invention, the system comprising:
第一通信终端,用于获取即时视频,第一通信终端至少包括视频输入模块。The first communication terminal is configured to acquire an instant video, and the first communication terminal includes at least a video input module.
获取设备,用于从即时视频的当前画面中获取人脸图像区域;Obtaining a device, configured to obtain a face image area from a current picture of the instant video;
美化设备,用于对人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the device for beautifying the face image area and obtaining the beautified face image area;
融合设备,用于将美化后的人脸图像区域与人脸图像区域进行融合,生成融合后的人脸图像区域;a fusion device, configured to fuse the beautified face image area and the face image area to generate a merged face image area;
第二通信终端,至少包括显示模块,用于显示至少包括融合后的人脸图像区域的当前画面;The second communication terminal includes at least a display module, configured to display a current picture including at least the merged face image area;
存储设备,用于存储人脸图像区域、美化后的人脸图像区域以及融合后的人脸图像区域。The storage device is configured to store a face image area, a beautified face image area, and a merged face image area.
值得注意的是,获取设备、美化设备、美化设备以及存储设备可以是单独的设备,也可以是集成在其他设备中,本发明实施例不加以限定。It is to be noted that the acquiring device, the beautifying device, the beautifying device, and the storage device may be separate devices, or may be integrated into other devices, which are not limited in the embodiment of the present invention.
本发明实施例提供了一种即时视频中的人脸美化系统,通过在移动终端上实现即时视频中的人脸美化,增加了用户的体验效果。The embodiment of the invention provides a face beautification system in real-time video, which achieves the user's experience effect by realizing the face beautification in the instant video on the mobile terminal.
需要说明的是:上述实施例提供的检测终端位置的装置及设备在检测终端位置时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置及设备的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的检测终端位置的装置及设备与检测终端位置的方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。It should be noted that the device and the device for detecting the location of the terminal provided by the foregoing embodiment are only illustrated by the division of the foregoing functional modules when detecting the location of the terminal. In actual applications, the functions may be allocated differently according to requirements. The function module is completed, that is, the internal structure of the device and the device are divided into different functional modules to complete all or part of the functions described above. In addition, the device and the device for detecting the location of the terminal provided by the foregoing embodiment are the same as the method for detecting the location of the terminal. For the specific implementation process, refer to the method embodiment, and details are not described herein again.
上述本发明实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.
以上所述仅为本发明的较佳实施例,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的 保护范围之内。 The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., which are included in the spirit and scope of the present invention, should be included in the present invention. Within the scope of protection.

Claims (24)

  1. 一种即时视频中的人脸美化的方法,其特征在于,所述方法包括:A method for facial landscaping in instant video, characterized in that the method comprises:
    从即时视频的当前画面中获取人脸图像区域;Obtaining a face image area from a current picture of the instant video;
    对所述人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the face image area to obtain a beautified face image area;
    将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括所述融合后的人脸图像区域的当前画面。The beautified face image area is merged with the face image area to generate a merged face image area, and a current picture including at least the merged face image area is displayed.
  2. 根据权利要求1所述的方法,其特征在于,所述从即时视频的当前画面中获取人脸图像区域包括:The method according to claim 1, wherein the obtaining the face image area from the current picture of the instant video comprises:
    获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
    根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域。The face image region in the current picture is acquired from the grayscale image according to the face frame size, the face pixel distance, and the search zoom factor.
  3. 根据权利要求2所述的方法,其特征在于,所述获取所述当前画面中的人脸图像区域的灰度图像之前,所述方法还包括:The method according to claim 2, wherein before the acquiring the grayscale image of the face image region in the current picture, the method further comprises:
    对所述当前画面进行压缩,在完成压缩后,执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。Compressing the current picture, and after completing the compression, performing the step of acquiring a grayscale image of the face image area in the current picture.
  4. 根据权利要求2或3所述的方法,其特征在于,所述获取所述当前画面中的人脸图像区域的灰度图像之后,所述方法还包括:The method according to claim 2 or 3, wherein after the acquiring the grayscale image of the face image region in the current picture, the method further comprises:
    对所述人脸图像区域的灰度图像进行直方图均衡化处理。A histogram equalization process is performed on the grayscale image of the face image region.
  5. 根据权利要求2至4任一所述的方法,其特征在于,所述获取当前画面中的人脸图像区域之后,所述方法还包括:The method according to any one of claims 2 to 4, wherein after the acquiring the face image area in the current picture, the method further comprises:
    对所述人脸图像区域进行缩放,获取所述人脸图像区域在未压缩的当前画面中的坐标。The face image area is scaled, and the coordinates of the face image area in the uncompressed current picture are acquired.
  6. 根据权利要求1至5任一所述的方法,其特征在于,对所述人脸图像区域进行美化包括:The method according to any one of claims 1 to 5, wherein the beautifying the face image area comprises:
    对所述灰度图像中的人脸细节进行检测; Detecting face details in the grayscale image;
    将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除至少包括眼睛、嘴和鼻子之外的人脸部分;Separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the face mask image including at least an eye, a mouth And the face part outside the nose;
    对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The face mask image is beautified to obtain a beautified face mask image.
  7. 根据权利要求6所述的方法,其特征在于,所述对所述人脸面具图像进行美化包括:The method according to claim 6, wherein the beautifying the face mask image comprises:
    对所述人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
    对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
    对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
    对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
    融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。And integrating the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
  8. 根据权利要求7所述的方法,其特征在于,所述获取美化后的人脸图像区域包括:The method according to claim 7, wherein the obtaining the beautified face image area comprises:
    根据所述人脸图像区域在未压缩的当前画面中的坐标,将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。And merging the face detail image with the beautified face mask image according to the coordinates of the face image region in the uncompressed current picture to obtain the beautified face image region.
  9. 一种电子设备,其特征在于,所述设备包括:An electronic device, the device comprising:
    获取模块,用于从即时视频的当前画面中获取人脸图像区域;An obtaining module, configured to obtain a face image area from a current picture of the instant video;
    美化模块,用于对所述人脸图像区域进行美化,获取美化后的人脸图像区域;a beautification module, configured to beautify the face image area, and obtain a beautified face image area;
    融合模块,将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域;a fusion module, the beautified face image area is merged with the face image area to generate a merged face image area;
    显示模块,用于显示至少包括所述融合后的人脸图像区域的当前画面;a display module, configured to display a current picture including at least the merged face image area;
    存储模块,所述存储模块用于存储所述人脸图像区域、美化后的人脸图像区域以及所述融合后的人脸图像区域。 And a storage module, configured to store the face image area, the beautified face image area, and the merged face image area.
  10. 根据权利要求9所述的设备,其特征在于,所述获取模块具体用于:The device according to claim 9, wherein the obtaining module is specifically configured to:
    获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
    根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域;Obtaining a face image region in the current picture from the grayscale image according to a face frame size, a face pixel distance, and a search scaling factor;
    所述存储模块还用于存储所述灰度图像。The storage module is further configured to store the grayscale image.
  11. 根据权利要求10所述的设备,其特征在于,所述设备还包括:The device according to claim 10, wherein the device further comprises:
    压缩模块,用于对所述当前画面进行压缩,在完成压缩后,触发所述获取模块执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。And a compression module, configured to compress the current picture, and after the compression is completed, trigger the acquiring module to perform the step of acquiring the gray image of the face image area in the current picture.
  12. 根据权利要求10或11所述的设备,其特征在于,所述设备还包括:The device according to claim 10 or 11, wherein the device further comprises:
    处理模块,用于对所述人脸图像区域的灰度图像进行直方图均衡化处理。And a processing module, configured to perform a histogram equalization process on the grayscale image of the face image region.
  13. 根据权利要求10至12任一所述的设备,其特征在于,Apparatus according to any one of claims 10 to 12, wherein
    所述压缩模块还用于对所述人脸图像区域进行缩放,所述获取模块还用于获取所述人脸图像区域在未压缩的当前画面中的坐标;The compression module is further configured to scale the face image area, and the acquiring module is further configured to acquire coordinates of the face image area in an uncompressed current picture;
    所述存储模块还用于存储所述坐标。The storage module is further configured to store the coordinates.
  14. 根据权利要求9至13任一所述的设备,其特征在于,Apparatus according to any one of claims 9 to 13 wherein:
    所述设备还包括检测模块,用于对所述灰度图像中的人脸细节进行检测;The device further includes a detecting module configured to detect a face detail in the grayscale image;
    所述设备还包括分离模块,用于将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除眼睛、嘴和鼻子之外的人脸部分;The apparatus further includes a separation module for separating a face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the person The face mask image includes a face portion other than the eyes, mouth, and nose;
    所述美化模块用于对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The beautification module is configured to beautify the face mask image to obtain a beautified face mask image.
  15. 根据权利要求14所述的设备,其特征在于,The device of claim 14 wherein:
    所述美化模块具体用于:The beautification module is specifically used to:
    对所述人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
    对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
    对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波 图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain adjusted low-pass filtering image;
    对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
    所述融合模块用于融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。The fusion module is configured to fuse the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
  16. 根据权利要求15所述的设备,其特征在于,所述融合模块用于:The device according to claim 15, wherein the fusion module is configured to:
    将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
  17. 一种电子设备,其特征在于,包括视频输入模块、视频输出模块、发送模块、接收模块、存储器以及分别与所述视频输入模块、所述视频输出模块、所述发射模块、所述接收模块和所述存储器连接的处理器,其中,所述存储器中存储一组程序代码,所述处理器用于调用所述存储器中存储的程序代码,执行以下操作:An electronic device, comprising: a video input module, a video output module, a transmitting module, a receiving module, a memory, and the video input module, the video output module, the transmitting module, the receiving module, and The memory-connected processor, wherein the memory stores a set of program code, the processor is configured to invoke program code stored in the memory, and perform the following operations:
    从即时视频的当前画面中获取人脸图像区域;Obtaining a face image area from a current picture of the instant video;
    对所述人脸图像区域进行美化,获取美化后的人脸图像区域;Beautifying the face image area to obtain a beautified face image area;
    将所述美化后的人脸图像区域与所述人脸图像区域进行融合,生成融合后的人脸图像区域,并显示至少包括所述融合后的人脸图像区域的当前画面。The beautified face image area is merged with the face image area to generate a merged face image area, and a current picture including at least the merged face image area is displayed.
  18. 根据权利要求17所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to claim 17, wherein the processor is further configured to invoke program code stored in the memory to perform the following operations:
    获取所述当前画面中的人脸图像区域的灰度图像;Obtaining a grayscale image of a face image area in the current picture;
    根据人脸框大小、人脸像素距离和搜索缩放因子,从所述灰度图像中获取当前画面中的人脸图像区域。The face image region in the current picture is acquired from the grayscale image according to the face frame size, the face pixel distance, and the search zoom factor.
  19. 根据权利要求18所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to claim 18, wherein the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
    对所述当前画面进行压缩,在完成压缩后,执行所述获取所述当前画面中的人脸图像区域的灰度图像的步骤。Compressing the current picture, and after completing the compression, performing the step of acquiring a grayscale image of the face image area in the current picture.
  20. 根据权利要求18或19所述的电子设备,其特征在于,所述处理器还 用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to claim 18 or 19, wherein said processor further Used to call the program code stored in the memory, and perform the following operations:
    对所述人脸图像区域的灰度图像进行直方图均衡化处理。A histogram equalization process is performed on the grayscale image of the face image region.
  21. 根据权利要求18至20任一所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to any one of claims 18 to 20, wherein the processor is further configured to call the program code stored in the memory, and perform the following operations:
    对所述人脸图像区域进行缩放,获取所述人脸图像区域在未压缩的当前画面中的坐标。The face image area is scaled, and the coordinates of the face image area in the uncompressed current picture are acquired.
  22. 根据权利要求17至21任一所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to any one of claims 17 to 21, wherein the processor is further configured to call the program code stored in the memory, and perform the following operations:
    对所述灰度图像中的人脸细节进行检测;Detecting face details in the grayscale image;
    将所述灰度图像中的人脸分离为人脸细节图像和人脸面具图像,其中,所述人脸细节图像至少包括眼睛、嘴和鼻子,所述人脸面具图像包括除至少包括眼睛、嘴和鼻子之外的人脸部分;Separating the face in the grayscale image into a face detail image and a face mask image, wherein the face detail image includes at least an eye, a mouth, and a nose, the face mask image including at least an eye, a mouth And the face part outside the nose;
    对所述人脸面具图像进行美化,获取美化后的人脸面具图像。The face mask image is beautified to obtain a beautified face mask image.
  23. 根据权利要求22所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to claim 22, wherein the processor is further configured to call the program code stored in the memory, and perform the following operations:
    对所述人脸面具图像进行高斯滤波,获取反差图像;Performing Gaussian filtering on the face mask image to obtain a contrast image;
    对所述反差图像进行高斯滤波,获取低通滤波图像;Performing Gaussian filtering on the contrast image to obtain a low-pass filtered image;
    对所述低通滤波图像进行对比度调整和亮度调整,获取调整后的低通滤波图像;Performing contrast adjustment and brightness adjustment on the low-pass filtered image to obtain an adjusted low-pass filtered image;
    对所述调整后的低通滤波图像进行高通滤波,获取高通滤波图像;Performing high-pass filtering on the adjusted low-pass filtered image to obtain a high-pass filtered image;
    融合所述低通滤波图像、所述人脸面具图像以及所述高通滤波图像,获取所述美化后的人脸面具图像。And integrating the low-pass filtered image, the face mask image, and the high-pass filtered image to obtain the beautified face mask image.
  24. 根据权利要求23所述的电子设备,其特征在于,所述处理器还用于调用所述存储器中存储的程序代码,执行以下操作:The electronic device according to claim 23, wherein the processor is further configured to invoke program code stored in the memory to perform the following operations:
    将所述人脸细节图像与所述美化后的人脸面具图像进行融合,获得美化后的人脸图像区域。 The face detail image is merged with the beautified face mask image to obtain a beautified face image area.
PCT/CN2015/098355 2014-12-24 2015-12-23 Method for face beautification in real-time video and electronic equipment WO2016101883A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201410840316.1A CN104574306A (en) 2014-12-24 2014-12-24 Face beautifying method for real-time video and electronic equipment
CN201410840316.1 2014-12-24

Publications (1)

Publication Number Publication Date
WO2016101883A1 true WO2016101883A1 (en) 2016-06-30

Family

ID=53090287

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/098355 WO2016101883A1 (en) 2014-12-24 2015-12-23 Method for face beautification in real-time video and electronic equipment

Country Status (2)

Country Link
CN (1) CN104574306A (en)
WO (1) WO2016101883A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107424130A (en) * 2017-07-10 2017-12-01 北京小米移动软件有限公司 Picture U.S. face method and apparatus

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104574306A (en) * 2014-12-24 2015-04-29 掌赢信息科技(上海)有限公司 Face beautifying method for real-time video and electronic equipment
CN104469253A (en) * 2015-01-05 2015-03-25 掌赢信息科技(上海)有限公司 Face beautification method in real-time video and electronic equipment
CN104915981A (en) * 2015-05-08 2015-09-16 寇懿 Three-dimensional hairstyle design method based on somatosensory sensor
CN106170069A (en) * 2015-05-21 2016-11-30 华硕电脑股份有限公司 Electronic installation and non-momentary computer-readable medium storing
CN104952036B (en) * 2015-06-18 2018-11-06 福州瑞芯微电子股份有限公司 Method for beautifying faces and electronic equipment in a kind of instant video
CN105007446A (en) * 2015-06-18 2015-10-28 美国掌赢信息科技有限公司 Instant video display method and electronic device
CN106920208A (en) * 2015-12-24 2017-07-04 掌赢信息科技(上海)有限公司 A kind of face stylizing method and electronic equipment based on image procossing
CN105631417B (en) * 2015-12-24 2018-11-09 武汉鸿瑞达信息技术有限公司 Video enhancement system and method applied to internet video live streaming
CN106920212A (en) * 2015-12-24 2017-07-04 掌赢信息科技(上海)有限公司 A kind of method and electronic equipment for sending stylized video
CN106921840B (en) * 2015-12-25 2020-01-07 掌赢信息科技(上海)有限公司 Face beautifying method, device and system in instant video
CN106028138A (en) * 2016-07-22 2016-10-12 乐视控股(北京)有限公司 Live broadcast video processing method and device
CN106231367B (en) * 2016-09-23 2020-03-06 广州华多网络科技有限公司 Beautifying processing method and device
CN106341672A (en) * 2016-09-30 2017-01-18 乐视控股(北京)有限公司 Image processing method, apparatus and terminal
CN106612396B (en) * 2016-11-15 2020-06-16 努比亚技术有限公司 Photographing device, terminal and method
CN106971165B (en) * 2017-03-29 2018-08-10 武汉斗鱼网络科技有限公司 A kind of implementation method and device of filter
CN107045714A (en) * 2017-05-11 2017-08-15 杭州知聊信息技术有限公司 A kind of beautifying faces algorithm for live video communication
CN107392110A (en) * 2017-06-27 2017-11-24 五邑大学 Beautifying faces system based on internet
CN108537155B (en) * 2018-03-29 2021-01-26 Oppo广东移动通信有限公司 Image processing method, image processing device, electronic equipment and computer readable storage medium
CN108762829B (en) * 2018-05-03 2021-07-09 北京奇虎科技有限公司 Method, client and terminal for beautifying image
JP2020136738A (en) * 2019-02-13 2020-08-31 シャープ株式会社 Image processing device, image communication system, and image processing method
CN111131744B (en) * 2019-12-26 2021-04-20 杭州当虹科技股份有限公司 Privacy protection method based on video communication

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060115172A1 (en) * 2004-11-30 2006-06-01 Qian Lin Face enhancement in a digital video
US20070041640A1 (en) * 2003-03-20 2007-02-22 Omron Corporation Image processing device
CN101299267A (en) * 2008-07-02 2008-11-05 北京中星微电子有限公司 Method and device for processing human face image
CN101916370A (en) * 2010-08-31 2010-12-15 上海交通大学 Method for processing non-feature regional images in face detection
CN103927719A (en) * 2014-04-04 2014-07-16 北京金山网络科技有限公司 Picture processing method and device
CN104318262A (en) * 2014-09-12 2015-01-28 上海明穆电子科技有限公司 Method and system for replacing skin through human face photos
CN104574306A (en) * 2014-12-24 2015-04-29 掌赢信息科技(上海)有限公司 Face beautifying method for real-time video and electronic equipment
CN104853134A (en) * 2014-02-13 2015-08-19 腾讯科技(深圳)有限公司 Video communication method and video communication device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070041640A1 (en) * 2003-03-20 2007-02-22 Omron Corporation Image processing device
US20060115172A1 (en) * 2004-11-30 2006-06-01 Qian Lin Face enhancement in a digital video
CN101299267A (en) * 2008-07-02 2008-11-05 北京中星微电子有限公司 Method and device for processing human face image
CN101916370A (en) * 2010-08-31 2010-12-15 上海交通大学 Method for processing non-feature regional images in face detection
CN104853134A (en) * 2014-02-13 2015-08-19 腾讯科技(深圳)有限公司 Video communication method and video communication device
CN103927719A (en) * 2014-04-04 2014-07-16 北京金山网络科技有限公司 Picture processing method and device
CN104318262A (en) * 2014-09-12 2015-01-28 上海明穆电子科技有限公司 Method and system for replacing skin through human face photos
CN104574306A (en) * 2014-12-24 2015-04-29 掌赢信息科技(上海)有限公司 Face beautifying method for real-time video and electronic equipment

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107424130A (en) * 2017-07-10 2017-12-01 北京小米移动软件有限公司 Picture U.S. face method and apparatus

Also Published As

Publication number Publication date
CN104574306A (en) 2015-04-29

Similar Documents

Publication Publication Date Title
WO2016101883A1 (en) Method for face beautification in real-time video and electronic equipment
US10580140B2 (en) Method and system of real-time image segmentation for image processing
CN105323425B (en) Scene motion correction in blending image system
EP2889832B1 (en) Image processing apparatus and method
DE112017002119T5 (en) Real-time eye contact correction using neural network-based machine learning
CN106683100B (en) Image segmentation defogging method and terminal
US20110148868A1 (en) Apparatus and method for reconstructing three-dimensional face avatar through stereo vision and face detection
WO2016110188A1 (en) Method and electronic device for aesthetic enhancements of face in real-time video
CN107771391B (en) Method and apparatus for determining exposure time of image frame
WO2016165060A1 (en) Skin detection based on online discriminative modeling
US9342751B2 (en) User hand detecting device for detecting user's hand region and method thereof
KR102277048B1 (en) Preview photo blurring method and device and storage medium
EP3644599A1 (en) Video processing method and apparatus, electronic device, and storage medium
CN108594997B (en) Gesture skeleton construction method, device, equipment and storage medium
US10810462B2 (en) Object detection with adaptive channel features
CN112840376A (en) Image processing method, device and equipment
CN108259770B (en) Image processing method, image processing device, storage medium and electronic equipment
CN108574803B (en) Image selection method and device, storage medium and electronic equipment
KR20190096748A (en) electronic device and method for correcting image using external electronic device
CN109981989B (en) Method and device for rendering image, electronic equipment and computer readable storage medium
US10764509B2 (en) Image processing device, image processing method, and program
CN106303366B (en) Video coding method and device based on regional classification coding
WO2017101570A1 (en) Photo processing method and processing system
CN109961453B (en) Image processing method, device and equipment
WO2021102928A1 (en) Image processing method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15871951

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24.10.2017)

122 Ep: pct application non-entry in european phase

Ref document number: 15871951

Country of ref document: EP

Kind code of ref document: A1