WO2021012599A1 - 图像调整方法、装置和计算机设备 - Google Patents

图像调整方法、装置和计算机设备 Download PDF

Info

Publication number
WO2021012599A1
WO2021012599A1 PCT/CN2019/126719 CN2019126719W WO2021012599A1 WO 2021012599 A1 WO2021012599 A1 WO 2021012599A1 CN 2019126719 W CN2019126719 W CN 2019126719W WO 2021012599 A1 WO2021012599 A1 WO 2021012599A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
feature points
deformed
triangle
facial
Prior art date
Application number
PCT/CN2019/126719
Other languages
English (en)
French (fr)
Inventor
邹超洋
Original Assignee
广州视源电子科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州视源电子科技股份有限公司 filed Critical 广州视源电子科技股份有限公司
Publication of WO2021012599A1 publication Critical patent/WO2021012599A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the field of image processing technology, for example, to an image adjustment method, device, and computer equipment.
  • portrait beautification technology With the development and promotion of portrait beautification technology, users are no longer satisfied with the basic portrait beautification technology, and users are more and more interested in face deformation technology.
  • the most typical application functions of portrait beautification technology are portrait big eyes and face thinning, portrait big head and other partial deformation functions.
  • the current portrait beautification technology usually performs overall operations on the entire image. Although the portrait beautification effect reaches the standard, the calculation is very large, it takes up a lot of computing resources, and the processing speed is slow; and the background is not considered when beautifying the local area of the face Maintain, the background at the edge of the area will be significantly deformed.
  • the purpose of the embodiments of the present disclosure is to provide an image adjustment method, device, and computer equipment, which has the advantages of local deformation and less calculation, and has a good effect of maintaining non-deformed areas and image background.
  • an embodiment of the present disclosure provides an image adjustment method, including the following steps:
  • the corresponding original triangle image and the deformed triangle image use the same three facial feature points before and after the coordinate adjustment as the vertices; if they are not the same, check all
  • the pixels in the original triangle image are subjected to affine transformation to obtain the transformed coordinates of the pixels in the deformed triangle image;
  • the facial feature point set includes facial contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
  • the step of obtaining the facial feature point set further includes: selecting a plurality of the facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area.
  • the step of selecting a plurality of the facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area includes:
  • a highest point and a lowest point are selected on the left and right sides of the facial contour feature points, and the highest point and the lowest point selected on the left and right sides are respectively weighted and calculated to obtain the cheek feature points located in the cheek regions on both sides .
  • the image adjustment method of the embodiment of the present disclosure obtains the cheek feature points in the middle of the human cheek by performing coordinate weighting operation from the facial contour feature points, thereby triangulating the human cheek to obtain more fine triangles, thereby correcting
  • the adjustment of cheeks is smoother, especially when beautifying portraits with thin and fat faces, making the effect of thin or fat faces more natural.
  • the adjusting the coordinates of part of the face feature points of the face feature point set includes:
  • i represents the number of the face feature point
  • fx and fy represent the displacement of the abscissa and ordinate of the face feature point
  • a 1 , b 1 , c 1 , d 1 , e 1 and a 2 , b 2 , c 2 , d 2 , e 2 , f, and g are constants
  • w 1 and w 2 are the frequencies of the sine and cosine components.
  • the adjustment formula of the facial feature points provided by the image adjustment method of the embodiment of the present disclosure is obtained through equation fitting based on multiple experiments and statistical results.
  • the image is deformed through the adjustment formula, and the smoothest and natural image can be obtained.
  • the triangulating the input image according to the facial feature point set and triangulating the transition image according to the facial feature point set after coordinate adjustment includes the following step:
  • the feature points of all human faces and the four vertices of the input image or transition image are connected into several triangles, and the image area is divided.
  • the step of performing affine transformation on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image includes:
  • the filling the pixel values of the deformed triangle image according to the transformed coordinates includes performing bilinear interpolation on the deformed triangle image according to the transformed coordinates.
  • an image adjustment device including:
  • the face detection module is used to perform face detection on the input image and obtain a set of facial feature points
  • the first triangulation module is configured to triangulate the input image according to the set of facial feature points to obtain multiple original triangular images of the input image;
  • a coordinate adjustment module which adjusts the coordinates of part of the facial feature points in the facial feature point set to obtain a transition image
  • the second triangulation module is configured to triangulate the transition image according to the facial feature point set after coordinate adjustment to obtain a plurality of deformed triangle images related to the transition image;
  • the coordinate transformation module is used to determine whether the vertex coordinates of the corresponding original triangle image and the deformed triangle image are the same, wherein the corresponding original triangle image and the deformed triangle image respectively take the same three facial feature points before and after coordinate adjustment as the vertices; if If they are not the same, perform affine transformation on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image;
  • the pixel value filling module is used to fill the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
  • the embodiments of the present disclosure also provide a computer device, the computer device including:
  • the memory is used to store one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image adjustment method according to the first aspect of the present disclosure.
  • the input image and the transition image are divided into a plurality of triangles by triangulation by accurately positioning the feature points of the face, aiming at the vertex coordinates of the corresponding original triangle image and the deformed triangle image
  • the changed pixels can achieve smooth local deformation of the face by means of affine transformation and filling pixel values.
  • Fig. 1 is a flowchart of an image adjustment method of the present disclosure shown in an exemplary embodiment
  • Fig. 2 is a face feature point diagram of the image adjustment method of the present disclosure shown in an exemplary embodiment
  • FIG. 3 is a diagram showing the displacement change of facial contour feature points of the image adjustment method of the present disclosure shown in an exemplary embodiment
  • FIG. 4 is a schematic diagram of triangulation of the image adjustment method of the present disclosure shown in an exemplary embodiment
  • FIG. 5 is a flowchart of performing affine transformation in the image adjustment method of the present disclosure shown in an exemplary embodiment
  • Fig. 6 is an effect diagram obtained by using the image adjustment method of the present disclosure in an exemplary embodiment
  • Fig. 7 is a structural block diagram of an image adjusting device of the present disclosure shown in an exemplary embodiment
  • Fig. 8 is a structural block diagram of a computer device of the present disclosure shown in an exemplary embodiment.
  • first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information, and similarly, the second information may also be referred to as first information.
  • word “if” as used herein can be interpreted as "when” or “when” or "in response to determination”.
  • the image adjustment method of this embodiment includes the following steps:
  • Step S101 Perform face detection on the input image to obtain a set of face feature points.
  • Step S102 Triangulate the input image according to the set of facial feature points to obtain multiple original triangular images of the input image.
  • Step S103 Perform coordinate adjustment on part of the face feature points of the face feature point set to obtain a transition image.
  • Step S104 Triangulate the transition image according to the facial feature point set after coordinate adjustment, to obtain a plurality of deformed triangle images about the transition image;
  • Step S105 Determine whether the vertex coordinates of the corresponding original triangle image and the deformed triangle image are the same, wherein the corresponding original triangle image and the deformed triangle image respectively take the same three facial feature points before and after coordinate adjustment as the vertices; if they are not the same, Then, affine transformation is performed on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image.
  • Step S106 Fill the pixel values of the deformed triangle image according to the transformed coordinates to obtain a target image.
  • the face feature point set described in the embodiment of the present disclosure includes multiple face feature points located in different parts of the face.
  • the face feature point set includes face contour feature points, eyebrow feature points, Eye feature points, nose feature points, and mouth feature points. Selecting the above-mentioned face feature points can characterize most of the features of the face and effectively distinguish different faces.
  • the face feature point set can be used to detect and recognize the input image through various current face detection algorithms. In the embodiment of the present disclosure, face detection is performed on the input image through the Dlib face detection algorithm to obtain 68 Personal face feature points.
  • the cheek feature points in the middle of the cheeks on the left and right sides of the nose (commonly known as the position of the face), leading to When the face feature point set is triangulated.
  • a triangular area with a large area is formed in the middle of the cheek, so that the middle area of the cheek cannot be effectively adjusted.
  • only the contour of the face is adjusted in the processing algorithm of a thin or fat face. After a thin or fat face, the middle of the cheek (face ) No change, the image after thin or fat face is not smooth and natural enough, and the image distortion effect is poor.
  • the step of obtaining the facial feature point set further includes: selecting several facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area.
  • the reason for selecting the facial contour feature points is that the distribution of the facial contour feature points determines the information such as the shape and size of the cheek, and the cheek feature points obtained by the weighted calculation of the facial contour feature point coordinates can more accurately reflect the characteristics of the cheek.
  • the selection of the number and position of the facial contour feature points can be determined according to experience and/or calculation requirements.
  • the step of selecting a plurality of the facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area includes:
  • a highest point and a lowest point are selected on the left and right sides of the facial contour feature points, and the highest point and the lowest point selected on the left and right sides are respectively weighted and calculated to obtain the cheek feature points located in the cheek regions on both sides .
  • the highest point and the lowest point are selected for coordinate weighting calculation, which can more accurately locate the middle position of the cheek, so that the coordinate adjustment of the obtained cheek feature points has a smoother and less abrupt overall adjustment effect on the cheek.
  • the cheek feature points are two points; the face contour feature point located at the highest point of the left cheek and the face contour feature point located at the chin are selected for coordinate weighting calculation to obtain the cheek feature point located on the left cheek ; Select the facial contour feature points located at the highest point of the right cheek and the facial contour feature points located at the chin for coordinate weighting calculation to obtain the cheek feature points located at the right cheek.
  • the selected facial contour feature points at the chin are the same point.
  • step S101 on the basis of obtaining 68 facial feature points using the Dlib face detection algorithm, the facial feature points in the facial feature point set are first performed according to preset rules Number, and then select the facial contour feature point at the highest point of the left cheek (numbered 0), the facial contour feature point located at the highest point of the right cheek (numbered 16), and the facial contour feature point at the chin (numbered 8) Perform coordinate weighting calculation:
  • [0].x and [0].y are the coordinates of the facial contour feature points numbered 0, and [8].x and [8].y are the coordinates of the facial contour feature points numbered 8. , [16].x and [16].y are the coordinates of the facial contour feature points numbered 16, and x and y are the coordinates of the cheek feature points.
  • 0.4 and 0.6 are the weight ratios. The weights are determined after many tests. Using the weight ratios of 0.4 and 0.6, the adjustment of the cheeks will not cause distortion, and the adjusted image will be natural.
  • the displacement of the coordinate adjustment is determined by the difference between the fitted sine and cosine components of different harmonics. Obtained by stacking.
  • coordinate adjustment is performed on part of the face feature points of the face feature point set according to the following formula:
  • i represents the number of the facial feature points.
  • the face contour points are numbered from left to right, and then the eyebrow feature points and eye feature points are sequentially numbered.
  • Nose feature points and mouth feature points are numbered.
  • fx and fy respectively represent the displacement of the abscissa and ordinate of the facial feature points, a 1 , b 1 , c 1 , d 1 , e 1 and a 2 , b 2 , c 2 , d 2 , e 2 , f, g is a constant, w 1 and w 2 are the frequencies of the sine and cosine components.
  • the method for obtaining the displacement amount of the coordinate adjustment of the facial feature points provided by the image adjustment method of the embodiment of the present disclosure is obtained through equation fitting based on multiple experiments and statistical results, and the image is deformed according to the displacement amount calculated by the displacement amount obtaining method , Can get the smoothest and natural image.
  • a reference design can be:
  • the above adjustment formula controls the facial contour feature points in the X direction and Y direction to change with the face feature point numbers as shown in the figure. If the values of fx and fy are positive, the face-lifting operation is performed on the face, and the fat face transformation is to take the values of fx and fy negative.
  • the adjustment strategy of the facial feature points of the eyes is the same as that of the facial contour feature points.
  • the coordinate displacement of the facial feature points of the eye is calculated proportionally by the coordinate displacement of the facial contour feature points.
  • the input image is triangulated according to the face feature point set
  • the transition image is triangulated according to the face feature point set after coordinate adjustment.
  • Triangulation includes the following steps:
  • the triangles in the divided image area do not contain any points in the facial feature point set.
  • All the graphics obtained by subdivision in the image area are triangles, and the collection of all triangles is the convex hull of the facial feature point set.
  • the convex quadrilateral formed by any two adjacent triangles obtained by splitting has the following characteristics: after the diagonals are interchanged, the minimum angle of the two internal angles no longer increases.
  • the image adjustment method of the embodiment of the present disclosure uses Delaunay triangulation to perform triangulation. Since the four vertices of the input image or transition image are included, for the face pixels between the facial feature points and the four vertices The points and the image background can also be adjusted, which helps make the adjusted image smoother and more natural.
  • the step of performing affine transformation on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image includes:
  • Step S501 Calculate the affine transformation matrix between the corresponding original triangle image and the deformed triangle image according to the vertex coordinates of the corresponding original triangle image and the deformed triangle image;
  • Step S502 Perform affine transformation on the pixel points in the original triangle image according to the affine transformation matrix to obtain the transformed coordinates of the pixel points in the deformed triangle image.
  • the filling of the pixel values of the deformed triangle image according to the transformed coordinates includes performing bilinear interpolation on the deformed triangle image according to the transformed coordinates. Since the affine transformation involves transformations in the X direction and the Y direction, on this basis, bilinear interpolation is more appropriate. Exemplarily, the result of performing big-eyed face-lifting or small-eyed fat face on the portrait through steps S101-106 is shown in FIG. 6.
  • the deformation control amount of cheeks, eyes, nose and other areas can be adaptively obtained.
  • the amount of calculation is small, only the adjusted triangular area is transformed, and the processing speed is fast.
  • Multiple local positions can be adjusted simultaneously, without mutual influence and interference. For example, processing the eyes, nose, cheeks and other areas can be carried out at the same time, and the processing effect of other areas will not be affected by the adjustment of one area.
  • An image adjustment device including:
  • the face detection module 701 is used to perform face detection on an input image and obtain a set of facial feature points
  • the first triangulation module 702 is configured to triangulate the input image according to the facial feature point set to obtain multiple original triangular images of the input image;
  • the coordinate adjustment module 703 performs coordinate adjustment on part of the face feature points of the face feature point set to obtain a transition image
  • the second triangulation module 704 is configured to triangulate the transition image according to the facial feature point set after coordinate adjustment to obtain a plurality of deformed triangle images related to the transition image;
  • the coordinate transformation module 705 is configured to determine whether the vertex coordinates of the corresponding original triangle image and the deformed triangle image are the same, wherein the corresponding original triangle image and the deformed triangle image respectively take the same three facial feature points before and after coordinate adjustment as the vertices; If they are not the same, perform affine transformation on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image;
  • the pixel value filling module 706 is configured to fill the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
  • the facial feature point set includes facial contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
  • the face detection module 701 further includes:
  • Auxiliary feature point generating module 7011 used to select a number of the facial contour feature points for coordinate weighting calculation to obtain cheek feature points located in the cheek area.
  • the cheek feature points generated by the auxiliary feature point generating module 7011 are two points, which include:
  • the left cheek feature point generation module is used to select the facial contour feature points located at the highest point of the left cheek and the facial contour feature points located at the chin for coordinate weighting calculation to obtain the cheek feature points located on the left cheek;
  • the right cheek feature point generation module is also used to select the facial contour feature points located at the highest point of the right cheek and the facial contour feature points located at the chin for coordinate weighting calculation to obtain the cheek feature points located on the right cheek.
  • the displacement of the coordinate adjustment is obtained by the superposition of fitted sine components and cosine components of different harmonics.
  • the coordinate adjustment module 703 also includes a displacement calculation module 7031: configured to adjust the coordinates of part of the face feature points of the face feature point set according to the following formula:
  • i represents the number of the face feature point
  • fx and fy represent the displacement of the abscissa and ordinate of the face feature point
  • a 1 , b 1 , c 1 , d 1 , e 1 and a 2 , b 2 , c 2 , d 2 , e 2 , f, and g are constants
  • w 1 and w 2 are the frequencies of the sine and cosine components.
  • the first triangulation module 702 and the second triangulation module 704 are used to connect all facial feature points and the four vertices of the input image or transition image into several Triangle, which divides the image area; among them, two adjacent triangles formed on the diagonal of the convex quadrilateral, after mutual exchange, the smallest angle of the two internal angles no longer increases.
  • the coordinate transformation module 705 includes
  • Affine transformation matrix calculation module 7051 used to calculate the affine transformation matrix between the corresponding original triangle image and the deformed triangle image according to the vertex coordinates of the corresponding original triangle image and the deformed triangle image;
  • the pixel point affine transformation module 7052 performs affine transformation on the pixel points in the original triangle image according to the affine transformation matrix to obtain the transformed coordinates of the pixel points in the deformed triangle image.
  • the pixel value filling module 706 is configured to perform bilinear interpolation on the deformed triangle image according to the transformed coordinates.
  • the present disclosure also provides a computer device, including:
  • a memory for storing a computer program executable by the processor
  • Fig. 8 is a structural block diagram of a computer device according to an exemplary embodiment of the present disclosure.
  • the computer device may be a computer, a mobile phone, a tablet computer, an interactive smart tablet, a PDA (Personal Digital Assistant, personal digital assistant), an e-book reader, a multimedia player, and the like.
  • the computer device is an interactive smart tablet as an example for description.
  • the memory 801 can be used to store software programs, computer-executable programs, and modules, such as the image adjustment method program described in any embodiment of the present disclosure, and the image adjustment method described in any embodiment of the present disclosure Corresponding program instructions/modules (for example, the face detection module 701 in the image adjustment device, the first triangulation module 702, the coordinate adjustment module 703, the second triangulation module 704, the coordinate transformation module 705, the pixel value filling module 706 etc.).
  • the memory 801 may mainly include a program storage area and a data storage area.
  • the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the device, and the like.
  • the memory 801 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the memory 801 may include a memory remotely provided with respect to the processor 800, and these remote memories may be connected to the device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the processor 800 executes various functional applications and data processing of the device by running software programs, instructions, and modules stored in the memory 801, that is, realizes the above-mentioned image adjustment method.
  • the processor 800 executes one or more programs stored in the memory 801, the following operations are implemented: input an image for face detection, and obtain a set of facial feature points; according to the facial feature points The set triangulates the input image to obtain multiple original triangle images about the input image; adjusts the coordinates of part of the facial feature points of the facial feature point set to obtain a transition image; after adjusting according to the coordinates The face feature point set triangulates the transition image to obtain multiple deformed triangle images about the transition image; it is determined whether the vertex coordinates of the corresponding original triangle image and the deformed triangle image are the same, where the corresponding The original triangle image and the deformed triangle image of, respectively take the same three facial feature points before and after the coordinate adjustment as the vertices; if they are not the same, perform affine transformation on the pixels in the original triangle image to obtain the pixels in the deformed triangle The transformed coordinates in the image; the pixel values of the deformed triangular image are filled according to the transformed coordinates to obtain the target image.
  • the facial feature point set includes facial contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
  • the acquiring the facial feature point set further includes: selecting a plurality of the facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area.
  • the selecting a plurality of the facial contour feature points to perform coordinate weighting calculation to obtain the cheek feature points located in the cheek area includes:
  • the feature points of the cheek are two points;
  • the coordinate adjustment is performed on part of the facial feature points of the facial feature point set, and the displacement of the coordinate adjustment is obtained by the superposition of the fitted sine and cosine components of different harmonics, include:
  • the coordinates of some facial feature points in the facial feature point set are adjusted:
  • i represents the number of the face feature point
  • fx and fy represent the displacement of the abscissa and ordinate of the face feature point
  • a 1 , b 1 , c 1 , d 1 , e 1 and a 2 , b 2 , c 2 , d 2 , e 2 , f, and g are constants
  • w 1 and w 2 are the frequencies of the sine and cosine components.
  • the triangulating the input image according to the face feature point set and triangulating the transition image according to the face feature point set after coordinate adjustment include:
  • the performing affine transformation on the pixels in the original triangle image to obtain the transformed coordinates of the pixels in the deformed triangle image includes:
  • the filling the pixel values of the deformed triangle image according to the transformed coordinates includes performing bilinear interpolation on the deformed triangle image according to the transformed coordinates.
  • the computer device provided above can be used to execute the image adjustment method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
  • the implementation process of the functions and roles of the various components in the above-mentioned device refer to the implementation process of the corresponding steps in the above-mentioned method for details, which will not be repeated here.
  • the relevant part can refer to the part of the description of the method embodiment.
  • the device embodiments described above are merely illustrative, and the components described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the present disclosure. Those of ordinary skill in the art can understand and implement it without creative work.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种图像调整方法、装置和计算机设备，该方法包括如下步骤：对输入图像进行人脸检测，获取人脸特征点集（S101）；根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像（S102）；对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像（S103）；根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像（S104）；判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。该方法具有局部变形，计算量少的优点，对非变形区域和图像背景具有很好的保持效果。

Description

图像调整方法、装置和计算机设备
本申请要求在2019年7月24日提交中国专利局、申请号为201910670790.7的中国专利申请的优先权，以上申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，例如涉及一种图像调整方法、装置和计算机设备。
背景技术
随着人像美化技术的发展与推广，用户已不满足于基础的人像美化技术，用户对人脸变形技术的兴趣愈发浓厚。人像美化技术中最为典型的应用功能为人像大眼瘦脸、人像大头和其他局部变形功能。
目前的人像美化技术通常是对整幅图像进行整体操作，虽然人像美化效果达标，但是计算量非常大，占用很多计算资源，处理速度慢；而且在进行人脸局部区域美化的时候没有考虑背景的维持，在区域边缘的背景会出现明显的变形。
发明内容
基于此，本公开实施例的目的在于，提供一种图像调整方法、装置和计算机设备，其具有局部变形，计算量少的优点，并且对非变形区域和图像背景具有很好的保持效果。
第一方面，本公开实施例提供一种图像调整方法，包括如下步骤：
对输入图像进行人脸检测，获取人脸特征点集；
根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；
对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；
根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；
判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；
根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
在一实施方式中，所述人脸特征点集包括人脸轮廓特征点、眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点。
在一实施方式中，所述获取人脸特征点集的步骤还包括：选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点。
在一实施方式中，所述选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点的步骤，包括：
在所述人脸轮廓特征点的左右两边各选取一个最高点和一个最低点，分别对左右两边选取的所述最高点和所述最低点进行坐标加权计算，获取位于两边脸颊区域的脸颊特征点。
本公开实施例的图像调整方法通过由人脸轮廓特征点进行坐标加权运算获得在人脸脸颊中部出的脸颊特征点，由此在人脸脸颊处三角剖分获得更多细小的三角形，从而对脸颊的调整更加平滑，特别是进行瘦脸和胖脸的人像美化时，使得瘦脸或胖脸的效果更自然。
在一实施方式中，所述对所述人脸特征点集的部分人脸特征点进行坐标调整，包括：
按照如下公式对所述人脸特征点集的部分人脸特征点进行坐标调整：
fx＝a 1+b 1cos(w 1i)+c 1sin(w 1i)+d 1cos(2w 1i)+e 1sin(2w 1i)
fy＝a 2+b 2cos(w 2i)+c 2sin(w 2i)+d 2cos(2w 2i)+e 2sin(2w 2i)+f cos(3w 2i)+g sin(3w 2i)
上述公式中，i表示人脸特征点的编号，fx、fy分别表示人脸特征点的横坐标和纵坐标的位移量，a 1、b 1、c 1、d 1、e 1和a 2、b 2、c 2、d 2、e 2、f、g为常数，w 1和w 2为正弦分量和余弦分量的频率。
本公开实施例的图像调整方法提供的人脸特征点的调整公式是基于多次试验和统计结果通过方程拟合获得的，通过该调整公式进行图像变形，能获得最为平滑自然的图像。
在一实施方式中，所述根据所述人脸特征点集对所述输入图像进行三角剖分和根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，包括如下步骤：
将所有人脸特征点以及所述输入图像或过渡图像的四个顶点连接为若干个三角形，对图像区域进行剖分。
在一实施方式中，所述对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标的步骤，包括：
根据对应的原始三角形图像和变形三角形图像的顶点坐标计算对应的原始三角形图像和变形三角形图像之间的仿射变换矩阵；
根据所述仿射变换矩阵对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。
在一实施方式中，所述根据所述变换坐标填充所述变形三角形图像的像素值，包括根据所述变换坐标对所述变形三角形图像进行双线性插值。
第二方面，本公开实施例提供了一种图像调整装置，所述装置包括：
人脸检测模块，用于对输入图像进行人脸检测，获取人脸特征点集；
第一三角剖分模块，用于根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；
坐标调整模块，对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；
第二三角剖分模块，用于根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；
坐标变换模块，用于判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；
像素值填充模块，用于根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
第三方面，本公开实施例还提供了一种计算机设备，所述计算机设备包括：
存储器以及处理器；
所述存储器，用于存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本公开第一方面所述的图像调整方法。
本公开实施例所述的图像调整方法，通过对人脸特征点进行精准定位，通过三角剖分将输入图像和过渡图像划分为多个三角形，针对对应的原始三角形图像和变形三角形图像的顶点坐标发生了变化的像素点，通过仿射变换与填充像素值的方式，可实现平滑的人脸局部变形。由于仅调整顶点坐标发生了变化的三角形区域的像素点，而不是对整幅图像进行调整，计算量小，处理速度快，并且对非变形区域和图像背景具有很好的保持效果；此外，还可以同时对输入图像的多个局部位置进行变形，只要同时变形的多个局部位置涉及的人脸特征点不相同，就不会互相影响和干扰。
为了更好地理解和实施，下面结合附图详细说明本公开。
附图说明
图1为在一个示例性实施例中示出的本公开图像调整方法的流程图；
图2为在一个示例性实施例中示出的本公开图像调整方法的人脸特征点图；
图3为在一个示例性实施例中示出的本公开图像调整方法的人脸轮廓特征点位移变化图；
图4为在一个示例性实施例中示出的本公开图像调整方法的三角剖分示意图；
图5为在一个示例性实施例中示出的本公开图像调整方法的进行仿射变换的流程图；
图6为在一个示例性实施例中利用本公开图像调整方法获得的效果图；
图7为在一个示例性实施例中示出的本公开图像调整装置的结构框图；
图8为在一个示例性实施例中示出的本公开计算机设备的结构框图。
具体实施方式
下面结合附图和实施例对本公开作详细说明。可以理解的是，此处所描述的实施例仅仅用于解释本公开，而非对本公开的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本公开相关的部分而非全部内容。
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
如图1所示，在一个示例性的实施例中，本实施例的图像调整方法包括如下步骤：
步骤S101：对输入图像进行人脸检测，获取人脸特征点集。
步骤S102：根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像。
步骤S103：对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像。
步骤S104：根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；。
步骤S105：判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角 形图像中的变换坐标。
步骤S106：根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
本公开实施例所述的人脸特征点集包括了位于人脸不同部位的多个人脸特征点，在一实施方式中，所述人脸特征点集包括人脸轮廓特征点、眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点。选取上述的人脸特征点能够表征人脸的大部分特征和有效区分不同的人脸。所述人脸特征点集可以通过目前的各种人脸检测算法对输入图像进行检测和识别，在本公开实施例中，通过Dlib人脸检测算法对所述输入图像进行人脸检测，获得68个人脸特征点。
在一个示例性实施例中，考虑到目前的人脸检测算法识别的人脸特征点几乎没有涉及到眼睛以下，鼻子左右两侧的脸颊中部(俗称脸蛋位置)的脸颊特征点，导致在根据人脸特征点集进行三角剖分的时候。在脸颊中部的形成面积较大的三角形区域，从而脸颊中部区域得不到有效的调整，例如在瘦脸或胖脸的处理算法中只调整了人脸轮廓，瘦脸或胖脸后，脸颊中部(脸蛋)不改变，瘦脸或胖脸后的图像不够平滑自然，图像变形效果差。为了克服上述缺陷，在本公开实施例中，所述获取人脸特征点集的步骤还包括：选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点。选取人脸轮廓特征点的原因在于：人脸轮廓特征点的分布决定了脸颊的形状和大小等信息，由人脸轮廓特征点坐标加权计算获得的脸颊特征点，能更准确反映脸颊的特点。人脸轮廓特征点个数和位置的选取可根据经验和/或计算量要求决定。
在一实施方式中，所述选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点的步骤，包括：
在所述人脸轮廓特征点的左右两边各选取一个最高点和一个最低点，分别对左右两边选取的所述最高点和所述最低点进行坐标加权计算，获取位于两边脸颊区域的脸颊特征点。
选取最高点和最低点进行坐标加权计算，能够较准确地定位脸颊中部位置，使得获取到的脸颊特征点的坐标调整对脸颊的整体调整效果更平滑不突兀。
在一实施方式中，所述脸颊特征点为两点；选取位于左脸颊最高点的人脸轮廓特征点和位于下巴处的人脸轮廓特征点进行坐标加权计算，获取位于左脸颊的脸颊特征点；选取位于右脸颊最高点的人脸轮廓特征点和位于下巴处的人脸轮廓特征点进行坐标加权计算，获取位于右脸颊的脸颊特征点。在进行左、右脸颊的脸颊特征点的获取时，选取的位于下巴处的人脸轮廓特征点为同一点。
在一个实施例中，请参阅图2，在步骤S101采用Dlib人脸检测算法获得68个人脸特征点的基础上，先对所述人脸特征点集的人脸特征点按照预设的规则进行编号，再选取位于左脸颊最高点的人脸轮廓特征点(编号为0)、位于右脸颊最高点的人脸轮廓特征点(编号为16) 和位于下巴处的人脸轮廓特征点(编号为8)进行坐标加权运算：
左脸颊的脸颊特征点：
Figure PCTCN2019126719-appb-000001
右脸颊的脸颊特征点：
Figure PCTCN2019126719-appb-000002
式中，[0].x和[0].y为编号为0的人脸轮廓特征点的坐标，[8].x和[8].y为编号为8的人脸轮廓特征点的坐标，[16].x和[16].y为编号为16的人脸轮廓特征点的坐标，x和y为脸颊特征点的坐标。0.4和0.6为权重比例，该权重是经过多次测验确定，使用0.4和0.6的权重比例，对于脸颊的调整不会造成扭曲，并且调整后的图像自然。
在一个示例性实施例中，所述对所述人脸特征点集的部分人脸特征点进行坐标调整的步骤中，坐标调整的位移量由拟合的不同谐波的正弦分量和余弦分量的叠加获得。
在一实施方式中，按照如下公式对所述人脸特征点集的部分人脸特征点进行坐标调整：
fx＝a 1+b 1cos(w 1i)+c 1sin(w 1i)+d 1cos(2w 1i)+e 1sin(2w 1i)
fy＝a 2+b 2cos(w 2i)+c 2sin(w 2i)+d 2cos(2w 2i)+e 2sin(2w 2i)+f cos(3w 2i)+g sin(3w 2i)
上述公式中，i表示人脸特征点的编号，对于人脸特征点的编号可以参阅图2的方式，先对人脸轮廓点从左到右进行编号，再依次对眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点进行编号。fx、fy分别表示人脸特征点横坐标和纵坐标的位移量，a 1、b 1、c 1、d 1、e 1和a 2、b 2、c 2、d 2、e 2、f、g为常数，w 1和w 2为正弦分量和余弦分量的频率。
本公开实施例的图像调整方法提供的人脸特征点的坐标调整的位移量获得方法是基于多次试验和统计结果通过方程拟合获得的，根据该位移量获得方法计算的位移量进行图像变形，能获得最为平滑自然的图像。
通过人像美化试验，基于统计结果进行方程拟合，获得a 1、b 1、c 1、d 1、e 1和a 2、b 2、c 2、d 2、e 2、f、g，w 1和w 2的常数值，一个参考设计可以是：
f x＝-0.4856+1.725cos(w 1i)+2.111sin(w 1i)-0.8245cos(2w 1i)-0.152sin(2w 1i)
f y＝-2.348+0.7059cos(w 2i)-0.2756sin(w 2i)+0.9887cos(2w 2i)-0.1879sin(2w 2i)
-0.3125cos(3w 2i)+0.5605sin(3w 2i)
其中，w 1＝0.3181，w 2＝0.3612
以人像大眼瘦脸或小眼胖脸为例，请参阅图3，由上述调整公式控制人脸轮廓特征点在X 方向、Y方向随人脸特征点编号变化如图所示。fx、fy的值为正，则对人脸执行瘦脸变形操作，胖脸变换则是将fx、fy的值取负。眼睛部位人脸特征点的调整策略与人脸轮廓特征点一样，眼睛部位人脸特征点的坐标位移量由人脸轮廓特征点的坐标位移量依比例计算得到。
在一个示例性实施例中，请参阅图4，所述根据所述人脸特征点集对所述输入图像进行三角剖分和根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，包括如下步骤：
将所有人脸特征点以及所述输入图像或过渡图像的四个顶点连接为若干个三角形，对图像区域进行剖分；在进行三角剖分时，按照如下特性进行三角形剖分：
a.除了顶点，剖分的图像区域中的三角形不包含人脸特征点集中的任何点。
b.没有相交边。
c.图像区域中剖分获得所有的图形都是三角形，且所有三角形的合集是人脸特征点集的凸包。
与此同时，剖分获得的任意两个相邻的三角形构成的凸四边形具有如下特点：其对角线互换后，两个内角的最小角不再增大。
本公开实施例的图像调整方法进行三角剖分采用的是Delaunay三角剖分，由于包括了所述输入图像或过渡图像的四个顶点，对于人脸特征点与四个顶点之间的人脸像素点和图像背景也能进行调整，有利于使得调整后的图像更平滑自然。
对于输入图像的每一像素点来说，其与一个原始三角形图像对应，位于由特定的三个人脸特征点或图像顶点构成的原始三角形内部，而该原始三角形又通过坐标调整后的上述特定的三个人脸特征点与变形三角形相对应，在通过步骤S103将部分人脸特征点移动到目标位置后，由于部分人脸特征点移动而使原始三角形图像发生变化成为变形三角形图像，再将原始三角形内部的像素点按照相同的变形方式变换至变形三角形图像中。该种相同的变形方式可以参照对应的原始三角形图像和变形三角形图像的顶点坐标之间的仿射变换。在一个示例性实施例中，请参阅图5，所述对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标的步骤，包括：
步骤S501：根据对应的原始三角形图像和变形三角形图像的顶点坐标计算对应的原始三角形图像和变形三角形图像之间的仿射变换矩阵；
步骤S502：根据所述仿射变换矩阵对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。
在进行了像素点的坐标变换之后，还需进行像素值的填充。在一个示例性实施例中，所 述根据所述变换坐标填充所述变形三角形图像的像素值，包括根据所述变换坐标对所述变形三角形图像进行双线性插值。由于仿射变换涉及到X方向和Y方向两个方向的变换，在此基础上，进行双线性插值是更为适宜的。示例性地，通过步骤S101～106对人像进行大眼瘦脸或小眼胖脸的结果如图6所示。
本公开的有益效果包括：
1、针对不同尺寸人脸，可自适应获得脸颊、眼睛、鼻子等区域的形变控制量。
2、计算量小，仅对调整的三角形区域进行变换，处理速度快。
3、多个局部位置可以同时进行调整，不会互相影响和干扰。如处理眼睛、鼻子、脸颊等区域，可同时进行，不会因一个区域调整而影响其他区域的处理效果。
一种图像调整装置，包括：
人脸检测模块701，用于对输入图像进行人脸检测，获取人脸特征点集
第一三角剖分模块702，用于根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；
坐标调整模块703，对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；
第二三角剖分模块704，用于根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；
坐标变换模块705，用于判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；
像素值填充模块706，用于根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
在一个示例性实施例中，所述人脸特征点集包括人脸轮廓特征点、眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点。
在一个示例性实施例中，所述人脸检测模块701还包括：
辅助特征点生成模块7011：用于选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点。
在一个示例性实施例中，所述辅助特征点生成模块7011生成的脸颊特征点为两点，其包括：
左脸颊特征点生成模块，用于选取位于左脸颊最高点的人脸轮廓特征点和位于下巴处的 人脸轮廓特征点进行坐标加权计算，获取位于左脸颊的脸颊特征点；
右脸颊特征点生成模块，还用于选取位于右脸颊最高点的人脸轮廓特征点和位于下巴处的人脸轮廓特征点进行坐标加权计算，获取位于右脸颊的脸颊特征点。
在一个示例性实施例中，所述坐标调整模块703中，坐标调整的位移量由拟合的不同谐波的正弦分量和余弦分量的叠加获得。所述坐标调整模块703还包括位移量计算模块7031：用于按照如下公式对所述人脸特征点集的部分人脸特征点进行坐标调整：
fx＝a 1+b 1cos(w 1i)+c 1sin(w 1i)+d 1cos(2w 1i)+e 1sin(2w 1i)
fy＝a 2+b 2cos(w 2i)+c 2sin(w 2i)+d 2cos(2w 2i)+e 2sin(2w 2i)+f cos(3w 2i)+g sin(3w 2i)
上述公式中，i表示人脸特征点的编号，fx、fy分别表示人脸特征点的横坐标和纵坐标的位移量，a 1、b 1、c 1、d 1、e 1和a 2、b 2、c 2、d 2、e 2、f、g为常数，w 1和w 2为正弦分量和余弦分量的频率。
在一个示例性实施例中，所述第一三角剖分模块702和第二三角剖分模块704，用于将所有人脸特征点以及所述输入图像或过渡图像的四个顶点连接为若干个三角形，对图像区域进行剖分；其中，在形成的两个相邻的三角形构成凸四边形的对角线，在相互交换后，两个内角的最小角不再增大。
在一个示例性实施例中，所述坐标变换模块705，包括
仿射变换矩阵计算模块7051：用于根据对应的原始三角形图像和变形三角形图像的顶点坐标计算对应的原始三角形图像和变形三角形图像之间的仿射变换矩阵；
像素点仿射变换模块7052：根据所述仿射变换矩阵对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。
在一个示例性实施例中，所述像素值填充模块706，用于根据所述变换坐标对所述变形三角形图像进行双线性插值。
本公开还提供一种计算机设备，包括：
处理器；
存储器，用于存储可由所述处理器执行的计算机程序；
其中，所述处理器执行所述程序时实现上述任一实施例所记载的图像调整方法。
如图8所示，图8是本公开根据一示例性实施例示出的一种计算机设备的结构框图。
实施例中，计算机设备可以是计算机、手机、平板电脑、交互式智能平板、PDA(Personal Digital Assistant，个人数字助理)、电子书阅读器、多媒体播放器等。实施例中，以计算机设备为交互智能平板为例，进行描述。
存储器801作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本公开任意实施例所述的图像调整方法程序，以及本公开任意实施例所述的图像调整方法对应的程序指令/模块(例如，图像调整装置中的人脸检测模块701，第一三角剖分模块702，坐标调整模块703，第二三角剖分模块704，坐标变换模块705，像素值填充模块706等)。存储器801可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据设备的使用所创建的数据等。此外，存储器801可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器801可包括相对于处理器800远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
处理器800通过运行存储在存储器801中的软件程序、指令以及模块，从而执行设备的各种功能应用以及数据处理，即实现上述的图像调整方法。
在一个示例性的实施例中，处理器800执行存储器801中存储的一个或多个程序时，实现如下操作：输入图像进行人脸检测，获取人脸特征点集；根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
在上述实施例的基础上，所述人脸特征点集包括人脸轮廓特征点、眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点。
在上述实施例的基础上，所述获取人脸特征点集，还包括：选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点。
在上述实施例的基础上，所述选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点，包括：
所述脸颊特征点为两点；
选取位于左脸颊最高点的人脸轮廓特征点和位于下巴处的人脸轮廓特征点进行坐标加权计算，获取位于左脸颊的脸颊特征点；
选取位于右脸颊最高点的人脸轮廓特征点和位于下巴处的人脸轮廓特征点进行坐标加权 计算，获取位于右脸颊的脸颊特征点。
在上述实施例的基础上，所述对所述人脸特征点集的部分人脸特征点进行坐标调整，坐标调整的位移量由拟合的不同谐波的正弦分量和余弦分量的叠加获得，包括：
对所述人脸特征点集的人脸特征点按照预设的规则进行编号；
根据人脸特征点的编号并按照如下公式对所述人脸特征点集的部分人脸特征点进行坐标调整：
fx＝a 1+b 1cos(w 1i)+c 1sin(w 1i)+d 1cos(2w 1i)+e 1sin(2w 1i)
fy＝a 2+b 2cos(w 2i)+c 2sin(w 2i)+d 2cos(2w 2i)+e 2sin(2w 2i)+f cos(3w 2i)+g sin(3w 2i)
上述公式中，i表示人脸特征点的编号，fx、fy分别表示人脸特征点的横坐标和纵坐标的位移量，a 1、b 1、c 1、d 1、e 1和a 2、b 2、c 2、d 2、e 2、f、g为常数，w 1和w 2为正弦分量和余弦分量的频率。
在上述实施例的基础上，所述根据所述人脸特征点集对所述输入图像进行三角剖分和根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，包括：
将所有人脸特征点以及所述输入图像或过渡图像的四个顶点连接为若干个三角形，对图像区域进行剖分；其中，在形成的两个相邻的三角形构成凸四边形的对角线，在相互交换后，两个内角的最小角不再增大。
在上述实施例的基础上，所述对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标，包括：
根据对应的原始三角形图像和变形三角形图像的顶点坐标计算对应的原始三角形图像和变形三角形图像之间的仿射变换矩阵；
根据所述仿射变换矩阵对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。
在上述实施例的基础上，所述根据所述变换坐标填充所述变形三角形图像的像素值，包括根据所述变换坐标对所述变形三角形图像进行双线性插值。
上述提供的计算机设备可用于执行上述任意实施例提供的图像调整方法，具备相应的功能和有益效果。上述设备中各个组件的功能和作用的实现过程详见上述方法中对应步骤的实现过程，在此不再赘述。
对于设备实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的设备实施例仅仅是示意性的，其中所述作为分离部件说明的组件可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单 元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本公开方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。

Claims (10)

  1. 一种图像调整方法，包括如下步骤：
    对输入图像进行人脸检测，获取人脸特征点集；
    根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；
    对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；
    根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；
    判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；
    根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
  2. 根据权利要求1所述的图像调整方法，其中：所述人脸特征点集包括人脸轮廓特征点、眉毛特征点、眼睛特征点、鼻子特征点和嘴巴特征点。
  3. 根据权利要求2所述的图像调整方法，其中，所述获取人脸特征点集的步骤还包括：选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点。
  4. 根据权利要求3所述的图像调整方法，其中，所述选取若干个所述人脸轮廓特征点进行坐标加权计算，获得位于脸颊区域的脸颊特征点的步骤，包括：
    在所述人脸轮廓特征点的左右两边各选取一个最高点和一个最低点，分别对左右两边选取的所述最高点和所述最低点进行坐标加权计算，获取位于两边脸颊区域的脸颊特征点。
  5. 根据权利要求1～4任一项所述的图像调整方法，其中，所述对所述人脸特征点集的部分人脸特征点进行坐标调整的步骤中，坐标调整的位移量由拟合的不同谐波的正弦分量和余弦分量的叠加获得。
  6. 根据权利要求1～4任一项所述的图像调整方法，其中，所述根据所述人脸特征点集对所述输入图像进行三角剖分和根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，包括如下步骤：
    将所有人脸特征点以及所述输入图像或过渡图像的四个顶点连接为若干个三角形，对图像区域进行剖分。
  7. 根据权利要求1所述的图像调整方法，其中，所述对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标的步骤，包括：
    根据对应的原始三角形图像和变形三角形图像的顶点坐标计算对应的原始三角形图像和变形三角形图像之间的仿射变换矩阵；
    根据所述仿射变换矩阵对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标。
  8. 根据权利要求1所述的图像调整方法，其中：所述根据所述变换坐标填充所述变形三角形图像的像素值，包括根据所述变换坐标对所述变形三角形图像进行双线性插值。
  9. 一种图像调整装置，包括：
    人脸检测模块，用于对输入图像进行人脸检测，获取人脸特征点集
    第一三角剖分模块，用于根据所述人脸特征点集对所述输入图像进行三角剖分，获得关于所述输入图像的多个原始三角形图像；
    坐标调整模块，对所述人脸特征点集的部分人脸特征点进行坐标调整，获得过渡图像；
    第二三角剖分模块，用于根据坐标调整后的所述人脸特征点集对所述过渡图像进行三角剖分，获得关于所述过渡图像的多个变形三角形图像；
    坐标变换模块，用于判断对应的原始三角形图像和变形三角形图像的顶点坐标是否相同，其中，对应的原始三角形图像和变形三角形图像分别以坐标调整前后的相同的三个人脸特征点为顶点；若不相同，则对所述原始三角形图像中的像素点进行仿射变换，获得像素点在变形三角形图像中的变换坐标；
    像素值填充模块，用于根据所述变换坐标填充所述变形三角形图像的像素值，获得目标图像。
  10. 一种计算机设备，包括：
    存储器以及处理器；
    所述存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-8中任一所述的图像调整方法。
PCT/CN2019/126719 2019-07-24 2019-12-19 图像调整方法、装置和计算机设备 WO2021012599A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910670790.7 2019-07-24
CN201910670790.7A CN110555812A (zh) 2019-07-24 2019-07-24 图像调整方法、装置和计算机设备

Publications (1)

Publication Number Publication Date
WO2021012599A1 true WO2021012599A1 (zh) 2021-01-28

Family

ID=68735883

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/126719 WO2021012599A1 (zh) 2019-07-24 2019-12-19 图像调整方法、装置和计算机设备

Country Status (2)

Country Link
CN (1) CN110555812A (zh)
WO (1) WO2021012599A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112884642A (zh) * 2021-03-23 2021-06-01 福州大学 一种基于人脸识别技术的实时面部老化模拟方法
CN113436063A (zh) * 2021-07-30 2021-09-24 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113961746A (zh) * 2021-09-29 2022-01-21 北京百度网讯科技有限公司 视频生成方法、装置、电子设备及可读存储介质
CN115187822A (zh) * 2022-07-28 2022-10-14 广州方硅信息技术有限公司 人脸图像数据集分析方法、直播人脸图像处理方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110555812A (zh) * 2019-07-24 2019-12-10 广州视源电子科技股份有限公司 图像调整方法、装置和计算机设备
CN112488977B (zh) * 2020-12-22 2024-06-11 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894446A (zh) * 2016-05-09 2016-08-24 西安北升信息科技有限公司 一种视频中的自动脸部轮廓修饰方法
CN107527034A (zh) * 2017-08-28 2017-12-29 维沃移动通信有限公司 一种面部轮廓调整方法及移动终端
CN108537126A (zh) * 2018-03-13 2018-09-14 东北大学 一种人脸图像处理系统及方法
US20190122376A1 (en) * 2017-10-20 2019-04-25 Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. Method and device for image processing
CN110049351A (zh) * 2019-05-23 2019-07-23 北京百度网讯科技有限公司 视频流中人脸变形的方法和装置、电子设备、计算机可读介质
CN110555812A (zh) * 2019-07-24 2019-12-10 广州视源电子科技股份有限公司 图像调整方法、装置和计算机设备

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7046728B1 (en) * 2000-06-30 2006-05-16 Intel Corporation Method of video coding the movement of a human face from a sequence of images
CN101347332A (zh) * 2008-08-22 2009-01-21 深圳先进技术研究院 一种人脸三维面形数字化测量系统的测量方法及设备
CN102663413B (zh) * 2012-03-09 2013-11-27 中盾信安科技(江苏)有限公司 一种面向多姿态和跨年龄的人脸图像认证方法
CN106980809B (zh) * 2016-01-19 2020-08-21 深圳市朗驰欣创科技股份有限公司 一种基于asm的人脸特征点检测方法
CN108876705B (zh) * 2017-11-23 2022-03-22 北京旷视科技有限公司 图像合成的方法、装置及计算机存储介质
CN109242905B (zh) * 2018-08-31 2021-02-05 华中科技大学 一种基于数字图像衍射确定物体位移量及方向的方法
CN109522910B (zh) * 2018-12-25 2020-12-11 浙江商汤科技开发有限公司 关键点检测方法及装置、电子设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105894446A (zh) * 2016-05-09 2016-08-24 西安北升信息科技有限公司 一种视频中的自动脸部轮廓修饰方法
CN107527034A (zh) * 2017-08-28 2017-12-29 维沃移动通信有限公司 一种面部轮廓调整方法及移动终端
US20190122376A1 (en) * 2017-10-20 2019-04-25 Arcsoft (Hangzhou) Multimedia Technology Co., Ltd. Method and device for image processing
CN108537126A (zh) * 2018-03-13 2018-09-14 东北大学 一种人脸图像处理系统及方法
CN110049351A (zh) * 2019-05-23 2019-07-23 北京百度网讯科技有限公司 视频流中人脸变形的方法和装置、电子设备、计算机可读介质
CN110555812A (zh) * 2019-07-24 2019-12-10 广州视源电子科技股份有限公司 图像调整方法、装置和计算机设备

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112884642A (zh) * 2021-03-23 2021-06-01 福州大学 一种基于人脸识别技术的实时面部老化模拟方法
CN112884642B (zh) * 2021-03-23 2022-07-08 福州大学 一种基于人脸识别技术的实时面部老化模拟方法
CN113436063A (zh) * 2021-07-30 2021-09-24 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113436063B (zh) * 2021-07-30 2024-03-05 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113961746A (zh) * 2021-09-29 2022-01-21 北京百度网讯科技有限公司 视频生成方法、装置、电子设备及可读存储介质
CN113961746B (zh) * 2021-09-29 2023-11-21 北京百度网讯科技有限公司 视频生成方法、装置、电子设备及可读存储介质
CN115187822A (zh) * 2022-07-28 2022-10-14 广州方硅信息技术有限公司 人脸图像数据集分析方法、直播人脸图像处理方法及装置
CN115187822B (zh) * 2022-07-28 2023-06-30 广州方硅信息技术有限公司 人脸图像数据集分析方法、直播人脸图像处理方法及装置

Also Published As

Publication number Publication date
CN110555812A (zh) 2019-12-10

Similar Documents

Publication Publication Date Title
WO2021012599A1 (zh) 图像调整方法、装置和计算机设备
US20210019503A1 (en) Face detection method and apparatus, service processing method, terminal device, and storage medium
US10691927B2 (en) Image deformation processing method and apparatus, and computer storage medium
US9742994B2 (en) Content-aware wide-angle images
WO2020119458A1 (zh) 脸部关键点检测方法、装置、计算机设备和存储介质
CN111401266B (zh) 绘本角点定位的方法、设备、计算机设备和可读存储介质
AU2018253460B2 (en) Framework for local parameterization of 3d meshes
WO2021012596A1 (zh) 图像调整方法、装置、存储介质以及设备
US10977767B2 (en) Propagation of spot healing edits from one image to multiple images
US11367263B1 (en) Image-guided three dimensional modeling
WO2021003807A1 (zh) 一种图像深度估计方法及装置、电子设备、存储介质
WO2021197466A1 (zh) 眼球检测方法、装置、设备及存储介质
CN113643414A (zh) 一种三维图像生成方法、装置、电子设备及存储介质
JP6852224B2 (ja) 全視角方向の球体ライトフィールドレンダリング方法
CN113223137B (zh) 透视投影人脸点云图的生成方法、装置及电子设备
US11080920B2 (en) Method of displaying an object
CN108876704B (zh) 人脸图像变形的方法、装置及计算机存储介质
TWI711004B (zh) 圖片處理方法和裝置
US20230169755A1 (en) Apparatus and method with image processing
CN116993955A (zh) 三维模型的重拓扑方法、装置、设备及存储介质
US10861174B2 (en) Selective 3D registration
CN116977539A (zh) 图像处理方法、装置、计算机设备、存储介质和程序产品
CN113610864B (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN108009576A (zh) 一种目标匹配的物体识别方法、设备及存储设备
CN115760888A (zh) 图像处理方法、装置、计算机及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19938176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19938176

Country of ref document: EP

Kind code of ref document: A1