CN108961149A

CN108961149A - Image processing method, device and system and storage medium

Info

Publication number: CN108961149A
Application number: CN201710390286.2A
Authority: CN
Inventors: 周而进; 黄志翱; 刘研绎
Original assignee: Beijing Megvii Technology Co Ltd; Beijing Maigewei Technology Co Ltd
Current assignee: Beijing Megvii Technology Co Ltd; Beijing Maigewei Technology Co Ltd
Priority date: 2017-05-27
Filing date: 2017-05-27
Publication date: 2018-12-07
Anticipated expiration: 2037-05-27
Also published as: CN108961149B

Abstract

The embodiment provides a kind of image processing method, device and system and storage mediums.This method comprises: obtaining Initial Face image；Three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform；The predefined face key point information on three-dimensional headform；Three-dimensional headform is converted；Transformed three-dimensional headform is rendered into two-dimension human face image；And according to the projection relation and face key point information between transformed three-dimensional headform and two-dimension human face image, determines the two-dimension human face key point in two-dimension human face image and obtain the location information of two-dimension human face key point as face key point labeled data.The facial image and face key labeled data under a large amount of, various different scenes can be quickly and easily generated very much using the above method, device and system and storage medium.

Description

Image processing method, device and system and storage medium

Technical field

The present invention relates to the field artificial intelligence (AI), relate more specifically to a kind of image processing method, device and system and Storage medium.

Background technique

Face key point is located in recognition of face, Expression Recognition, has important application in gesture recognition.Mainstream at present Face key independent positioning method mainly utilizes machine learning algorithm to learn from face key point labeled data.Current face closes Key point labeled data depends on artificial mark, that is, needs to mark that acquisition is a large amount of, (the example under different scenes manually Under various different postures, illumination, expression) face key point labeled data.This artificial notation methods low efficiency, cost It is high.

Summary of the invention

The present invention is proposed in view of the above problem.The present invention provides a kind of image processing methods, device and system And storage medium.

According to an aspect of the present invention, a kind of image processing method is provided.This method comprises: obtaining Initial Face image； Three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform；Predefined face closes on three-dimensional headform Key point information；Three-dimensional headform is converted；Transformed three-dimensional headform is rendered into two-dimension human face image；And According to the projection relation and face key point information between transformed three-dimensional headform and two-dimension human face image, two are determined It ties up the two-dimension human face key point in facial image and obtains the location information of two-dimension human face key point as face key point mark Data.

Illustratively, before transformed three-dimensional headform is rendered into two-dimension human face image, image processing method Further include: additional information is added on transformed three-dimensional headform, wherein additional information includes background, hair and illumination In it is one or more.

Illustratively, additional information is added on transformed three-dimensional headform includes: in the transformed three-dimensional number of people Additional information is added on model at random.

Illustratively, face key point information includes internal point information relevant to key point inside face, two-dimension human face Key point includes the internal key point of two dimension.

Illustratively, internal point information includes the location information of the three-dimensional internal key point in three-dimensional headform, according to Projection relation and face key point information between transformed three-dimensional headform and two-dimension human face image determine two dimension people Two-dimension human face key point in face image simultaneously obtains the location information of two-dimension human face key point as face key point labeled data Include: the location information according to projection relation and three-dimensional internal key point, determines three-dimensional internal key point in two-dimension human face figure As place imaging plane in the internal key point of two dimension that obtains of projection and obtain the location information of key point inside two dimension.

Illustratively, face key point information includes contour point information relevant to facial contour key point, two-dimension human face Key point includes two-dimensional silhouette key point.

Illustratively, contour point information includes three-dimensional in three-dimensional headform, being likely to become facial contour key point The location information of profile point set, according between transformed three-dimensional headform and two-dimension human face image projection relation and people Face key point information determines the two-dimension human face key point in two-dimension human face image and obtains the location information of two-dimension human face key point The predefined starting point plane in three-D profile point set and three-dimensional headform is comprised determining that as face key point labeled data Intersection be three-dimensional starting point Candidate Set；According to projection relation and the location information of three-D profile point set, three-D profile point is determined Collection and three-dimensional starting point Candidate Set project the two-dimensional silhouette point set and two dimension of acquisition in the imaging plane where two-dimension human face image The position of starting point Candidate Set and two-dimensional silhouette point set and two-dimentional starting point Candidate Set；Profile starting point is selected from two-dimentional starting point Candidate Set With profile terminal；Facial contour line is determined based on the outer contour of profile starting point, profile terminal and two-dimensional silhouette point set；And base The two-dimensional silhouette key point in two-dimension human face image is determined in facial contour line and obtains the location information of two-dimensional silhouette key point.

Illustratively, the two-dimensional silhouette key point in two-dimension human face image is determined based on facial contour line and obtains two-dimentional wheel The location information of wide key point include: according to predefined face position kick-out condition reject facial contour line in, belong to spy The point at face position is determined, to obtain new facial contour line；By new facial contour line equal part；And determine the Along ent obtained For the two-dimensional silhouette key point in two-dimension human face image and obtain the location information of two-dimensional silhouette key point.

Illustratively, the two-dimensional silhouette key point in two-dimension human face image is determined based on facial contour line and obtains two-dimentional wheel The location information of wide key point includes: by facial contour line equal part；And determine that the Along ent obtained is in two-dimension human face image Two-dimensional silhouette key point and obtain the location information of two-dimensional silhouette key point.

Illustratively, carrying out transformation to three-dimensional headform includes: to carry out rigid body translation to three-dimensional headform；And/or Non-rigid transformation is carried out to three-dimensional headform.

Illustratively, three-dimensional headform includes grid model and texture model.

Illustratively, image processing method further include: utilize two-dimension human face image and the training of face key point labeled data Recurrence device for face key point location algorithm.

According to a further aspect of the invention, a kind of image processing apparatus is provided.The device includes: that initial pictures obtain mould Block, for obtaining Initial Face image；Three-dimensional reconstruction module, for carrying out three-dimensional reconstruction based on Initial Face image, to obtain Three-dimensional headform；Predefined module, for the predefined face key point information on three-dimensional headform；Conversion module is used It is converted in three-dimensional headform；Rendering module, for transformed three-dimensional headform to be rendered into two-dimension human face figure Picture；And key point determining module, for being closed according to the projection between transformed three-dimensional headform and two-dimension human face image System and face key point information, determine the two-dimension human face key point in two-dimension human face image and obtain two-dimension human face key point Location information is as face key point labeled data.

Illustratively, image processing apparatus further include: adding module, in rendering module by the transformed three-dimensional number of people Before model rendering is two-dimension human face image, additional information is added on transformed three-dimensional headform, wherein additional information Including one or more in background, hair and illumination.

Illustratively, adding module includes: random addition submodule, for random on transformed three-dimensional headform Add additional information.

Illustratively, internal point information includes the location information of the three-dimensional internal key point in three-dimensional headform, crucial Point determining module includes: that internal key point determines submodule, for the position according to projection relation and three-dimensional internal key point Information determines that three-dimensional internal key point projects the internal key point of the two dimension obtained in the imaging plane where two-dimension human face image And obtain the location information of key point inside two dimension.

Illustratively, contour point information includes three-dimensional in three-dimensional headform, being likely to become facial contour key point The location information of profile point set, key point determining module includes: three-dimensional determining submodule, for determining three-D profile point set and three The intersection for tieing up the predefined starting point plane in headform is three-dimensional starting point Candidate Set；Two dimension determines submodule, is used for basis The location information of projection relation and three-D profile point set determines three-D profile point set and three-dimensional starting point Candidate Set in two-dimension human face The two-dimensional silhouette point set and two-dimentional starting point Candidate Set and two-dimensional silhouette point set and two of acquisition are projected in imaging plane where image Tie up the position of starting point Candidate Set；Submodule is selected, for selecting profile starting point and profile terminal from two-dimentional starting point Candidate Set；Wheel Profile determines submodule, for determining facial contour based on the outer contour of profile starting point, profile terminal and two-dimensional silhouette point set Line；And profile key point determines submodule, for determining that the two-dimensional silhouette in two-dimension human face image is closed based on facial contour line Key point and the location information for obtaining two-dimensional silhouette key point.

Illustratively, profile key point determines that submodule includes: culling unit, for being picked according to predefined face position Except condition rejects point in facial contour line, to belong to Given Face position, to obtain new facial contour line；First equal part list Member, for by new facial contour line equal part；And first profile key point determination unit, it is for the determining Along ent obtained Two-dimensional silhouette key point in two-dimension human face image and the location information for obtaining two-dimensional silhouette key point.

Illustratively, profile key point determines that submodule includes: the second equal sub-units, is used for facial contour line equal part； And the second profile key point determination unit, for determining that the Along ent obtained is that two-dimensional silhouette in two-dimension human face image is crucial Put and obtain the location information of two-dimensional silhouette key point.

Illustratively, conversion module includes: rigid body translation submodule, for carrying out rigid body translation to three-dimensional headform； And/or non-rigid transformation submodule, for carrying out non-rigid transformation to three-dimensional headform.

Illustratively, image processing apparatus further include: training module, for utilizing two-dimension human face image and face key point Labeled data training is used for the recurrence device of face key point location algorithm.

According to a further aspect of the invention, a kind of image processing system, including processor and memory are provided, wherein deposit Computer program instructions are stored in reservoir, for executing following steps when computer program instructions are run by processor: obtaining Initial Face image；Three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform；On three-dimensional headform Predefined face key point information；Three-dimensional headform is converted；Transformed three-dimensional headform is rendered into two dimension Facial image；And according to the projection relation and face key between transformed three-dimensional headform and two-dimension human face image Point information, determines the two-dimension human face key point in two-dimension human face image and obtains the location information of two-dimension human face key point as people Face key point labeled data.

Illustratively, when computer program instructions are run by processor used execution by the transformed three-dimensional number of people Before the step of model rendering is two-dimension human face image, it is also used to execute following step when computer program instructions are run by processor It is rapid: to add additional information on transformed three-dimensional headform, wherein additional information includes in background, hair and illumination It is one or more.

Illustratively, used execution in transformed three-dimensional number of people mould when computer program instructions are run by processor The step of additional information is added in type includes: to add additional information at random on transformed three-dimensional headform.

Illustratively, internal point information includes the location information of the three-dimensional internal key point in three-dimensional headform, is calculated Machine program instruction when being run by processor used execution according to transformed three-dimensional headform and two-dimension human face image it Between projection relation and face key point information, determine the two-dimension human face key point in two-dimension human face image and obtain two-dimentional people The location information of face key point includes: to be closed according to projection relation and three-dimensional internal as the step of face key point labeled data The location information of key point determines that three-dimensional internal key point projects the two dimension of acquisition in the imaging plane where two-dimension human face image Internal key point and the location information for obtaining key point inside two dimension.

Illustratively, contour point information includes three-dimensional in three-dimensional headform, being likely to become facial contour key point The location information of profile point set, used execution according to transformed three-dimensional people when computer program instructions are run by processor Projection relation and face key point information between head model and two-dimension human face image, determine the two dimension in two-dimension human face image The face key point and location information for obtaining two-dimension human face key point is comprised determining that as the step of face key point labeled data The intersection of three-D profile point set and the predefined starting point plane in three-dimensional headform is three-dimensional starting point Candidate Set；According to projection The location information of relationship and three-D profile point set determines three-D profile point set and three-dimensional starting point Candidate Set in two-dimension human face image The two-dimensional silhouette point set and two-dimentional starting point Candidate Set and two-dimensional silhouette point set and two dimension that acquisition is projected in the imaging plane at place rise The position of point Candidate Set；Profile starting point and profile terminal are selected from two-dimentional starting point Candidate Set；Based on profile starting point, profile terminal Facial contour line is determined with the outer contour of two-dimensional silhouette point set；And it is determined in two-dimension human face image based on facial contour line Two-dimensional silhouette key point and the location information for obtaining two-dimensional silhouette key point.

Illustratively, used execution determines two based on facial contour line when computer program instructions are run by processor It ties up the two-dimensional silhouette key point in facial image and includes: according to predetermined the step of obtaining the location information of two-dimensional silhouette key point The face position kick-out condition of justice rejects point in facial contour line, to belong to Given Face position, to obtain new face wheel Profile；By new facial contour line equal part；And determine that the Along ent obtained is the two-dimensional silhouette key in two-dimension human face image Put and obtain the location information of two-dimensional silhouette key point.

Illustratively, used execution determines two based on facial contour line when computer program instructions are run by processor The step of tieing up the two-dimensional silhouette key point in facial image and obtaining the location information of two-dimensional silhouette key point includes: by face wheel Profile equal part；And determine that the Along ent obtained is the two-dimensional silhouette key point in two-dimension human face image and obtains two-dimensional silhouette pass The location information of key point.

Illustratively, used execution becomes three-dimensional headform when computer program instructions are run by processor The step of changing includes: to carry out rigid body translation to three-dimensional headform；And/or non-rigid transformation is carried out to three-dimensional headform.

Illustratively, it is also used to execute following steps when computer program instructions are run by processor: utilizing two-dimension human face Image and the training of face key point labeled data are used for the recurrence device of face key point location algorithm.

According to a further aspect of the invention, a kind of storage medium is provided, stores program instruction, program on a storage medium Instruction is at runtime for executing following steps: obtaining Initial Face image；Three-dimensional reconstruction is carried out based on Initial Face image, with Obtain three-dimensional headform；The predefined face key point information on three-dimensional headform；Three-dimensional headform is converted； Transformed three-dimensional headform is rendered into two-dimension human face image；And according to transformed three-dimensional headform and two-dimentional people Projection relation and face key point information between face image determine the two-dimension human face key point in two-dimension human face image and obtain The location information of two-dimension human face key point is obtained as face key point labeled data.

Illustratively, program instruction at runtime it is used execute transformed three-dimensional headform is rendered into two dimension Before the step of facial image, it is also used to execute following steps when computer program instructions are run by processor: transformed Additional information is added on three-dimensional headform, wherein additional information includes one or more in background, hair and illumination.

Illustratively, the used addition on transformed three-dimensional headform executed is additional at runtime for program instruction The step of information includes: to add additional information at random on transformed three-dimensional headform.

Illustratively, internal point information includes the location information of the three-dimensional internal key point in three-dimensional headform, program Instruction at runtime it is used execute according to the projection relation between transformed three-dimensional headform and two-dimension human face image And face key point information, it determines the two-dimension human face key point in two-dimension human face image and obtains the position of two-dimension human face key point Confidence breath includes: to be believed according to the position of projection relation and three-dimensional internal key point as the step of face key point labeled data Breath determines that three-dimensional internal key point projects the internal key point of the two dimension obtained simultaneously in the imaging plane where two-dimension human face image Obtain the location information of the internal key point of two dimension.

Illustratively, contour point information includes three-dimensional in three-dimensional headform, being likely to become facial contour key point The location information of profile point set, program instruction at runtime it is used execute according to transformed three-dimensional headform and two dimension Projection relation and face key point information between facial image determine two-dimension human face key point in two-dimension human face image simultaneously The location information for obtaining two-dimension human face key point comprises determining that three-D profile point set as the step of face key point labeled data Intersection with the predefined starting point plane in three-dimensional headform is three-dimensional starting point Candidate Set；According to projection relation and three-dimensional The location information of profile point set determines that three-D profile point set and three-dimensional starting point Candidate Set are flat in the imaging where two-dimension human face image The two-dimensional silhouette point set of acquisition and the position of two-dimentional starting point Candidate Set and two-dimensional silhouette point set and two-dimentional starting point Candidate Set are projected in face It sets；Profile starting point and profile terminal are selected from two-dimentional starting point Candidate Set；Based on profile starting point, profile terminal and two-dimensional silhouette point The outer contour of collection determines facial contour line；And the key of the two-dimensional silhouette in two-dimension human face image is determined based on facial contour line Put and obtain the location information of two-dimensional silhouette key point.

Illustratively, program instruction at runtime it is used execute determined in two-dimension human face image based on facial contour line Two-dimensional silhouette key point and the step of obtaining the location information of two-dimensional silhouette key point include: according to predefined face position Kick-out condition rejects point in facial contour line, to belong to Given Face position, to obtain new facial contour line；By new people Face contour line equal part；And determine that the Along ent obtained is the two-dimensional silhouette key point in two-dimension human face image and obtains two-dimentional wheel The location information of wide key point.

Illustratively, program instruction at runtime it is used execute determined in two-dimension human face image based on facial contour line Two-dimensional silhouette key point and the step of obtaining the location information of two-dimensional silhouette key point include: by facial contour line equal part；With And determine that the Along ent obtained is the two-dimensional silhouette key point in two-dimension human face image and the position for obtaining two-dimensional silhouette key point Information.

Illustratively, program instruction used packet the step of transformation to three-dimensional headform executed at runtime It includes: rigid body translation is carried out to three-dimensional headform；And/or non-rigid transformation is carried out to three-dimensional headform.

Illustratively, program instruction is also used to execute following steps at runtime: being closed using two-dimension human face image and face The training of key point labeled data is used for the recurrence device of face key point location algorithm.

Image processing method, device and system and storage medium according to an embodiment of the present invention, can be very simple and quick Ground generates facial image and face key labeled data under a large amount of, various different scenes.Compared with artificial notation methods, on It states methods, devices and systems and storage medium is high-efficient, at low cost, also, will be situated between by the above method, device and system storage The facial image and face key labeled data that matter obtains can train as training data and obtain the more robust face pass of performance Key spot detector.

Detailed description of the invention

The embodiment of the present invention is described in more detail in conjunction with the accompanying drawings, the above and other purposes of the present invention, Feature and advantage will be apparent.Attached drawing is used to provide to further understand the embodiment of the present invention, and constitutes explanation A part of book, is used to explain the present invention together with the embodiment of the present invention, is not construed as limiting the invention.In the accompanying drawings, Identical reference label typically represents same parts or step.

Fig. 1 shows showing for the exemplary electronic device for realizing image processing method according to an embodiment of the present invention and device Meaning property block diagram；

Fig. 2 shows the schematic flow charts of image processing method according to an embodiment of the invention；

Texture model that is according to an embodiment of the present invention, obtaining based on same Initial Face image is shown respectively in Fig. 3 a and 3b With the schematic diagram of grid model；

The signal of texture model and grid model corresponding with Fig. 3 a and Fig. 3 b, transformed is shown respectively in Fig. 3 c and 3d Figure；

Fig. 4 a-4d shows the schematic diagram of grid model according to an embodiment of the present invention；

Fig. 5 shows the schematic block diagram of image processing apparatus according to an embodiment of the invention；And

Fig. 6 shows the schematic block diagram of image processing system according to an embodiment of the invention.

Specific embodiment

In order to enable the object, technical solutions and advantages of the present invention become apparent, root is described in detail below with reference to accompanying drawings According to example embodiments of the present invention.Obviously, described embodiment is only a part of the embodiments of the present invention, rather than this hair Bright whole embodiments, it should be appreciated that the present invention is not limited by example embodiment described herein.Based on described in the present invention The embodiment of the present invention, those skilled in the art's obtained all other embodiment in the case where not making the creative labor It should all fall under the scope of the present invention.

In order to solve problem as described above, the embodiment of the present invention provide a kind of image processing method, device and system and Storage medium, can be obtained under different scenes by the transformation and rendering of three-dimensional headform (such as various different postures, light According under, expression) facial image, and obtain corresponding face key point labeled data.Utilize figure provided in an embodiment of the present invention As processing method, the facial image and face key under a large amount of, various different scenes can simply, be easily and quickly generated Labeled data.

Firstly, describing the example for realizing image processing method according to an embodiment of the present invention and device referring to Fig.1 Electronic equipment 100.

As shown in Figure 1, electronic equipment 100 include one or more processors 102, it is one or more storage device 104, defeated Enter device 106, output device 108 and image collecting device 110, these components pass through bus system 112 and/or other forms The interconnection of bindiny mechanism's (not shown).It should be noted that the component and structure of electronic equipment 100 shown in FIG. 1 are only exemplary, and Unrestricted, as needed, the electronic equipment also can have other assemblies and structure.

The processor 102 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution The processing unit of the other forms of ability, and the other components that can control in the electronic equipment 100 are desired to execute Function.

The storage device 104 may include one or more computer program products, and the computer program product can To include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easy The property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non- Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage medium On can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter institute The client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meter Can also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/or The various data etc. generated.

The input unit 106 can be the device that user is used to input instruction, and may include keyboard, mouse, wheat One or more of gram wind and touch screen etc..

The output device 108 can export various information (such as image and/or sound) to external (such as user), and It and may include one or more of display, loudspeaker etc..

Described image acquisition device 110 can acquire facial image (including video frame), and by face figure collected As being stored in the storage device 104 for the use of other components.Image collecting device 110 can be regular camera.It answers Work as understanding, image collecting device 110 is only example, and electronic equipment 100 can not include image collecting device 110.In this feelings Under condition, it can use other image acquisition device facial images, and the facial image of acquisition is sent to electronic equipment 100。

Illustratively, the exemplary electronic device for realizing image processing method according to an embodiment of the present invention and device can To be realized in the equipment of personal computer or remote server etc..

In the following, image processing method according to an embodiment of the present invention will be described with reference to Fig. 2.Fig. 2 shows according to the present invention one The schematic flow chart of the image processing method 200 of a embodiment.As shown in Fig. 2, image processing method 200 includes following step Suddenly.

In step S210, Initial Face image is obtained.

Initial Face image can be any suitable image comprising face.Initial Face image can be camera etc. The original image that image acquisition device arrives is also possible to the image obtained after being pre-processed to original image.

Initial Face image can be sent to electronic equipment 100 by external equipment (such as cloud server) to be set by electronics Standby 100 processor 102 is handled, can also be by image collecting device 110 (such as camera) that electronic equipment 100 includes It acquires and is transmitted to processor 102 and handled, the storage device 104 that can also include by electronic equipment 100 is stored and transmitted It is handled to processor 102.

In step S220, three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform.

It is exemplary and not restrictive, step S220 can be using any suitable existing or people in the cards in the future Face three-dimensional reconstruction is realized.Illustratively, step S220 three-dimensional headform obtained may include grid model and Texture model.It will be appreciated by those skilled in the art that the data structure and meaning of grid model and texture model, do not do superfluous herein It states.

In step S230, the predefined face key point information on three-dimensional headform.

Illustratively, face key point may include key point and/or facial contour key point inside face.Correspondingly, Face key point information may include to the relevant internal point information of key point inside face and/or with facial contour key point phase The contour point information of pass.Illustratively, key point refers to the point on the positions such as eyebrow, eye, nose, mouth inside face.

Illustratively, face key point out can be pre-defined on grid model or is likely to become face key point Point.Such as, if it is desired to the face key point labeled data finally obtained includes the location information of key point inside face, then may be used It is key point inside face to pre-define which point on grid model.In another example, if it is desired to the face finally obtained closes Key point labeled data includes the location information of facial contour key point, then which point can be pre-defined on grid model may As facial contour key point.

In step S240, three-dimensional headform is converted.

It is exemplary and not restrictive, step S240 may include: to three-dimensional headform carry out rigid body translation；And/or Non-rigid transformation is carried out to three-dimensional headform.Illustratively, rigid body translation may include one of rotation, zooming and panning Or it is a variety of.Carrying out rigid body translation to three-dimensional headform can change the posture of face, position etc..Illustratively, non-rigid becomes Changing may include expression synthesis.Carrying out non-rigid transformation to three-dimensional headform can change the expression of face.

In step S250, transformed three-dimensional headform is rendered into two-dimension human face image.

Step S250 can be realized using conventional Rendering.It illustratively, can be by above-mentioned grid model and texture Input of the model as Rendering algorithms, Rendering algorithms can export the two-dimension human face image that rendering obtains.

In step S260, according between transformed three-dimensional headform and two-dimension human face image projection relation and people Face key point information determines the two-dimension human face key point in two-dimension human face image and obtains the position conduct of two-dimension human face key point Face key point labeled data.

As described above, three-dimensional headform has predefined face key point information.For example, can be in the three-dimensional number of people It pre-defines face key point on model or is likely to become the point of face key point.Which point on three-dimensional headform is people It is not change with the transformation of three-dimensional headform that face key point, which is either likely to become information as the point of face key point, 's.Therefore, transformed three-dimensional headform can maintain predefined face key point information.

There is a kind of projection relation between transformed three-dimensional headform and two-dimension human face image.It can based on projection relation With each point on the transformed three-dimensional headform of determination after projecting to the imaging plane where two-dimension human face image Location information (such as coordinate).For example, if on three-dimensional headform inside predefined face key point location information, It can determine the location information of key point inside on two-dimension human face image, corresponding face.In the following description, for side Just it distinguishes, key point inside the face on three-dimensional headform is known as three-dimensional internal key point, the people in two-dimension human face image Key point is known as the internal key point of two dimension inside face.For facial contour key point, if predefined on three-dimensional headform It is likely to become the location information of the point of facial contour key point, can equally be determined in two-dimension human face image according to projection relation The location information of facial contour key point.

It will be understood by those skilled in the art that the execution sequence of each step of image processing method 200 shown in Fig. 2 is only Example rather than limit, for example, step S250 can be executed after step S260 or simultaneously.

Image processing method according to an embodiment of the present invention generates three-dimensional headform based on Initial Face image, and leads to The transformation and rendering for crossing three-dimensional headform obtain two-dimension human face image and corresponding face key point labeled data.The above method The facial image and face key labeled data under a large amount of, various different scenes can quickly and easily be generated very much.With people Work notation methods are compared, and the above method is high-efficient, at low cost, also, the facial image and face that will be obtained by the above method Crucial labeled data can train as training data and obtain the more robust face Keypoint detector of performance.

Illustratively, image processing method according to an embodiment of the present invention can be in setting with memory and processor It is realized in standby, device or system.

Image processing method according to an embodiment of the present invention can be deployed at independent equipment, for example, can be deployed in At personal computer or server.Alternatively, image processing method according to an embodiment of the present invention can also be deployed in being distributed At server end (or cloud) and client.For example, can will be adopted in client acquisition or storing initial facial image, client The Initial Face image for collecting or being stored sends server end (or cloud) to, carries out image by server end (or cloud) Reason.

According to embodiments of the present invention, before step S250, image processing method 200 can also include: transformed Additional information is added on three-dimensional headform.

Illustratively, additional information may include one or more in background, hair and illumination.Add additional information it Afterwards, the two-dimension human face image rendered may include the additional information in addition to face, in this way, obtaining each with can be convenient Facial image under the scenes such as kind different background, illumination condition.If utilizing the two-dimension human face image comprising various additional informations Training face Keypoint detector then can obtain the higher face Keypoint detector of precision with training for promotion effect.

In one example, additional information is added on transformed three-dimensional headform may include: transformed The upper random addition additional information of three-dimensional headform.Addition additional information can obtain more abundant, variation in a random basis Property bigger two-dimension human face image and face key point labeled data, so that using the two-dimension human face image and face key point mark It is more preferable as the training effect of the face Keypoint detector of training data to infuse data.It is of course also possible in transformed three-dimensional Preset additional information is added on headform.

According to embodiments of the present invention, face key point information may include that internal point relevant to key point inside face is believed Breath, two-dimension human face key point may include key point inside two dimension.If it is desire to marking number in the face key point finally obtained Location information including key point inside face in, then can pre-define on three-dimensional headform and close with inside face The relevant internal point information of key point.As described below, which may include the three-dimensional internal in three-dimensional headform The location information of key point.

According to embodiments of the present invention, internal point information may include the position of the three-dimensional internal key point in three-dimensional headform Confidence breath, step S260 may include: the location information according to projection relation and three-dimensional internal key point, determine the three-dimensional Internal key point projects the internal key point of the two dimension obtained in the imaging plane where the two-dimension human face image and obtains institute State the location information of key point inside two dimension.Face key point labeled data includes the location information of key point inside two dimension.

Texture model that is according to an embodiment of the present invention, obtaining based on same Initial Face image is shown respectively in Fig. 3 a and 3b With the schematic diagram of grid model；Texture model corresponding with Fig. 3 a and Fig. 3 b, transformed and grid is shown respectively in Fig. 3 c and 3d The schematic diagram of model.As shown in Figure 3a and Figure 3b shows, there is key point inside predefined face on three-dimensional headform, i.e., it is three-dimensional Internal key point.It is appreciated that inside face on three-dimensional headform the definition of key point be relative dimensional headform and Speech, the definition of key point does not change with the variation of the posture of face and expression inside the face.For example, it is assumed that defining pupil Central point is one of key point inside face, even if then three-dimensional headform's rotation, as shown in figs. 3 c and 3d as so that people For face towards other directions, pupil center's point is that this definition of key point is also constant inside face.Therefore, in three-dimensional number of people mould After type rotation terminates, it can be closed according to the direction of three-dimensional headform at this time and its projection between two-dimension human face image System calculates the coordinate (i.e. the location information of pupil center's point) for obtaining pupil center's point in two-dimension human face image, so that it is determined that The position of key point inside this face.Key point situation is identical inside other faces, does not repeat them here.

According to embodiments of the present invention, face key point information may include profile point letter relevant to facial contour key point Breath, two-dimension human face key point may include two-dimensional silhouette key point.If it is desire to marking number in the face key point finally obtained Including the location information of facial contour key point in, then it can be pre-defined on three-dimensional headform and facial contour closes The relevant contour point information of key point.As described below, the contour point information may include it is in three-dimensional headform, be likely to become The location information of the three-D profile point set of facial contour key point.

According to embodiments of the present invention, contour point information may include it is in three-dimensional headform, be likely to become facial contour The location information of the three-D profile point set of key point, step S260 may include: determining three-D profile point set and three-dimensional number of people mould The intersection of predefined starting point plane in type is three-dimensional starting point Candidate Set；According to the position of projection relation and three-D profile point set Confidence breath determines that three-D profile point set and three-dimensional starting point Candidate Set project acquisition in the imaging plane where two-dimension human face image Two-dimensional silhouette point set and two-dimentional starting point Candidate Set and two-dimensional silhouette point set and two-dimentional starting point Candidate Set position；From two-dimentional starting point Profile starting point and profile terminal are selected in Candidate Set；Outer contour based on profile starting point, profile terminal and two-dimensional silhouette point set Determine facial contour line；And the two-dimensional silhouette key point in two-dimension human face image is determined based on facial contour line and obtains two dimension The location information of profile key point.Face key point labeled data includes the location information of two-dimensional silhouette key point.

Facial contour key point can change with the variation of human face posture.If rendering the two-dimension human face image obtained In face it is different from the facial orientation in Initial Face image, then the facial contour key point in the two may change. Therefore, on three-dimensional headform, the three-D profile point set for being likely to become facial contour key point can be pre-defined.Finally obtain Certain points of the facial contour key point (i.e. two-dimensional silhouette key point) and three-D profile point concentration in two-dimension human face image obtained Between there are corresponding relationships.

Fig. 4 a-4d shows the schematic diagram of grid model according to an embodiment of the present invention.A referring to fig. 4, in three-dimensional headform The upper pre-defined three-D profile point set S for being likely to become facial contour key point, that is, pre-defined three-D profile point set S exist Location information in three-dimensional headform.In addition, " starting point plane " can also be pre-defined on three-dimensional headform.One In a example, starting point plane definition is horizontal plane under the front view of three-dimensional headform, through remarkable two lines of centres, As shown in Figure 4 b.Starting point plane can be used for defining the left and right starting point (i.e. profile starting point and profile terminal) of facial contour.Starting point The intersection of plane and three-D profile point set S are known as three-dimensional starting point Candidate Set T.It is appreciated that the definition of above-mentioned starting point plane is only Example rather than limitation of the present invention, starting point plane can have other and suitable define.

After three-dimensional headform rotates by a certain angle (as shown in Fig. 4 c or 4d), the three-dimensional number of people mould that will currently obtain Type (i.e. transformed three-dimensional headform) is projected in imaging plane, to obtain two-dimension human face image.Imaging plane is two-dimentional people Plane where face image.Three-D profile point set S and three-dimensional starting point Candidate Set T correspond respectively to the two-dimensional silhouette under imaging plane Point set S ' and two-dimentional starting point Candidate Set T '.It illustratively, as illustrated in fig. 4 c, can be by the ultra-left point A of two-dimentional starting point Candidate Set T ' Profile starting point and profile terminal are respectively defined as with rightest point B.Based on profile starting point A, profile terminal B and two-dimensional silhouette point set S ' Outer contour can determine facial contour line L (in Fig. 4 c for by endpoint of A and B and by C curve).It then, can be with Certain points on facial contour line L are considered as facial contour key point, to obtain required two-dimensional silhouette key point and its position Information.

According to embodiments of the present invention, the two-dimensional silhouette key point in two-dimension human face image is determined based on facial contour line and obtained The location information for obtaining two-dimensional silhouette key point may include: to reject facial contour line according to predefined face position kick-out condition In, the point that belongs to Given Face position, to obtain new facial contour line；By new facial contour line equal part；And it determines The Along ent of acquisition is the two-dimensional silhouette key point in two-dimension human face image and the location information for obtaining two-dimensional silhouette key point.

Given Face position can be pre-defined by user, can be any suitable position.Exemplary rather than limitation Property, Given Face position may include nose.That is, rejecting the point of nose areas in facial contour line.Continue to join Fig. 4 d is examined, the new facial contour line obtained after the point for rejecting nose areas is shown.It can be by face new shown in Fig. 4 d Contour line equal part, and Along ent is considered as to required two-dimensional silhouette key point.Given Face position (such as nose portion will be belonged to Position) point reject, can obtain and be more in line with actual facial contour key point, so as to more truely and accurately reflect people The profile information of face.

According to embodiments of the present invention, the two-dimensional silhouette key point in two-dimension human face image is determined based on facial contour line and obtained The location information for obtaining two-dimensional silhouette key point may include: by facial contour line equal part；And determine that the Along ent obtained is two It ties up the two-dimensional silhouette key point in facial image and obtains the location information of two-dimensional silhouette key point.

Illustratively, the point for belonging to Given Face position can not be rejected, facial contour line equal part is directly obtained two Tie up profile key point.

According to embodiments of the present invention, image processing method 200 can also include: crucial using two-dimension human face image and face Point labeled data training is used for the recurrence device of face key point location algorithm.

Illustratively, returning device (can be used for forming above-mentioned face Keypoint detector) may include random forest or mind Through network.It will be appreciated by those skilled in the art that returning the training method of device, do not repeat them here herein.Using two-dimension human face image and What the training of face key point labeled data obtained returns device to the positioning accuracy height of face key point, and speed is fast.

According to a further aspect of the invention, a kind of image processing apparatus is provided.Fig. 5 is shown according to an embodiment of the present invention Image processing apparatus 500 schematic block diagram.

As shown in figure 5, image processing apparatus 500 according to an embodiment of the present invention includes that initial pictures obtain module 510, three Dimension rebuilds module 520, predefined module 530, conversion module 540, rendering module 550 and key point determining module 560.It is described each A module can execute each step/function above in conjunction with Fig. 2-4 image processing method described respectively.Below only to the figure As the major function of each component of processing unit 500 is described, and omit the detail content having been described above.

Initial pictures obtain module 510 for obtaining Initial Face image.Initial pictures obtain module 510 can be by Fig. 1 Shown in the program instruction that stores in 102 Running storage device 104 of processor in electronic equipment realize.

Three-dimensional reconstruction module 520 is used to carry out three-dimensional reconstruction based on Initial Face image, to obtain three-dimensional headform.Three Dimension rebuilds the program that module 520 can store in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1 Instruction is to realize.

Predefined module 530 is used for the predefined face key point information on three-dimensional headform.Predefined module 530 can It is realized with the program instruction that is stored in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1.

Conversion module 540 is for converting three-dimensional headform.Conversion module 540 can electronics as shown in Figure 1 The program instruction that stores in 102 Running storage device 104 of processor in equipment is realized.

Rendering module 550 is used to transformed three-dimensional headform being rendered into two-dimension human face image.Rendering module 550 can It is realized with the program instruction that is stored in 102 Running storage device 104 of processor in electronic equipment as shown in Figure 1.

Key point determining module 560 is used for according to the projection between transformed three-dimensional headform and two-dimension human face image Relationship and face key point information determine the two-dimension human face key point in two-dimension human face image and obtain two-dimension human face key point Location information as face key point labeled data.Key point determining module 560 can be in electronic equipment as shown in Figure 1 The program instruction that stores in 102 Running storage device 104 of processor is realized.

According to embodiments of the present invention, image processing apparatus 500 further include: adding module (not shown), in rendering mould Before transformed three-dimensional headform is rendered into two-dimension human face image by block, added on transformed three-dimensional headform attached Add information, wherein additional information includes one or more in background, hair and illumination.

According to embodiments of the present invention, adding module includes: random addition submodule, in transformed three-dimensional number of people mould Additional information is added in type at random.

According to embodiments of the present invention, face key point information includes internal point information relevant to key point inside face, Two-dimension human face key point includes the internal key point of two dimension.

According to embodiments of the present invention, internal point information includes the position letter of the three-dimensional internal key point in three-dimensional headform Breath, key point determining module 560 includes: that internal key point determines submodule, for being closed according to projection relation and three-dimensional internal The location information of key point determines that three-dimensional internal key point projects the two dimension of acquisition in the imaging plane where two-dimension human face image Internal key point and the location information for obtaining key point inside two dimension.

According to embodiments of the present invention, face key point information includes contour point information relevant to facial contour key point, Two-dimension human face key point includes two-dimensional silhouette key point.

According to embodiments of the present invention, contour point information include it is in three-dimensional headform, be likely to become facial contour key The location information of the three-D profile point set of point, key point determining module 560 includes: three-dimensional determining submodule, three-dimensional for determining The intersection of profile point set and the predefined starting point plane in three-dimensional headform is three-dimensional starting point Candidate Set；Two dimension determines submodule Block determines that three-D profile point set and three-dimensional starting point are candidate for the location information according to projection relation and three-D profile point set Collect the two-dimensional silhouette point set that acquisition is projected in the imaging plane where two-dimension human face image and two-dimentional starting point Candidate Set and two dimension The position of profile point set and two-dimentional starting point Candidate Set；Submodule is selected, for selecting profile starting point from two-dimentional starting point Candidate Set With profile terminal；Contour line determines submodule, for the outer contour based on profile starting point, profile terminal and two-dimensional silhouette point set Determine facial contour line；And profile key point determines submodule, for being determined in two-dimension human face image based on facial contour line Two-dimensional silhouette key point and obtain the location information of two-dimensional silhouette key point.

According to embodiments of the present invention, profile key point determines that submodule includes: culling unit, for according to predefined people Face's position kick-out condition rejects point in facial contour line, to belong to Given Face position, to obtain new facial contour line；The First-class sub-unit, for by new facial contour line equal part；And first profile key point determination unit, for determining acquisition Along ent is the two-dimensional silhouette key point in two-dimension human face image and the location information for obtaining two-dimensional silhouette key point.

According to embodiments of the present invention, profile key point determines that submodule includes: the second equal sub-units, is used for facial contour Line equal part；And the second profile key point determination unit, for determining that the Along ent obtained is the two dimension in two-dimension human face image Profile key point and the location information for obtaining two-dimensional silhouette key point.

According to embodiments of the present invention, conversion module 540 includes: rigid body translation submodule, for three-dimensional headform into Row rigid body translation；And/or non-rigid transformation submodule, for carrying out non-rigid transformation to three-dimensional headform.

According to embodiments of the present invention, three-dimensional headform includes grid model and texture model.

According to embodiments of the present invention, image processing apparatus 500 further include: training module (not shown), for utilizing two dimension Facial image and the training of face key point labeled data are used for the recurrence device of face key point location algorithm.

Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed The scope of the present invention.

Fig. 6 shows the schematic block diagram of image processing system 600 according to an embodiment of the invention.Image procossing system System 600 includes image collecting device 610, memory 620 and processor 630.

Image collecting device 610 is for acquiring Initial Face image.Image collecting device 610 is optional, image procossing System 600 can not include image collecting device 610.In such a case, it is possible to utilize other image acquisition device institutes The Initial Face image needed, and the Initial Face image of acquisition is sent to image processing system 600.In addition, Initial Face figure Picture can also be stored in memory 620, or be sent to image processing system 600 by other External memory equipments.

The storage of memory 620 is for realizing the corresponding steps in image processing method according to an embodiment of the present invention Program code.

The processor 630 is used to run the program code (i.e. computer program instructions) stored in the memory 620, To execute the corresponding steps of image processing method according to an embodiment of the present invention, and for realizing according to an embodiment of the present invention Initial pictures in image processing apparatus 500 obtain module 510, three-dimensional reconstruction module 520, predefined module 530, conversion module 540, rendering module 550 and key point determining module 560.

In one embodiment, for executing following steps when said program code is run by the processor 630: obtaining Initial Face image；Three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform；On three-dimensional headform Predefined face key point information；Three-dimensional headform is converted；Transformed three-dimensional headform is rendered into two dimension Facial image；And according to the projection relation and face key between transformed three-dimensional headform and two-dimension human face image Point information, determines the two-dimension human face key point in two-dimension human face image and obtains the location information of two-dimension human face key point as people Face key point labeled data.

In one embodiment, when said program code is run by the processor 630 used execution will be through becoming Before the step of three-dimensional headform changed is rendered into two-dimension human face image, said program code is run by the processor 630 When be also used to execute: add additional information on transformed three-dimensional headform, wherein additional information includes background, hair With it is one or more in illumination.

In one embodiment, used execution transformed when said program code is run by the processor 630 Three-dimensional headform on the step of adding additional information include: the additional letter of addition at random on transformed three-dimensional headform Breath.

In one embodiment, face key point information include to the relevant internal point information of key point inside face, two Tieing up face key point includes the internal key point of two dimension.

In one embodiment, internal point information includes the position letter of the three-dimensional internal key point in three-dimensional headform Breath, said program code when being run by the processor 630 used execution according to transformed three-dimensional headform and two The projection relation and face key point information between facial image are tieed up, determines the two-dimension human face key point in two-dimension human face image And obtain the location information of two-dimension human face key point as the step of face key point labeled data include: according to projection relation with And the location information of three-dimensional internal key point, determine that three-dimensional internal key point is thrown in the imaging plane where two-dimension human face image The internal key point of the two dimension that shadow obtains and the location information for obtaining key point inside two dimension.

In one embodiment, face key point information includes contour point information relevant to facial contour key point, and two Tieing up face key point includes two-dimensional silhouette key point.

In one embodiment, contour point information include it is in three-dimensional headform, be likely to become facial contour key point Three-D profile point set location information, said program code when being run by the processor 630 used execution according to warp Projection relation and face key point information between the three-dimensional headform of transformation and two-dimension human face image, determine two-dimension human face Two-dimension human face key point in image simultaneously obtains the location information of two-dimension human face key point as face key point labeled data Step comprises determining that the intersection of three-D profile point set and the predefined starting point plane in three-dimensional headform is waited for three-dimensional starting point Selected works；According to projection relation and the location information of three-D profile point set, three-D profile point set and three-dimensional starting point Candidate Set are determined The two-dimensional silhouette point set and two-dimentional starting point Candidate Set and two dimension wheel of acquisition are projected in the imaging plane where two-dimension human face image The position of wide point set and two-dimentional starting point Candidate Set；Profile starting point and profile terminal are selected from two-dimentional starting point Candidate Set；Based on wheel The outer contour of wide starting point, profile terminal and two-dimensional silhouette point set determines facial contour line；And it is determined based on facial contour line Two-dimensional silhouette key point in two-dimension human face image and the location information for obtaining two-dimensional silhouette key point.

In one embodiment, used execution based on face when said program code is run by the processor 630 The step of contour line determines the two-dimensional silhouette key point in two-dimension human face image and obtains the location information of two-dimensional silhouette key point It include: that point in facial contour line, to belong to Given Face position is rejected according to predefined face position kick-out condition, to obtain Obtain facial contour line newly；By new facial contour line equal part；And determine that the Along ent obtained is in two-dimension human face image Two-dimensional silhouette key point and the location information for obtaining two-dimensional silhouette key point.

In one embodiment, used execution based on face when said program code is run by the processor 630 The step of contour line determines the two-dimensional silhouette key point in two-dimension human face image and obtains the location information of two-dimensional silhouette key point It include: by facial contour line equal part；And determine the Along ent obtained for the two-dimensional silhouette key point in two-dimension human face image simultaneously Obtain the location information of two-dimensional silhouette key point.

In one embodiment, used execution to three-dimensional people when said program code is run by the processor 630 The step of head model is converted includes: to carry out rigid body translation to three-dimensional headform；And/or three-dimensional headform is carried out non- Rigid body translation.

In one embodiment, three-dimensional headform includes grid model and texture model.

In one embodiment, it is also used to execute when said program code is run by the processor 630: utilizing two-dimentional people Face image and the training of face key point labeled data are used for the recurrence device of face key point location algorithm.

In addition, according to embodiments of the present invention, additionally providing a kind of storage medium, storing program on said storage Instruction, when described program instruction is run by computer or processor for executing the image processing method of the embodiment of the present invention Corresponding steps, and for realizing the corresponding module in image processing apparatus according to an embodiment of the present invention.The storage medium It such as may include the storage card of smart phone, the storage unit of tablet computer, the hard disk of personal computer, read-only memory (ROM), Erasable Programmable Read Only Memory EPROM (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, Or any combination of above-mentioned storage medium.

In one embodiment, the computer program instructions can to calculate when being run by computer or processor Machine or processor realize each functional module of image processing apparatus according to an embodiment of the present invention, and/or can execute Image processing method according to an embodiment of the present invention.

In one embodiment, the computer program instructions are used to execute following steps at runtime: obtaining initial people Face image；Three-dimensional reconstruction is carried out based on Initial Face image, to obtain three-dimensional headform；It is predefined on three-dimensional headform Face key point information；Three-dimensional headform is converted；Transformed three-dimensional headform is rendered into two-dimension human face figure Picture；And according to the projection relation and face key point letter between transformed three-dimensional headform and two-dimension human face image Breath determines the two-dimension human face key point in two-dimension human face image and obtains the location information of two-dimension human face key point as face pass Key point labeled data.

In one embodiment, the computer program instructions at runtime it is used execute by transformed three-dimensional Headform was rendered into before the step of two-dimension human face image, and the computer program instructions are also used to execute at runtime: Add additional information on transformed three-dimensional headform, wherein additional information include one in background, hair and illumination or It is multinomial.

In one embodiment, the computer program instructions at runtime it is used execute in transformed three-dimensional people The step of additional information is added on head model includes: to add additional information at random on transformed three-dimensional headform.

In one embodiment, internal point information includes the position letter of the three-dimensional internal key point in three-dimensional headform Breath, the computer program instructions at runtime it is used execute according to transformed three-dimensional headform and two-dimension human face figure Projection relation and face key point information as between determine the two-dimension human face key point in two-dimension human face image and obtain two The location information of dimension face key point includes: according in projection relation and three-dimensional as the step of face key point labeled data The location information of portion's key point determines that three-dimensional internal key point projects acquisition in the imaging plane where two-dimension human face image Two-dimentional internal key point and the location information for obtaining key point inside two dimension.

In one embodiment, contour point information include it is in three-dimensional headform, be likely to become facial contour key point Three-D profile point set location information, the computer program instructions at runtime it is used execute according to transformed three The projection relation and face key point information between headform and two-dimension human face image are tieed up, is determined in two-dimension human face image The two-dimension human face key point and location information for obtaining two-dimension human face key point includes: as the step of face key point labeled data The intersection for determining three-D profile point set and the predefined starting point plane in three-dimensional headform is three-dimensional starting point Candidate Set；According to The location information of projection relation and three-D profile point set determines three-D profile point set and three-dimensional starting point Candidate Set in two-dimension human face The two-dimensional silhouette point set and two-dimentional starting point Candidate Set and two-dimensional silhouette point set and two of acquisition are projected in imaging plane where image Tie up the position of starting point Candidate Set；Profile starting point and profile terminal are selected from two-dimentional starting point Candidate Set；Based on profile starting point, profile The outer contour of terminal and two-dimensional silhouette point set determines facial contour line；And two-dimension human face image is determined based on facial contour line In two-dimensional silhouette key point and obtain the location information of two-dimensional silhouette key point.

In one embodiment, the computer program instructions at runtime it is used execute it is true based on facial contour line The step of determining the two-dimensional silhouette key point in two-dimension human face image and obtaining the location information of two-dimensional silhouette key point includes: basis Predefined face position kick-out condition rejects point in facial contour line, to belong to Given Face position, to obtain new people Face contour line；By new facial contour line equal part；And determine that the Along ent obtained is the two-dimensional silhouette in two-dimension human face image Key point and the location information for obtaining two-dimensional silhouette key point.

In one embodiment, the computer program instructions at runtime it is used execute it is true based on facial contour line The step of determining the two-dimensional silhouette key point in two-dimension human face image and obtaining the location information of two-dimensional silhouette key point includes: by people Face contour line equal part；And determine that the Along ent obtained is the two-dimensional silhouette key point in two-dimension human face image and obtains two-dimentional wheel The location information of wide key point.

In one embodiment, the computer program instructions at runtime it is used execute to three-dimensional headform into The step of row transformation includes: to carry out rigid body translation to three-dimensional headform；And/or non-rigid change is carried out to three-dimensional headform It changes.

In one embodiment, the computer program instructions are also used to execute at runtime: utilizing two-dimension human face image The recurrence device of face key point location algorithm is used for the training of face key point labeled data.

Each module in image processing system according to an embodiment of the present invention can pass through reality according to an embodiment of the present invention The processor computer program instructions that store in memory of operation of the electronic equipment of image procossing are applied to realize, or can be with The computer instruction stored in the computer readable storage medium of computer program product according to an embodiment of the present invention is counted Calculation machine is realized when running.

Image processing method and device according to an embodiment of the present invention can be generated quickly and easily a large amount of, each very much Facial image and face key labeled data under kind different scenes.Compared with artificial notation methods, the above method and device effect Rate is high, at low cost, also, using the facial image and face key labeled data that are obtained by the above method and device as training Data can train and obtain the more robust face Keypoint detector of performance.

Although describing example embodiment by reference to attached drawing here, it should be understood that above example embodiment are only exemplary , and be not intended to limit the scope of the invention to this.Those of ordinary skill in the art can carry out various changes wherein And modification, it is made without departing from the scope of the present invention and spiritual.All such changes and modifications are intended to be included in appended claims Within required the scope of the present invention.

In several embodiments provided herein, it should be understood that disclosed device and method can pass through it Its mode is realized.For example, apparatus embodiments described above are merely indicative, for example, the division of the unit, only Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied Another equipment is closed or is desirably integrated into, or some features can be ignored or not executed.

In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this specification.

Similarly, it should be understood that in order to simplify the present invention and help to understand one or more of the various inventive aspects, To in the description of exemplary embodiment of the present invention, each feature of the invention be grouped together into sometimes single embodiment, figure, Or in descriptions thereof.However, the method for the invention should not be construed to reflect an intention that i.e. claimed The present invention claims features more more than feature expressly recited in each claim.More precisely, such as corresponding power As sharp claim reflects, inventive point is that the spy of all features less than some disclosed single embodiment can be used Sign is to solve corresponding technical problem.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in this specific Embodiment, wherein each, the claims themselves are regarded as separate embodiments of the invention.

It will be understood to those skilled in the art that any combination pair can be used other than mutually exclusive between feature All features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed any method Or all process or units of equipment are combined.Unless expressly stated otherwise, this specification (is wanted including adjoint right Ask, make a summary and attached drawing) disclosed in each feature can be replaced with an alternative feature that provides the same, equivalent, or similar purpose.

In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention Within the scope of and form different embodiments.For example, in detail in the claims, embodiment claimed it is one of any Can in any combination mode come using.

Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice Microprocessor or digital signal processor (DSP) realize some moulds in image processing apparatus according to an embodiment of the present invention The some or all functions of block.The present invention is also implemented as a part or complete for executing method as described herein The program of device (for example, computer program and computer program product) in portion.It is such to realize that program of the invention can store On a computer-readable medium, it or may be in the form of one or more signals.Such signal can be from internet Downloading obtains on website, is perhaps provided on the carrier signal or is provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame Claim.

The above description is merely a specific embodiment or to the explanation of specific embodiment, protection of the invention Range is not limited thereto, and anyone skilled in the art in the technical scope disclosed by the present invention, can be easily Expect change or replacement, should be covered by the protection scope of the present invention.Protection scope of the present invention should be with claim Subject to protection scope.

Claims

1. a kind of image processing method, comprising:

Obtain Initial Face image；

Three-dimensional reconstruction is carried out based on the Initial Face image, to obtain three-dimensional headform；

The predefined face key point information on the three-dimensional headform；

The three-dimensional headform is converted；

Transformed three-dimensional headform is rendered into two-dimension human face image；And

According to the projection relation and the face between the transformed three-dimensional headform and the two-dimension human face image Key point information determines the two-dimension human face key point in the two-dimension human face image and obtains the position of the two-dimension human face key point Confidence breath is used as face key point labeled data.

2. image processing method as described in claim 1, wherein transformed three-dimensional headform is rendered into two described Before tieing up facial image, described image processing method further include:

Add additional information on the transformed three-dimensional headform, wherein the additional information include background, hair and It is one or more in illumination.

3. image processing method as claimed in claim 2, wherein described to be added on the transformed three-dimensional headform Additional information includes:

The additional information is added at random on the transformed three-dimensional headform.

4. image processing method as described in claim 1, wherein the face key point information includes and face inside is crucial The relevant internal point information of point, the two-dimension human face key point include the internal key point of two dimension.

5. image processing method as claimed in claim 4, wherein the internal point information includes in the three-dimensional headform Three-dimensional internal key point location information,

The projection relation according between the transformed three-dimensional headform and the two-dimension human face image and described Face key point information determines the two-dimension human face key point in the two-dimension human face image and obtains the two-dimension human face key point Location information include: as face key point labeled data

According to the projection relation and the location information of the three-dimensional internal key point, determine that the three-dimensional internal key point exists The two dimension inside key point of projection acquisition in imaging plane where the two-dimension human face image simultaneously obtains the internal pass of the two dimension The location information of key point.

6. such as image processing method described in any one of claim 1 to 5, wherein the face key point information includes and people The relevant contour point information of face profile key point, the two-dimension human face key point include two-dimensional silhouette key point.

7. image processing method as claimed in claim 6, wherein the contour point information includes in the three-dimensional headform , the location information of the three-D profile point set for being likely to become facial contour key point,

Determine that the three-D profile point set and the intersection of the predefined starting point plane in the three-dimensional headform rise to be three-dimensional Point Candidate Set；

According to the projection relation and the location information of the three-D profile point set, the three-D profile point set and described is determined Three-dimensional starting point Candidate Set projects the two-dimensional silhouette point set and two dimension of acquisition in the imaging plane where the two-dimension human face image The position of starting point Candidate Set and the two-dimensional silhouette point set and the two-dimentional starting point Candidate Set；

Profile starting point and profile terminal are selected from the two-dimentional starting point Candidate Set；

Facial contour line is determined based on the outer contour of the profile starting point, the profile terminal and the two-dimensional silhouette point set； And

The two-dimensional silhouette key point in the two-dimension human face image is determined based on the facial contour line and obtains the two dimension wheel The location information of wide key point.

8. image processing method as claimed in claim 7, wherein described to determine the two-dimentional people based on the facial contour line Two-dimensional silhouette key point in the face image and location information for obtaining the two-dimensional silhouette key point includes:

Point in the facial contour line, to belong to Given Face position is rejected according to predefined face position kick-out condition, To obtain new facial contour line；

By the new facial contour line equal part；And

Determine that the Along ent obtained is the two-dimensional silhouette key point in the two-dimension human face image and obtains the two-dimensional silhouette pass The location information of key point.

9. image processing method as claimed in claim 7, wherein described to determine the two-dimentional people based on the facial contour line Two-dimensional silhouette key point in the face image and location information for obtaining the two-dimensional silhouette key point includes:

By the facial contour line equal part；And

10. image processing method as described in claim 1, wherein described to carry out transformation to the three-dimensional headform and include:

Rigid body translation is carried out to the three-dimensional headform；And/or

Non-rigid transformation is carried out to the three-dimensional headform.

11. image processing method as described in claim 1, wherein the three-dimensional headform includes grid model and texture Model.

12. image processing method as described in claim 1, wherein described image processing method further include:

Using the two-dimension human face image and face key point labeled data training for face key point location algorithm Return device.

13. a kind of image processing apparatus, comprising:

Initial pictures obtain module, for obtaining Initial Face image；

Three-dimensional reconstruction module, for carrying out three-dimensional reconstruction based on the Initial Face image, to obtain three-dimensional headform；

Predefined module, for the predefined face key point information on the three-dimensional headform；

Conversion module, for being converted to the three-dimensional headform；

Rendering module, for transformed three-dimensional headform to be rendered into two-dimension human face image；And

Key point determining module, for according to the throwing between the transformed three-dimensional headform and the two-dimension human face image Shadow relationship and the face key point information determine described in the two-dimension human face key point in the two-dimension human face image and acquisition The location information of two-dimension human face key point is as face key point labeled data.

14. a kind of image processing system, including processor and memory, wherein be stored with computer program in the memory Instruction, for executing following steps when the computer program instructions are run by the processor:

Obtain Initial Face image；

The predefined face key point information on the three-dimensional headform；

The three-dimensional headform is converted；

15. a kind of storage medium stores program instruction on said storage, described program instruction is at runtime for holding Row following steps:

Obtain Initial Face image；

The predefined face key point information on the three-dimensional headform；

The three-dimensional headform is converted；