CN109671142A - A kind of intelligence makeups method and intelligent makeups mirror - Google Patents
A kind of intelligence makeups method and intelligent makeups mirror Download PDFInfo
- Publication number
- CN109671142A CN109671142A CN201811402312.XA CN201811402312A CN109671142A CN 109671142 A CN109671142 A CN 109671142A CN 201811402312 A CN201811402312 A CN 201811402312A CN 109671142 A CN109671142 A CN 109671142A
- Authority
- CN
- China
- Prior art keywords
- model
- makeups
- face
- opengl
- mirror
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000000694 effects Effects 0.000 claims abstract description 17
- 238000009877 rendering Methods 0.000 claims abstract description 15
- 238000003384 imaging method Methods 0.000 claims abstract description 7
- 230000000007 visual effect Effects 0.000 claims abstract description 6
- 238000001514 detection method Methods 0.000 claims abstract description 4
- 238000011161 development Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 14
- 239000000463 material Substances 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000007637 random forest analysis Methods 0.000 description 5
- 210000003128 head Anatomy 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000306 recurrent effect Effects 0.000 description 4
- 238000012706 support-vector machine Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 210000004709 eyebrow Anatomy 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000000744 eyelid Anatomy 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000000700 radioactive tracer Substances 0.000 description 2
- 229910001069 Ti alloy Inorganic materials 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- -1 color number Substances 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- A45D42/00—Hand, pocket, or shaving mirrors
- A45D42/08—Shaving mirrors
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- A45D42/00—Hand, pocket, or shaving mirrors
- A45D42/08—Shaving mirrors
- A45D42/10—Shaving mirrors illuminated
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0641—Shopping interfaces
- G06Q30/0643—Graphical representation of items or shoppers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Graphics (AREA)
- Marketing (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention discloses a kind of intelligent makeups method and intelligent makeups mirror, and makeups method includes: acquisition current video frame；Whether detection current video frame has face, backout feature dot position information；According to characteristic point position information, generate and the consistent 3D model of active user's face feature；Using source of parallel light rectangular projection, generates 3D model in video frame imaging position and project, visual difference is generated, so that 3D model shelters from video image frame face completely from OpenGL space observation；Makeups product is rendered into the corresponding characteristic point position of 3D model using OpenGL Shader, and sets transparent for 3D model rendering effect；According to the specific two o'clock positional distance of face, calculate user and OpenGL point of observation distance, 3D scaling of model ratio is calculated according to this distance, 3D model is deflected in real time according to the spatial deflection angle information that face tracking returns, generates the real-time examination adornment effect under human face action.The present invention, which realizes, does not influence to try adornment effect improving performance simultaneously.
Description
Technical field
The present invention relates to field of artificial intelligence, and in particular to a kind of intelligence makeups method and intelligent makeups mirror.
Background technique
Existing virtual examination adornment technical software is based on face tracking and rendering engine is realized, face tracking obtains face and closes
After key point information in such a way that the equal third parties' rendering engine of Unity or illusory carries out according to key point position rendering different product
It realizes.Use structure for " face tracking+third party's engine renders textures ", hardware is shown based on the Android touch-control of ordinary screen
Screen, additional ordinary white light compensating lamp, external power supply lamp realize examination adornment mirror entity.
At present technology there are the shortcomings that:
1) face tracking low efficiency, single frames tracking needs time-consuming 20ms under android system, RK3288 chip environments
Left and right, and the face tracking technology that virtual examination adornment uses at present is all based on 68,83 or 106 key points, wherein very much
Key point is that do not have effective, actually only needs mouth, eyes, eyebrow, the key point of the face such as nose, extra point
It will cause the waste of performance.
2) third party's rendering engine causes APP backpack body excessive, runs Caton.The engines such as Unity are because for convenience of production particle
Special efficacy etc. is widely used in big game production, and virtual adornment technology of trying is not related to complicated animation effect, depends on third party
Engine will cause operation examination adornment APP Caton sense as loading large-scale network game, and load also can be extremely slow.
3) adornment technology is much virtually tried in market at present, and examination adornment special efficacy uses puppet 3D method, 2D textures are rendered to picture, are added
Specific deflection angle is gone up, such way is true to nature not as good as 3D, in no time and when head rotation has and is not bonded and such as patch
The same effect of paper lacks actual use meaning.
4) ordinary screen does not have mirror surface speciality, and examination adornment mirror when common mirror surface, is had color difference and brightness by user
On difference, as being looked in the mirror with mobile phone screen.
Summary of the invention
Goal of the invention: present invention aims in view of the deficiencies of the prior art, provide a kind of intelligent makeups method and intelligence
Makeups mirror does not influence to try adornment effect improving performance simultaneously.
Technical solution: intelligence makeups method of the present invention includes the following steps:
(1) current video frame is acquired；
(2) whether detection current video frame has face: if testing result is yes, Face datection tracking backout feature point position
Information；If testing result be it is no, continue to detect；
(3) according to characteristic point position information, real time fusion deformation operation is carried out to master pattern, is generated and active user's face
The consistent 3D model of portion's feature；
(4) 3D model space position is placed in OpenGL space observation position and video frame imaging position middle, benefit
With source of parallel light rectangular projection, generates 3D model in video frame imaging position and project, visual difference is generated, so that from OpenGL
Space observation, 3D model shelter from video image frame face completely；
(5) makeups product is rendered into the corresponding characteristic point position of 3D model using OpenGL Shader, and by 3D mould
Type rendering effect is set as transparent；
(6) according to the specific two o'clock positional distance of face, user and OpenGL point of observation distance are calculated, is calculated according to this distance
3D scaling of model ratio deflects 3D model according to the spatial deflection angle information that face tracking returns in real time, generates face
Real-time examination adornment effect under movement.
Further, the identification of face key point is carried out to the face range detected in the step (2), is chased after by face
Track obtains 68 characteristic point position information.
Further, fusion deformation operation refers to the movement for establishing 12 groups of transformation ranges 0~1 in the step (3), moves
Make minimum to correspond to 3D model deformation extent to be 0, it is 1 that movement, which utmostly corresponds to 3D model transformation degree,；It is chased after according to face
68 characteristic point position information that track obtains place corresponding key point label on 3D model, guarantee face tracking point and 3D
Model key point label corresponds.
Further, the OpenGL Shader rendering makeups product includes the brand, material, color number, sample of makeups product
Formula, render transparency are set as controllable.
The present invention also provides the intelligent makeups mirror for realizing above-mentioned intelligent makeups method, including shell, specular screen, take the photograph
As head, bracket, light-supplementing system, development board, the specular screen is mounted on surface of shell, and the bracket is supported on back side of shell,
The camera and light-supplementing system are arranged on specular screen and are connected with the intracorporal development board of shell；The camera is for acquiring
Current video frame is simultaneously transmitted to development board, and the development board is stored with computer program for executing the step (2) to step
(6)。
Further, the computer program of the development board storage further includes teaching pattern, online store mode, light filling mirror
Surface model；
Intelligent makeups mirror chooses whether to open teaching pattern when opening first: if selection is, the small window of specular screen plays religion
Video is learned, online store mode is opened after finishing playing, pop-up window provides the purchase of instructional video auxiliary products for user；If
Select no, whether the prompt of intelligent makeups mirror opens examination adornment mode；
If examination adornment mode is opened in selection, the step (1) to (6) is executed, if examination adornment mode is not opened in selection, enters and mends
Light microscopic surface model, and recycle the judgement for being made whether to open examination adornment mode
The utility model has the advantages that compared with the prior art, the advantages of the present invention are as follows: intelligence makeups method and intelligence provided by the invention
Can makeups mirror, overcome traditional algorithm disadvantage, inexpensive small size high-performance made it possible using deep learning；It is unnecessary to abandon
Point, face tracking use 68 key points, do not need 106 key points used as present market prior art, can be improved to list
Frame tracks time-consuming 2ms, is most fastly compared with the existing technology 20ms in the every frame rate of RK3288 mainboard, and 10 times of performance boost.
Rendering aspect is rendered based on third parties' engines such as Unity3D or illusory in the prior art, using pseudo- 3D method
Carry out textures；The application abandons third party's engine, carries out 3D real-time rendering using the raw OpenGL method in source, improving performance, and
Cooperation fusion deformation technology, single frames realize that " pinching face " time is lower than 0.5ms.
It, can be by left eye eye is wide and right eye eye is wide, the left corners of the mouth raises up and the right corners of the mouth raises up, left nose because face has symmetry
Wing position and right wing of nose position merge, unified using left side face as calculation basis, can be reduced to 15 groups of fusion deformations present
12 groups.
The Face datection tracer technique that the application uses, is based on deep learning, precisely detects face and to 68 key of face
Characteristic point carries out real-time tracing, and precision is in LFPW, and for AFW test set up to 99.5%, cost reduces by 75%.It is rendered and is calculated by 3D
Method real-time virtual tries adornment, and OpenGL makes to try adornment algorithm in Android RK3288 development board operation smoothness from bottom layer treatment image.
Specular screen technology and exploitation version fusion are enabled household mirror surface material and screen compatible, thrown by hardware design
Tradition examination adornment mirror is abandoned using ordinary screen as material, has more the sense of reality.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of the intelligent makeups mirror of the present invention；
Fig. 2 is the flow chart of the intelligent makeups method of the present invention；
Fig. 3 is the flow chart of face tracking methods of the present invention；
Fig. 4 is characterized a location drawing；
Fig. 5 is functional block diagram of the invention.
Specific embodiment
Be described in detail below by technical solution of the present invention, but protection scope of the present invention be not limited to it is described
Embodiment.
Embodiment 1: intelligent makeups mirror as shown in Figure 1, including Android RK3288 development board, sieve's 1080P skill high-definition camera
First 2,15 cun of specular screens 1, titanium alloy metal dorsal shield and brackets 4, specular screen are mounted on surface of shell, and bracket is supported on shell
The back side, camera and light-supplementing system 3 are arranged on specular screen and are connected with the intracorporal development board of shell；Camera is worked as acquiring
Preceding video frame is simultaneously transmitted to development board, and development board is stored with computer program for teaching pattern, online store mode, light filling mirror
Surface model, examination adornment mode.
As shown in Fig. 2, may choose whether to open teaching pattern, if not, that is just when user's routine use intelligence makeups mirror
It is the common mirror surface of one side, normal use；If selection is, the small red teaching pattern of window net can be entered, correlation may be selected in user
Red or specific dressing teaching is netted, and obtains and recommends Related product purchasing channel.In the process, user also may choose whether to open
Virtual examination adornment mode, if opening virtual examination adornment mode, using face tracking algorithm and AR Rendering algorithms by virtual dressing effect
It is rendered in real time in face relevant position.If user opens makeups teaching and virtual examination two modes of adornment simultaneously, meeting exists
It is supplied to this practical dressing effect of user at the end of makeup and nets red standard dressing Contrast on effect score and discriminant amendment.
In whole flow process, user can select to open light filling mirror mode, can simulate the light intensity of different scenes, colour temperature in real time
Deng.
It tries adornment mode and real-time pictures face information is obtained according to camera first, be tracked 68 key feature points, then
It is complete by the 3D model of makeups product using visual difference real-time rendering to screen face corresponding position using graphics Rendering
At examination adornment process.It runs on domestic Android exploitation mainboard RK3288 development board, this set algorithm flow in conjunction with specular screen
Technology is fused in daily mirror surface, makes New Generation of Intelligent household new product from bottom.
A. algoritic module
A) by face tracking 68 characteristic point positions of practical acquisition, and the characteristic information of measurement different faces, as between eye
Away from, mouth and nose away from face is wide, and the cheek is wide etc., as shown in Figure 4.
The application is made using raw+12 group fusion deformation technology (as shown in the table) of OpenGL+3D standard human head model in source
The makeups effect of complete 3D carries out algorithm optimization for deformation group number is merged using the symmetry of face, a group number is reduced to 12 groups.
Table 1 merges deformation technology
| Fusion deformation number | Merge deformation position | Fusion deformation key point group |
| 1 | Upperlip distance | 62-66 |
| 2 | Chin | 66-8 |
| 3 | Lower eyelid distance on left eye | 37-41 |
| 4 | Lower eyelid distance on right eye | 44-46 |
| 5 | Eye spacing | 39-42 |
| 6 | Eye is wide | 36-39 |
| 7 | Left eyebrow height | 20-40 |
| 8 | Right eyebrow height | 23-47 |
| 9 | Nose length | 27-33 |
| 10 | The wing of nose | 31-39 |
| 11 | The corners of the mouth raises up | 48-8 |
| 12 | Mouth is wide | 48-54 |
B) 3D headform automatically generates: the face information data obtained according to previous step, special using FBX type 3D model
Some fusion deformation technologies carry out real-time deformation operation to master pattern, generate and the consistent 3D mould of active user's face feature
Type.
C) 3D model space position is placed in OpenGL space observation position and video frame imaging position middle, utilized
Source of parallel light rectangular projection generates 3D model in video frame imaging position and projects, generates visual difference.At this time from the space OpenGL
Observation sees that 3D model is to block video frame face completely.
D) makeups product matches: according to the different position of 3D headform, different beauty is rendered using OpenGL Shader
Adornment product material, and by 3D headform's material setting rendering effect be it is transparent, can thus be seen by visual difference in step c
To the fusion of makeups product and video frame face, makeup effect is realized.
E) according to face tracking point, wherein specific two o'clock positional distance, calculates user and OpenGL observation point distance (face
With makeups mirror distance), 3D scaling of model ratio is calculated according to this distance.According to face tracking return spatial deflection angle angle,
Three dimensional angle is carried out to model, realization is turned one's head, and waits the makeups effect under movements in no time.
B. module is designed
A) 3D model auto Deformation technology: FBX type 3D model has the function of fusion deformation, makes 12 groups of variable ranges and exists
The fusion of 0-1 deforms, and such as opens and shuts up, and it is maximum value 1 that deformation parameter is merged when opening one's mouth, and is 0 when shutting up.So that program calculates currently
Every group of deformation extent carrys out Controlling model deformation.
B) it the corresponding 68 key point technologies of 3D model: is put on the 3D model of FBX according to 68 key points of face tracking
The key point label of corresponding serial number is set, is corresponded with reaching face tracking point and model mark point.
C) makeups material design: generation different brands, the makeups product material of different styles, color number, pattern are consistent,
Transparency is that program is controllable, to adjust makeup concentration.
Face datection tracer technique uses deep learning algorithm in the application:
A) Recognition with Recurrent Neural Network (RNN) provides core technology and is trained data set
Recognition with Recurrent Neural Network (RNN) is a kind of neural network for being used for processing sequence data, can extend to longer sequence
Column, can also handle the sequence of variable-length.Training set to 100,000 samples for having marked good person's face key point absolute coordinate
When being trained, it then follows exist between hidden unit and be connected by circulation, but read and generate setting of individually exporting after entire sequence
Meter mode.By the face key point data extraction characteristic value of training set sample and after serializing is handled, by designed
Recognition with Recurrent Neural Network structure obtains final training pattern；
B) random forest method provides local binary characteristic processing
Random forest is made of multi-level multiple groups binary decision tree.By sample data training pattern and utilize mould
During type identifies face key point, need to handle extraction and the normalizing of local binary feature using random forest data structure
Change operation.When training, using existing face key point absolute coordinate, the pixel value of respective coordinates in image is extracted,
Obtain the data information of all nodes by the data structure of random forest, be supplied to Recognition with Recurrent Neural Network use, and finally with
The form of regression matrix is stored in model file；When using model identification face key point, according to existing recurrence square
Battle array obtains the face key point information rough with detection picture, and excellent by the calculating of model and the decision of random forest structure
Change the face key point information being finally identified to.
C) support vector machines (SVM) algorithm supports cross-platform operation
The algorithm is used for during identification face in real time, for determining whether is face key point that present frame recognizes
There is elegant or big error, if identification error is larger, next frame returns restarting Face datection to correct.In training
Increase error decision bits during model, in the training process to incorporate SVM model in overall model；?
When identification, the face key point information identified can be sent into SVM model, be determined for error.
As shown in figure 5, the present invention provides dressing recommendation simultaneously for user, real-time virtual tries adornment, makeups teaching, makeup are complete
The functions such as degree is analyzed, daily skin quality is analyzed, is placed an order online；Light filling lamp ring provide different scenes light simulation, allow makeup skin care more
The easy science of intelligence.Based on user behavior big data analysis, carries out launching recommended advertisements precisely for makeups products cooperating quotient, will close
Suitable Products Show to pair people.
As described above, must not be explained although the present invention has been indicated and described referring to specific preferred embodiment
For the limitation to invention itself.It without prejudice to the spirit and scope of the invention as defined in the appended claims, can be right
Various changes can be made in the form and details for it.
Claims (6)
1. a kind of intelligence makeups method, which comprises the steps of:
(1) current video frame is acquired；
(2) whether detection current video frame has face: if testing result is yes, Face datection tracking backout feature point confidence
Breath；If testing result be it is no, continue to detect；
(3) according to characteristic point position information, real time fusion deformation operation is carried out to master pattern, generates and active user face is special
Levy consistent 3D model；
(4) 3D model space position is placed in OpenGL space observation position and video frame imaging position middle, using flat
Line light source rectangular projection generates 3D model in video frame imaging position and projects, visual difference generated, so that from OpenGL Space View
It examines, 3D model shelters from video image frame face completely；
(5) makeups product is rendered into the corresponding characteristic point position of 3D model using OpenGL Shader, and by 3D model rendering
Effect is set as transparent；
(6) according to the specific two o'clock positional distance of face, user and OpenGL point of observation distance are calculated, 3D mould is calculated according to this distance
Type scaling deflects 3D model according to the spatial deflection angle information that face tracking returns in real time, generates human face action
Under real-time examination adornment effect.
2. intelligence makeups method according to claim 1, it is characterised in that: to the face detected in the step (2)
Range carries out the identification of face key point, obtains 68 characteristic point position information by face tracking.
3. intelligence makeups method according to claim 2, it is characterised in that: fusion deformation operation is in the step (3)
Finger establishes movement of 12 groups of transformation ranges 0 ~ 1, and it is 0 that movement minimum, which corresponds to 3D model deformation extent, and movement is utmostly
Corresponding 3D model transformation degree is 1；According to 68 characteristic point position information that face tracking obtains, places and correspond on 3D model
Key point label, guarantee face tracking point and 3D model key point label correspond.
4. intelligence makeups method according to claim 1, it is characterised in that: the OpenGL Shader rendering makeups produce
Product include the brand, material, color number, pattern of makeups product, and render transparency is set as controllable.
5. a kind of intelligent makeups mirror for realizing any intelligent makeups method of Claims 1-4, it is characterised in that: packet
Shell, specular screen, camera, bracket, light-supplementing system, development board are included, the specular screen is mounted on surface of shell, the branch
Frame is supported on back side of shell, and the camera and light-supplementing system are arranged on specular screen and are connected with the intracorporal development board of shell；
For the camera for acquiring current video frame and being transmitted to development board, the development board is stored with computer program for executing
The step (2) is to step (6).
6. intelligence makeups mirror according to claim 5, it is characterised in that: the computer program of the development board storage also wraps
Include teaching pattern, online store mode, light filling mirror mode；
Intelligent makeups mirror chooses whether to open teaching pattern when opening first: if selection is, the small window of specular screen plays teaching view
Frequently, online store mode is opened after finishing playing, pop-up window provides the purchase of instructional video auxiliary products for user；If selection
No, whether intelligent makeups mirror prompt opens examination adornment mode；
If examination adornment mode is opened in selection, the step (1) to (6) is executed, if examination adornment mode is not opened in selection, enters light filling mirror
Surface model, and recycle the judgement for being made whether to open examination adornment mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811402312.XA CN109671142B (en) | 2018-11-23 | 2018-11-23 | Intelligent cosmetic method and intelligent cosmetic mirror |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811402312.XA CN109671142B (en) | 2018-11-23 | 2018-11-23 | Intelligent cosmetic method and intelligent cosmetic mirror |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109671142A true CN109671142A (en) | 2019-04-23 |
| CN109671142B CN109671142B (en) | 2023-08-04 |
Family
ID=66142158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811402312.XA Active CN109671142B (en) | 2018-11-23 | 2018-11-23 | Intelligent cosmetic method and intelligent cosmetic mirror |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109671142B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110069716A (en) * | 2019-04-29 | 2019-07-30 | 清华大学深圳研究生院 | A kind of makeups recommended method, system and computer readable storage medium |
| CN110347134A (en) * | 2019-07-29 | 2019-10-18 | 南京图玩智能科技有限公司 | A kind of AI intelligence aquaculture specimen discerning method and cultivating system |
| CN110782309A (en) * | 2019-09-23 | 2020-02-11 | 苏州浩哥文化传播有限公司 | Cosmetic recommendation system based on light image analysis and working method thereof |
| CN111553220A (en) * | 2020-04-21 | 2020-08-18 | 海信集团有限公司 | Intelligent device and data processing method |
| CN111839124A (en) * | 2019-04-26 | 2020-10-30 | 京东方科技集团股份有限公司 | Intelligent mirror |
| CN112015934A (en) * | 2020-08-27 | 2020-12-01 | 华南理工大学 | Intelligent hair style recommendation method, device and system based on neural network and Unity |
| CN112418132A (en) * | 2020-12-01 | 2021-02-26 | 江西台德智慧科技有限公司 | Intelligent makeup mirror and intelligent makeup system |
| CN112800884A (en) * | 2021-01-15 | 2021-05-14 | 深圳市鑫海创达科技有限公司 | Intelligent auxiliary method based on cosmetic mirror |
| CN112932055A (en) * | 2020-01-08 | 2021-06-11 | 广东沁彩化妆品科技有限公司 | Intelligent lipstick |
| CN113468934A (en) * | 2020-04-30 | 2021-10-01 | 海信集团有限公司 | Symmetry detection method and intelligent equipment |
| CN113468932A (en) * | 2020-04-28 | 2021-10-01 | 海信集团有限公司 | Intelligent mirror and makeup teaching method |
| CN113780047A (en) * | 2021-01-11 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Virtual makeup trying method and device, electronic equipment and storage medium |
| CN114466218A (en) * | 2022-02-18 | 2022-05-10 | 广州方硅信息技术有限公司 | Live video character tracking method, device, equipment and storage medium |
| WO2022232404A1 (en) * | 2021-04-30 | 2022-11-03 | L'oreal | Integrated cosmetic design applicator |
| US11544876B2 (en) | 2021-04-30 | 2023-01-03 | L'oreal | Integrated cosmetic design applicator |
| FR3125949A1 (en) * | 2021-08-03 | 2023-02-10 | L'oreal | integrated cosmetic drawing applicator |
| CN117834838A (en) * | 2024-03-05 | 2024-04-05 | 成都白泽智汇科技有限公司 | Binocular vision detection projection equipment and detection projection method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101371272A (en) * | 2006-01-17 | 2009-02-18 | 株式会社资生堂 | Makeup simulation system, makeup simulation device, makeup simulation method and makeup simulation program |
| CN101763636A (en) * | 2009-09-23 | 2010-06-30 | 中国科学院自动化研究所 | Method for tracing position and pose of 3D human face in video sequence |
| CN102262788A (en) * | 2010-05-24 | 2011-11-30 | 上海一格信息科技有限公司 | Method and device for processing interactive makeup information data of personal three-dimensional (3D) image |
| CN104463938A (en) * | 2014-11-25 | 2015-03-25 | 福建天晴数码有限公司 | Three-dimensional virtual make-up trial method and device |
| CN105556508A (en) * | 2013-08-04 | 2016-05-04 | 艾斯适配有限公司 | Devices, systems and methods of virtualizing a mirror |
| CN106303706A (en) * | 2016-08-31 | 2017-01-04 | 杭州当虹科技有限公司 | The method realizing following visual angle viewing virtual reality video with leading role based on face and item tracking |
| CN106920274A (en) * | 2017-01-20 | 2017-07-04 | 南京开为网络科技有限公司 | Mobile terminal 2D key points rapid translating is the human face model building of 3D fusion deformations |
| CN107820591A (en) * | 2017-06-12 | 2018-03-20 | 美的集团股份有限公司 | Control method, controller, Intelligent mirror and computer-readable recording medium |
| CN107958479A (en) * | 2017-12-26 | 2018-04-24 | 南京开为网络科技有限公司 | A kind of mobile terminal 3D faces augmented reality implementation method |
| CN108171143A (en) * | 2017-12-25 | 2018-06-15 | 深圳市美丽控电子商务有限公司 | Makeups method, smart mirror and storage medium |
| CN108805090A (en) * | 2018-06-14 | 2018-11-13 | 广东工业大学 | A kind of virtual examination cosmetic method based on Plane Gridding Model |
-
2018
- 2018-11-23 CN CN201811402312.XA patent/CN109671142B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101371272A (en) * | 2006-01-17 | 2009-02-18 | 株式会社资生堂 | Makeup simulation system, makeup simulation device, makeup simulation method and makeup simulation program |
| CN101763636A (en) * | 2009-09-23 | 2010-06-30 | 中国科学院自动化研究所 | Method for tracing position and pose of 3D human face in video sequence |
| CN102262788A (en) * | 2010-05-24 | 2011-11-30 | 上海一格信息科技有限公司 | Method and device for processing interactive makeup information data of personal three-dimensional (3D) image |
| CN105556508A (en) * | 2013-08-04 | 2016-05-04 | 艾斯适配有限公司 | Devices, systems and methods of virtualizing a mirror |
| CN104463938A (en) * | 2014-11-25 | 2015-03-25 | 福建天晴数码有限公司 | Three-dimensional virtual make-up trial method and device |
| CN106303706A (en) * | 2016-08-31 | 2017-01-04 | 杭州当虹科技有限公司 | The method realizing following visual angle viewing virtual reality video with leading role based on face and item tracking |
| CN106920274A (en) * | 2017-01-20 | 2017-07-04 | 南京开为网络科技有限公司 | Mobile terminal 2D key points rapid translating is the human face model building of 3D fusion deformations |
| CN107820591A (en) * | 2017-06-12 | 2018-03-20 | 美的集团股份有限公司 | Control method, controller, Intelligent mirror and computer-readable recording medium |
| CN108171143A (en) * | 2017-12-25 | 2018-06-15 | 深圳市美丽控电子商务有限公司 | Makeups method, smart mirror and storage medium |
| CN107958479A (en) * | 2017-12-26 | 2018-04-24 | 南京开为网络科技有限公司 | A kind of mobile terminal 3D faces augmented reality implementation method |
| CN108805090A (en) * | 2018-06-14 | 2018-11-13 | 广东工业大学 | A kind of virtual examination cosmetic method based on Plane Gridding Model |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111839124A (en) * | 2019-04-26 | 2020-10-30 | 京东方科技集团股份有限公司 | Intelligent mirror |
| CN110069716A (en) * | 2019-04-29 | 2019-07-30 | 清华大学深圳研究生院 | A kind of makeups recommended method, system and computer readable storage medium |
| CN110069716B (en) * | 2019-04-29 | 2022-03-18 | 清华大学深圳研究生院 | Beautiful makeup recommendation method and system and computer-readable storage medium |
| CN110347134A (en) * | 2019-07-29 | 2019-10-18 | 南京图玩智能科技有限公司 | A kind of AI intelligence aquaculture specimen discerning method and cultivating system |
| CN110782309A (en) * | 2019-09-23 | 2020-02-11 | 苏州浩哥文化传播有限公司 | Cosmetic recommendation system based on light image analysis and working method thereof |
| CN112932055A (en) * | 2020-01-08 | 2021-06-11 | 广东沁彩化妆品科技有限公司 | Intelligent lipstick |
| CN111553220A (en) * | 2020-04-21 | 2020-08-18 | 海信集团有限公司 | Intelligent device and data processing method |
| CN113468932A (en) * | 2020-04-28 | 2021-10-01 | 海信集团有限公司 | Intelligent mirror and makeup teaching method |
| CN113468934A (en) * | 2020-04-30 | 2021-10-01 | 海信集团有限公司 | Symmetry detection method and intelligent equipment |
| CN112015934A (en) * | 2020-08-27 | 2020-12-01 | 华南理工大学 | Intelligent hair style recommendation method, device and system based on neural network and Unity |
| CN112015934B (en) * | 2020-08-27 | 2022-07-26 | 华南理工大学 | Intelligent hair style recommendation method, device and system based on neural network and Unity |
| CN112418132A (en) * | 2020-12-01 | 2021-02-26 | 江西台德智慧科技有限公司 | Intelligent makeup mirror and intelligent makeup system |
| CN113780047A (en) * | 2021-01-11 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Virtual makeup trying method and device, electronic equipment and storage medium |
| CN112800884A (en) * | 2021-01-15 | 2021-05-14 | 深圳市鑫海创达科技有限公司 | Intelligent auxiliary method based on cosmetic mirror |
| CN112800884B (en) * | 2021-01-15 | 2024-04-30 | 深圳市鑫海创达科技有限公司 | Intelligent auxiliary method based on cosmetic mirror |
| WO2022232404A1 (en) * | 2021-04-30 | 2022-11-03 | L'oreal | Integrated cosmetic design applicator |
| US11544876B2 (en) | 2021-04-30 | 2023-01-03 | L'oreal | Integrated cosmetic design applicator |
| FR3125949A1 (en) * | 2021-08-03 | 2023-02-10 | L'oreal | integrated cosmetic drawing applicator |
| CN114466218A (en) * | 2022-02-18 | 2022-05-10 | 广州方硅信息技术有限公司 | Live video character tracking method, device, equipment and storage medium |
| CN114466218B (en) * | 2022-02-18 | 2024-04-23 | 广州方硅信息技术有限公司 | Live video character tracking method, device, equipment and storage medium |
| CN117834838A (en) * | 2024-03-05 | 2024-04-05 | 成都白泽智汇科技有限公司 | Binocular vision detection projection equipment and detection projection method |
| CN117834838B (en) * | 2024-03-05 | 2024-05-14 | 成都白泽智汇科技有限公司 | Binocular vision detection projection equipment and detection projection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109671142B (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109671142A (en) | A kind of intelligence makeups method and intelligent makeups mirror | |
| Wood et al. | Rendering of eyes for eye-shape registration and gaze estimation | |
| US10573091B2 (en) | Systems and methods to create a virtual object or avatar | |
| CN105556508B (en) | The devices, systems, and methods of virtual mirror | |
| US11900569B2 (en) | Image-based detection of surfaces that provide specular reflections and reflection modification | |
| CN101779218B (en) | Makeup simulation system, makeup simulation apparatus, makeup simulation method, and makeup simulation program | |
| CN100468463C (en) | Method,apparatua and computer program for processing image | |
| CN101339606B (en) | Human face critical organ contour characteristic points positioning and tracking method and device | |
| CN108460398B (en) | Image processing method and device and cloud processing equipment | |
| CN109690617A (en) | System and method for digital vanity mirror | |
| CN109784281A (en) | Products Show method, apparatus and computer equipment based on face characteristic | |
| JP2019510297A (en) | Virtual try-on to the user's true human body model | |
| CN110363867A (en) | Virtual dress up system, method, equipment and medium | |
| CN109410298A (en) | A kind of production method and expression shape change method of dummy model | |
| CN109377557A (en) | Real-time three-dimensional facial reconstruction method based on single frames facial image | |
| US20220044311A1 (en) | Method for enhancing a user's image while e-commerce shopping for the purpose of enhancing the item that is for sale | |
| CN108805090A (en) | A kind of virtual examination cosmetic method based on Plane Gridding Model | |
| CN108537126A (en) | A kind of face image processing system and method | |
| JP2004094773A (en) | Head wearing object image synthesizing method and device, makeup image synthesizing method and device, and program | |
| CN109284778A (en) | Face face value calculating method, computing device and electronic equipment | |
| Zhang et al. | Facial expression cloning with elastic and muscle models | |
| CN110443872B (en) | Expression synthesis method with dynamic texture details | |
| Li et al. | EyeNeRF: a hybrid representation for photorealistic synthesis, animation and relighting of human eyes | |
| CN112116699A (en) | Real-time real-person virtual trial sending method based on 3D face tracking | |
| Lin et al. | eHeritage of shadow puppetry: creation and manipulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: An intelligent beauty method and intelligent beauty mirror Granted publication date: 20230804 Pledgee: Jiangsu Zijin Rural Commercial Bank Co.,Ltd. Yuhuatai sub branch Pledgor: NANJING TUWAN INTELLIGENT TECHNOLOGY Co.,Ltd. Registration number: Y2024320000004 |