CN108363964A - A kind of pretreated wrinkle of skin appraisal procedure and system - Google Patents
A kind of pretreated wrinkle of skin appraisal procedure and system Download PDFInfo
- Publication number
- CN108363964A CN108363964A CN201810084748.2A CN201810084748A CN108363964A CN 108363964 A CN108363964 A CN 108363964A CN 201810084748 A CN201810084748 A CN 201810084748A CN 108363964 A CN108363964 A CN 108363964A
- Authority
- CN
- China
- Prior art keywords
- image
- face
- skin
- fuzzy
- pixel point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000037303 wrinkles Effects 0.000 title claims abstract description 88
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000001514 detection method Methods 0.000 claims abstract description 95
- 239000013598 vector Substances 0.000 claims abstract description 48
- 238000011156 evaluation Methods 0.000 claims description 56
- 238000012549 training Methods 0.000 claims description 35
- 238000013210 evaluation model Methods 0.000 claims description 34
- 238000007781 pre-processing Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 18
- 238000002203 pretreatment Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 230000001815 facial effect Effects 0.000 abstract description 8
- 230000006870 function Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract 1
- 230000008447 perception Effects 0.000 abstract 1
- 238000005375 photometry Methods 0.000 abstract 1
- 239000002537 cosmetic Substances 0.000 description 11
- 230000003796 beauty Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000002500 effect on skin Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000002310 reflectometry Methods 0.000 description 2
- 235000017166 Bambusa arundinacea Nutrition 0.000 description 1
- 235000017491 Bambusa tulda Nutrition 0.000 description 1
- 241001330002 Bambuseae Species 0.000 description 1
- 235000015334 Phyllostachys viridis Nutrition 0.000 description 1
- 206010039792 Seborrhoea Diseases 0.000 description 1
- 239000011425 bamboo Substances 0.000 description 1
- 239000011324 bead Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000037312 oily skin Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/44—Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
- A61B5/441—Skin evaluation, e.g. for skin disorder diagnosis
- A61B5/442—Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Medical Informatics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Heart & Thoracic Surgery (AREA)
- Multimedia (AREA)
- Computer Graphics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Geometry (AREA)
- Dermatology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of pretreated wrinkle of skin appraisal procedure and systems, belong to Skin Detection field；Method includes shooting the facial image for being associated with same face respectively using image collecting device, image preprocess apparatus is compared the detected value that facial image obtains is detected with preset standard value, to obtain the reason that whether image is fuzzy and obscures, not fuzzy facial image is sent in Cloud Server；Cloud Server handles to obtain the unit normal vector of each pixel and is handled to obtain the case depth information of each pixel according to unit normal vector to form face's stereo-picture of face by stereo photometry；And wrinkle of skin assessment models judge face's stereo-picture, to obtain corresponding wrinkle of skin assessment result and export.The advantageous effect of above-mentioned technical proposal is：In conjunction with the function of skin detection and vanity mirror, the ability of wrinkle of skin three-dimensional reconstruction and three-dimensional assessment is provided so that user is capable of the wrinkle of skin situation of accurate perception itself.
Description
Technical Field
The invention relates to the technical field of skin detection, in particular to a method and a system for evaluating pretreated skin wrinkles.
Background
As the quality of life of people is improved, more and more people, especially women, begin to pay attention to their skin conditions, and more skin condition-oriented care products are also in the market place. Meanwhile, beauty institutions providing beauty services appear in the visual field of people like bamboo shoots after rain, users judge the skin states of the users through beauty therapists in the beauty institutions, such as whether the canthus has wrinkles or not, whether the face has legal lines or not, and the like, and then recommend corresponding care products to the users according to the problems reflected by the skin states. However, this skin detection method requires the user to go to the beauty institution periodically for skin care and detection, which requires the user to spare a lot of idle time, and is inconvenient.
Although some skin detection devices, such as skin detectors, exist on the market, the skin detection devices are expensive and complicated to operate, and are not suitable for users to use at home. Meanwhile, the detection principle of the skin detection devices is that the sensing data collected by some sensors are only used for carrying out planar data processing, and the problem of three-dimensional facial reconstruction is not involved, so that the skin detection devices have a good detection effect on skin conditions such as oily skin and skin water shortage, have a poor detection effect on skin conditions of facial skin wrinkles, and cannot meet the requirements of users.
Disclosure of Invention
According to the above problems in the prior art protection, a technical scheme of a method and a system for evaluating skin wrinkles in advance is provided, which aims to combine the functions of skin detection and cosmetic mirror to provide the capabilities of three-dimensional reconstruction and three-dimensional evaluation of skin wrinkles, so that a user can accurately grasp the skin wrinkle condition of the user, and the user experience is improved.
The technical scheme specifically comprises the following steps:
a method for evaluating the wrinkles of the skin processed in advance, wherein, a skin detection mirror is used for evaluating the wrinkles of the skin of the human face, the skin detection mirror comprises an image acquisition device and an interface display device of the image preprocessing device connected with the image acquisition device, the skin detection mirror is remotely connected with a cloud server, and the method also comprises the following steps:
step S1, under the irradiation of different light sources, the image acquisition device is adopted to respectively shoot different face images related to the same face;
step S2, the image preprocessing device provides a preset blurring detection strategy to carry out blurring detection on the face image so as to obtain a detection value;
step S3, the image preprocessing device compares the detection value with a preset standard value to judge whether the fuzzy detection value is larger than the standard preset value;
if yes, displaying the face image in a fuzzy state and the reason of the fuzzy state on the display interface;
if not, go to step S4;
step S4, the skin detection mirror sends the face image to the cloud server;
step S5, the cloud server obtains a unit normal vector of each pixel point in the face image through photometric stereo processing;
step S6, the cloud server processes the unit normal vector to obtain surface depth information of each pixel point;
step S7, the cloud server forms a face three-dimensional image of a face according to the surface depth information of each pixel point and the position information of each pixel point on the face image;
and step S8, the cloud server judges the face three-dimensional image by adopting a skin wrinkle evaluation model formed by pre-training so as to obtain a corresponding skin wrinkle evaluation result and output the skin wrinkle evaluation result to a user terminal remotely connected with the cloud server for a user to view.
Preferably, the preset fuzzification detection strategy comprises:
the image preprocessing device is used for acquiring the face image and acquiring a first fuzzy characteristic value in the face image;
and carrying out fuzzy analysis on the first fuzzy characteristic value through alpha channel characteristics to obtain a second fuzzy characteristic value, comparing the second fuzzy characteristic value with a preset first standard threshold, and if the second fuzzy characteristic value is greater than the first standard threshold, the face image is in a fuzzy state, and corresponding information displaying a fuzzy reason is that the person moves to cause the fuzzy shooting image.
Preferably, the preset fuzzification detection strategy comprises:
the image preprocessing device is used for acquiring the face image and acquiring a third fuzzy characteristic value in the face image;
and carrying out fuzzy analysis on the fuzzy characteristic value through alpha channel characteristics to obtain a fourth fuzzy value, comparing the second fuzzy characteristic value with a preset second standard threshold, and if the fourth fuzzy characteristic value is greater than the second standard threshold, the face image is in a fuzzy state, and the corresponding information for displaying the fuzzy reason is that the focusing is not accurate so as to cause image blurring.
Preferably, in the method for evaluating skin wrinkles by pre-processing, in step S1, the face image is captured under at least 3 different light sources, and all the light sources are not on the same straight line.
Preferably, the pre-treated skin wrinkle evaluation method, wherein the number of the light sources is 12.
Preferably, the method for evaluating skin wrinkles by pre-treatment, wherein the step S5 specifically includes:
step S51, one pixel point is taken as a pixel point to be processed;
step S22, determining whether the pixel to be processed is a highlight pixel:
if yes, go to step S53;
if not, go to step S54;
s53, replacing the non-highlight pixel points at the same position in different face images, and then turning to S54;
step S54, processing to obtain a surface normal vector of the pixel point to be processed;
and step S55, processing according to the surface normal vector of the pixel point to obtain a unit normal vector of the pixel point.
Preferably, the method for evaluating skin wrinkles by pre-treatment, wherein the step S6 specifically includes:
and aiming at each pixel point, establishing a preset constraint formula according to the unit normal vector, and processing according to the constraint formula to obtain the surface depth information of the pixel point.
Preferably, the method for evaluating skin wrinkles by pre-treatment, wherein the constraint formula is specifically:
wherein,
V1and V2Are all tangential vectors of the face surface of the human face;
n is used to represent the unit vector of the pixel point;
x and y are used for representing the position information of the plane of the pixel point on the face image;
z is used to represent the surface depth information.
Preferably, the method for evaluating skin wrinkles in advance, wherein in the step S8, the skin wrinkle evaluation model includes: a first evaluation model for evaluating canthus wrinkles of a human face;
the first evaluation model is formed by adopting deep neural network learning according to a first training set prepared in advance;
the first training set comprises a plurality of first training data pairs, and each first training data pair comprises stereo image data of a human face with different shapes of eye corner wrinkles and evaluation scores corresponding to the stereo image data.
Preferably, the method for evaluating skin wrinkles in advance, wherein in the step S8, the skin wrinkle evaluation model includes: a second evaluation model for evaluating a grain of the face;
the second evaluation model is formed by adopting deep neural network learning according to a second training set prepared in advance;
the second training set comprises a plurality of second training data pairs, and each second training data pair comprises stereo image data of a face with different-shape grain and evaluation scores corresponding to the stereo image data.
A pre-treatment skin wrinkle evaluation system, comprising:
the skin detection mirror is internally provided with an image acquisition device which is used for respectively shooting different face images related to the same face under the irradiation of different light sources; the image preprocessing device is connected with the image acquisition device and is used for detecting the fuzziness of the face image to obtain a fuzzy detection value and comparing the fuzzy detection value with a preset standard value; judging whether the fuzzy detection value is larger than the standard preset value or not;
the cloud server is remotely connected with the skin detection mirror, the skin detection mirror is used for sending the face image to the cloud server when the fuzzy detection value of the face image is not larger than the standard preset value, the cloud server adopts the skin wrinkle evaluation method, evaluates the skin wrinkles of the face according to the face image, and outputs an evaluation result to a user terminal remotely connected with the cloud server for a user to check.
The beneficial effects of the above technical scheme are: the functions of skin detection and a cosmetic mirror are combined, the capability of three-dimensional reconstruction and three-dimensional evaluation of skin wrinkles is provided, a user can accurately master the skin wrinkle condition of the user, meanwhile, the face image is detected by judging whether to shoot blurs and the reason of image blurs in advance when the face image is obtained, if the face image is shot blurs, the user carries out corresponding adjustment according to the blurs, the image acquisition device shoots the non-blurs face image again, the accuracy of face image processing is improved, and the user experience is improved.
Drawings
FIG. 1 is a schematic flow chart of a pre-treatment skin wrinkle evaluation method according to a preferred embodiment of the present invention;
FIG. 2 is a flowchart illustrating the detailed process of step S5 based on FIG. 1 according to a preferred embodiment of the present invention;
FIG. 3 is a schematic diagram of the structure of a pre-treatment skin wrinkle evaluation system according to a preferred embodiment of the present invention;
FIG. 4 is a schematic view of a skin detection mirror in accordance with a preferred embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
In light of the above-mentioned problems in the prior art, there is now provided
A method for evaluating the wrinkles of the skin processed in advance, wherein, a skin detection mirror is used for evaluating the wrinkles of the skin of the human face, the skin detection mirror comprises an image acquisition device and an interface display device of the image preprocessing device connected with the image acquisition device, the skin detection mirror is remotely connected with a cloud server, and the method also comprises the following steps:
step S1, under the irradiation of different light sources, different face images related to the same face are respectively shot by an image acquisition device;
step S2, the image preprocessing device provides a preset fuzzification detection strategy to carry out fuzzification detection on the face image so as to obtain a detection value;
step S3, the image preprocessing device compares the detection value with a preset standard value to judge whether the fuzzy detection value is larger than the standard preset value;
if so, displaying the face image in a fuzzy state and the reason of the fuzzy state on a display interface;
if not, go to step S4;
step S4, the skin detection mirror sends the face image to a cloud server;
step S5, the cloud server obtains a unit normal vector of each pixel point in the face image through photometric stereo processing;
step S6, the cloud server processes the surface depth information of each pixel point according to the unit normal vector to obtain the surface depth information of each pixel point;
step S7, the cloud server forms a face three-dimensional image of the face according to the surface depth information of each pixel point and the position information of each pixel point on the face image;
and step S8, the cloud server judges the face three-dimensional image by adopting a skin wrinkle evaluation model formed by pre-training so as to obtain a corresponding skin wrinkle evaluation result and output the skin wrinkle evaluation result to a user terminal remotely connected with the cloud server for a user to check.
In a preferred technical solution, the preset fuzzification detection strategy includes:
the motion blur detection strategy comprises an image preprocessing device, a motion blur detection device and a motion blur detection device, wherein the image preprocessing device is used for acquiring a face image and acquiring a first blur characteristic value in the face image;
and carrying out fuzzy analysis on the first fuzzy characteristic value through alpha channel characteristics to obtain a second fuzzy characteristic value, comparing the second fuzzy characteristic value with a preset first standard threshold, and if the second fuzzy characteristic value is greater than the first standard threshold, the face image is in a fuzzy state, and corresponding information displaying a fuzzy reason is that the person moves to cause the fuzzy of the shot image.
In a preferred embodiment, the preset fuzzification detection strategy comprises:
focusing a fuzzy detection strategy, wherein the image preprocessing device is used for acquiring a face image and acquiring a third fuzzy characteristic value in the face image;
and carrying out fuzzy analysis on the fuzzy characteristic value through the alpha channel characteristic to obtain a fourth fuzzy value, comparing the second fuzzy characteristic value with a preset second standard threshold, and if the fourth fuzzy characteristic value is greater than the second standard threshold, the face image is in a fuzzy state, and the corresponding information for displaying the fuzzy reason is that the focusing is not accurate to cause image blurring.
In the above technical solution, the first fuzzy characteristic value and the second fuzzy characteristic value may be obtained by analyzing using a singular value analysis method, and then the cause of the image blur may be determined based on an alpha channel characteristic analysis method.
Specifically, in this embodiment, the skin detection mirror may be modified by using a common cosmetic mirror as a mirror body, that is, the skin detection mirror is made into a cosmetic mirror that can be normally used by people, so that the functions of skin wrinkle detection and evaluation can be integrated into the cosmetic mirror while providing daily use for people.
In this embodiment, an image collecting device is disposed on the skin detection mirror, and the image collecting device may be a camera disposed on the skin detection mirror, and specifically may be disposed directly above the skin detection mirror, so as to conveniently capture an entire image of a face of a user.
In this embodiment, the image capturing device on the skin detection mirror respectively captures face images of the same face under different light sources, where the different light sources are point light sources in different directions and are placed at different positions around the skin detection mirror, so as to form different face images of the same face under irradiation of different point light sources. In these face images, the positions of the same pixel point relative to the face image are the same, and the difference is only the brightness of the pixel point, which will be described in detail below.
In this embodiment, if a face image acquired by an image acquisition device is blurred due to the movement of a user or the influence of other factors during the shooting process, and if the blurred face image is directly sent to a cloud server, the cloud server cannot correctly obtain a skin wrinkle evaluation result about the face;
the image preprocessing device can process the face image by adopting an algorithm, namely Laplacian transformation, then remove mean square error, and compare the obtained fuzzy detection value with a preset standard preset value based on a PYTHON programming language, for example, if the standard preset value is 100, the face image can be sent to a cloud server without being fuzzy;
if the face image is beyond 100 degrees, the face image is fuzzy, the face image cannot be sent to a cloud server, the face image is required to be shot again by an image acquisition device to obtain the face image, and the face image cannot be sent to the cloud server until the face image passes through the ambiguity detection.
In this embodiment, the skin detection mirror is remotely connected to a cloud server, and uploads the different face images associated with the same face to the cloud server. Specifically, a wireless communication module such as a WiFi module may be disposed in the skin detection mirror, and remotely connected to the cloud server through an indoor route.
In this embodiment, after receiving the face images uploaded by the skin detection mirror, the cloud server performs comprehensive processing on different face images shot under different point light sources. Specifically, the processing is performed in the cloud server according to each pixel point on the face image, and the difference of the same pixel point in different face images only lies in the difference of the light source directions of the point light sources and the difference of the brightness of the pixel point caused by the difference, so that the attributes of different face images can be integrated into the same pixel point, for example, the attribute of one pixel point includes information such as the light source directions and brightness values of the pixel point in different face images.
In this embodiment, the cloud server processes each pixel point by a photometric stereo method to obtain unit normal vectors of different pixel points, and then obtains surface depth information of each pixel point according to the unit normal vector processing. After the surface depth information exists, the cloud server can establish a face three-dimensional image of the face according to the pixel points, namely, the face is subjected to three-dimensional image reconstruction.
In this embodiment, after the face stereo image of the human face is acquired, the face stereo image is evaluated by using a skin wrinkle evaluation model formed by pre-training, and a corresponding skin wrinkle evaluation result is output. And the cloud server remotely issues the evaluation result to the corresponding user terminal for the user to check. Specifically, in the skin wrinkle evaluation model, the input data is a stereo image of the face, and the output data is an evaluation result of a corresponding skin wrinkle region in the stereo image. Further, the evaluation result may be an evaluation score. The specific operation principle of the skin wrinkle evaluation model will be described in detail below.
In summary, in the technical solution of the present invention, the image collecting device on the skin detection mirror is adopted to respectively shoot different face images for the same face under different light sources, the cloud server processes the face images by using a photometric stereo method according to the face images to finally form a stereo image of the face, and the stereo image is sent to the evaluation model for image recognition and wrinkle evaluation, and finally a corresponding evaluation result is output. Compared with the prior art, the technical scheme of the invention adopts a photometric stereo method, solves surface depth information and other modes to establish a three-dimensional image of the face, and carries out wrinkle evaluation on the image, thereby truly reflecting the condition of skin wrinkles on the face of a user. And the function of skin detection is integrated into the cosmetic mirror, so that the daily use of a user is facilitated.
It should be noted that, in consideration of the completeness of face image shooting, the skin detection mirror in the present invention should be a cosmetic mirror with a certain mirror surface area, such as a cosmetic mirror placed on a cosmetic table, or a cosmetic mirror directly installed on a bedroom wall, so as to avoid implementing the technical solution of the present invention by using a relatively small cosmetic mirror available on the market and capable of being held in the hand.
In a preferred embodiment of the present invention, in step S1, the face image is obtained by shooting under at least 3 different light sources, and all the light sources are not on the same straight line.
Further, in a preferred embodiment of the present invention, the number of the light sources is specifically 12, that is, 12 light sources are disposed around the skin detection mirror, directions of the light sources are different, and all the light sources are not on the same straight line.
In a preferred embodiment of the present invention, as shown in fig. 2, the step S5 specifically includes:
step S51, one pixel point is taken as a pixel point to be processed;
step S52, determining whether the pixel to be processed is a highlight pixel:
if yes, go to step S53;
if not, go to step S54;
step S53, replacing the non-highlight pixel points at the same position in different face images, and then turning to step S24;
step S54, processing to obtain surface normal vectors of the pixel points to be processed;
and step S55, processing according to the surface normal vector of the pixel point to obtain a unit normal vector of the pixel point.
Specifically, in this embodiment, because the highlight pixel cannot accurately determine the skin wrinkle image, the highlight pixel needs to be removed first when the facial image is reconstructed. The removing method comprises the following steps: firstly, judging whether the pixel points are highlight pixel points or not by a threshold value method (the brightness values of the pixel points are highlight pixel points if the brightness values are higher than a preset threshold value). Then, for the pixel point judged to be highlight, the non-highlight pixel point at the same position in other face images is used for replacing, specifically, as described above, the attributes of different face images are already used as the attributes of each pixel point to participate in the calculation, for example, the attribute value of one pixel point includes the brightness value of the pixel point in different face images, and for one pixel point, the principle of rejecting the highlight pixel point is to reject the brightness value of one pixel point higher than the preset threshold value, and only the brightness value of the non-highlight pixel point is left.
In this embodiment, after the highlight pixels are eliminated, all the non-highlight pixels are sequentially processed to obtain the surface normal vector of each pixel, and further obtain the unit normal vector of each pixel.
The process of solving the surface normal vector specifically includes:
for a color image, the luminance of each pixel is represented by the values of R, G, B three color channels. In this embodiment, the processing procedure of the surface normal vector is described by taking the R value as an example.
Assuming that the face surface in the invention conforms to an ideal Lambertian scattering model, the luminance equation of the pixel point should be:
IR＝ρRL·nR； (1)
wherein, I is used to represent the brightness of the pixel point, L is used to represent the direction vector of the light source, ρ is used to represent the texture reflectivity of the surface region corresponding to the pixel point, and n is used to represent the unit normal vector of the surface region corresponding to the pixel point.
The direction vectors of the light sources are known in advance, and the direction vectors can be realized by shooting a group of highlight black ball images in different light source directions, and the illumination direction can be obtained by searching the position of a highlight point in each highlight black ball image, so that the highlight black ball images are used as the light source directions of the point light sources.
In the above step, the brightness of the non-highlight pixel under the illumination of different light sources can be represented as:
IR＝(I1R,I2R,...,IqR)T； (2)
q is used for representing the number of light sources, wherein the light sources corresponding to the highlight pixels are removed, and T is the transposition calculation of the matrix.
Accordingly, the unit normal vector for each pixel point can be expressed as:
nR＝(n1R,n2R,...,nqR)T； (3)
by calculation, the direction vectors of the q light sources should be:
multiplying by L at both ends of the above equation (1) simultaneously-1It is possible to obtain:
L-1IR＝ρR·nR； (5)
the modulus of the left vector of the equation of equation (5) above is the value of the texture reflectivity, and the direction of the left vector is the direction of the surface normal vector.
After the surface normal vector of the R channel of the pixel point is obtained, calculating the unit normal vector of the R channel, namely:
nR＝(nRa,nRb,nRc)T； (6)
wherein, (a, b, c) is the vector direction coordinate of the pixel point in the normal vector space.
The unit normal vectors of the B channel and the G channel of the pixel point can be respectively obtained by comparing the formulas (1) to (6):
nB＝(nBa,nBb,nBc)T； (7)
nG＝(nGa,nGb,nGc)T； (8)
the unit normal vector of the final pixel point can be the average value of the unit normal vectors of the RGB channels, and is expressed as:
n＝(na,nb,nc)T； (9)
the gradient (-n) of each pixel point can be calculated according to the formula (9)a/nc,-nb/nc) And a normal vector map of facial skin wrinkles can be established.
In a preferred embodiment of the present invention, the step S6 specifically includes:
and aiming at each pixel point, establishing a preset constraint formula according to the unit normal vector, and processing according to the constraint formula to obtain the surface depth information of the pixel point.
Further, based on the tangent plane principle, the normal direction of each point on the object surface should be perpendicular to the tangential direction, and then the following constraint formula can be established:
wherein, V1And V2The (x, y, z) is a three-dimensional coordinate of the pixel point on the face surface, where (x, y) is a planar coordinate of the pixel point, that is, position information of a plane of the pixel point on the face image (the position information described in step S4), and z is used to represent a depth value of the pixel point on the face image.
For a human face image with m pixels, 2m constraint equations can be obtained, the unit normal vector n of each pixel is known, the position information x and y of the plane of each pixel is also known, and the depth value z is a scalar, so that the constraint equations of all the pixels can form an equation matrix, and the surface depth information of each pixel can be obtained by solving the equation matrix.
After the surface depth information of each pixel point is obtained, the three-dimensional coordinates (x, y, z) exist, so that a face three-dimensional image of a human face can be constructed and 3D display can be carried out.
In a preferred embodiment of the present invention, in the step S8, the skin wrinkle evaluation model includes: a first evaluation model for evaluating canthus wrinkles of a human face;
the first evaluation model is formed by adopting deep neural network learning according to a first training set prepared in advance;
the first training set includes a plurality of first training data pairs, each of which includes stereoscopic image data of a human face having different shapes of eye corner wrinkles and evaluation scores of the corresponding stereoscopic image data.
Specifically, in this embodiment, a plurality of first training data pairs are prepared in advance, each first training data pair includes a stereoscopic image of an eye corner wrinkle, and an evaluation score obtained by manually evaluating the stereoscopic image. The greater the number of first training data pairs, the more accurate the evaluation result of the first evaluation model formed by training.
In this embodiment, the first evaluation model is specifically used for evaluating the canthus wrinkles of the human face. In the practical application process, the first evaluation model firstly finds out a stereoscopic image of canthus wrinkles from the whole face stereoscopic image, then evaluates the stereoscopic image of canthus wrinkles, and finally outputs an evaluation result, wherein the evaluation result can be given in a mode of evaluation scores.
In another preferred embodiment of the present invention, in the step S8, the skin wrinkle evaluation model includes: a second evaluation model for evaluating a grain of the face;
a second evaluation model is formed by adopting deep neural network learning according to a second training set prepared in advance;
the second training set comprises a plurality of second training data pairs, and each second training data pair comprises stereo image data of a face with different-shape grain and evaluation scores of the corresponding stereo image data.
Specifically, in this embodiment, a plurality of second training data pairs are prepared in advance, each second training data pair includes a stereoscopic image of a grain, and an evaluation score obtained after the stereoscopic image is manually evaluated. The larger the number of the second training data pairs, the more accurate the evaluation result of the second evaluation model formed by training.
In this embodiment, the second evaluation model is specially used for evaluating a facial ordinance print. In the practical application process, the second evaluation model firstly finds out the stereo image of the grain from the whole face stereo image, then evaluates the stereo image of the grain, and finally outputs an evaluation result, wherein the evaluation result can also be given in a score evaluation mode.
In another preferred embodiment of the present invention, the first evaluation model and the second evaluation model may be used in a skin wrinkle evaluation model at the same time, so that the conditions of the canthus wrinkles and the statutory lines of the user can be comprehensively evaluated.
In a preferred embodiment of the present invention, based on the above-mentioned pre-treatment skin wrinkle evaluation method, there is now provided a pre-treatment skin wrinkle evaluation system a, as particularly shown in fig. 3, comprising:
the skin detection mirror A1 is characterized in that an image acquisition device A11 is arranged in the skin detection mirror A1 and is used for respectively shooting different face images related to the same face by adopting the image acquisition device A11 under the irradiation of different light sources;
the image preprocessing device A12 is connected with the image acquisition device A11, and the image preprocessing device A12 is used for detecting the fuzziness of the face image to obtain a fuzzy detection value and comparing the fuzzy detection value with a preset standard value; judging whether the fuzzy detection value is larger than a standard preset value or not;
the cloud server A2 is remotely connected with the skin detection mirror A1, the skin detection mirror A1 is used for sending the face image to the cloud server A2 when the fuzzy detection value of the face image is not more than the standard preset value,
the cloud server A2 is remotely connected with the skin detection mirror A1, the skin detection mirror A1 is used for sending the face image to the cloud server A2, the cloud server A2 adopts the skin wrinkle evaluation method, the skin wrinkles of the face are evaluated according to the face image, and the evaluation result is output to the user terminal B which is remotely connected with the cloud server A2 for the user to check.
Further, as shown in fig. 4, the mirror surface appearance schematic diagram of the skin detection mirror a1 is that a plurality of LED lamp beads a13 can be further arranged around the mirror surface body a12 of the skin detection mirror a1 as point light sources in different directions to assist the image acquisition device a11 in shooting the human face.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (11)
1. A method for evaluating the wrinkles of the skin processed in advance is characterized in that a skin detection mirror is adopted to evaluate the wrinkles of the skin of a human face, the skin detection mirror comprises an image acquisition device and an interface display device of the image preprocessing device connected with the image acquisition device, the skin detection mirror is remotely connected with a cloud server, and the method further comprises the following steps:
step S1, under the irradiation of different light sources, the image acquisition device is adopted to respectively shoot different face images related to the same face;
step S2, the image preprocessing device provides a preset blurring detection strategy to carry out blurring detection on the face image so as to obtain a detection value;
step S3, the image preprocessing device compares the detection value with a preset standard value to judge whether the fuzzy detection value is larger than the standard preset value;
if yes, displaying the face image in a fuzzy state and the reason of the fuzzy state on the display interface;
if not, go to step S4;
step S4, the skin detection mirror sends the face image to the cloud server;
step S5, the cloud server obtains a unit normal vector of each pixel point in the face image through photometric stereo processing;
step S6, the cloud server processes the unit normal vector to obtain surface depth information of each pixel point;
step S7, the cloud server forms a face three-dimensional image of a face according to the surface depth information of each pixel point and the position information of each pixel point on the face image;
and step S8, the cloud server judges the face three-dimensional image by adopting a skin wrinkle evaluation model formed by pre-training so as to obtain a corresponding skin wrinkle evaluation result and output the skin wrinkle evaluation result to a user terminal remotely connected with the cloud server for a user to view.
2. The pre-processed skin wrinkle evaluation method according to claim 1, characterized in that the preset blurring detection strategy comprises:
the image preprocessing device is used for acquiring the face image and acquiring a first fuzzy characteristic value in the face image;
and carrying out fuzzy analysis on the first fuzzy characteristic value through alpha channel characteristics to obtain a second fuzzy characteristic value, comparing the second fuzzy characteristic value with a preset first standard threshold, and if the second fuzzy characteristic value is greater than the first standard threshold, the face image is in a fuzzy state, and corresponding information displaying a fuzzy reason is that the person moves to cause the fuzzy shooting image.
3. The pre-treated skin wrinkle evaluation method according to claim 1,
the preset fuzzification detection strategy comprises the following steps:
the image preprocessing device is used for acquiring the face image and acquiring a third fuzzy characteristic value in the face image;
and carrying out fuzzy analysis on the fuzzy characteristic value through alpha channel characteristics to obtain a fourth fuzzy value, comparing the second fuzzy characteristic value with a preset second standard threshold, and if the fourth fuzzy characteristic value is greater than the second standard threshold, the face image is in a fuzzy state, and the corresponding information for displaying the fuzzy reason is that the focusing is not accurate so as to cause image blurring.
4. The pre-processed skin wrinkle evaluation method as claimed in claim 1, wherein in step S1, the face image is captured under at least 3 different light sources, all of which are not in the same straight line.
5. The pre-treated skin wrinkle evaluation method according to claim 2, characterized in that the number of light sources is 12.
6. The pre-treated skin wrinkle evaluation method according to claim 1, characterized in that said step S5 specifically comprises:
step S51, one pixel point is taken as a pixel point to be processed;
step S52, determining whether the pixel to be processed is a highlight pixel:
if yes, go to step S53;
if not, go to step S54;
s53, replacing the non-highlight pixel points at the same position in different face images, and then turning to S54;
step S54, processing to obtain a surface normal vector of the pixel point to be processed;
and step S55, processing according to the surface normal vector of the pixel point to obtain a unit normal vector of the pixel point.
7. The pre-treated skin wrinkle evaluation method according to claim 1, characterized in that said step S6 specifically comprises:
and aiming at each pixel point, establishing a preset constraint formula according to the unit normal vector, and processing according to the constraint formula to obtain the surface depth information of the pixel point.
8. The pre-processed skin wrinkle evaluation method according to claim 5, characterized in that the constraint formula is specifically:
wherein,
V1and V2Are all tangential vectors of the face surface of the human face;
n is used to represent the unit vector of the pixel point;
x and y are used for representing the position information of the plane of the pixel point on the face image;
z is used to represent the surface depth information.
9. The pre-processed skin wrinkle evaluation method according to claim 1, characterized in that in step S8, the skin wrinkle evaluation model includes: a first evaluation model for evaluating canthus wrinkles of a human face;
the first evaluation model is formed by adopting deep neural network learning according to a first training set prepared in advance;
the first training set comprises a plurality of first training data pairs, and each first training data pair comprises stereo image data of a human face with different shapes of eye corner wrinkles and evaluation scores corresponding to the stereo image data.
10. The pre-processed skin wrinkle evaluation method according to claim 1, characterized in that in step S8, the skin wrinkle evaluation model includes: a second evaluation model for evaluating a grain of the face;
the second evaluation model is formed by adopting deep neural network learning according to a second training set prepared in advance;
the second training set comprises a plurality of second training data pairs, and each second training data pair comprises stereo image data of a face with different-shape grain and evaluation scores corresponding to the stereo image data.
11. A pre-treatment skin wrinkle evaluation system, comprising:
the skin detection mirror is internally provided with an image acquisition device which is used for respectively shooting different face images related to the same face under the irradiation of different light sources;
the image preprocessing device is connected with the image acquisition device and is used for detecting the fuzziness of the face image to obtain a fuzzy detection value and comparing the fuzzy detection value with a preset standard value; judging whether the fuzzy detection value is larger than the standard preset value or not;
the cloud server is remotely connected with the skin detection mirror, the skin detection mirror is used for sending the face image to the cloud server when the fuzzy detection value of the face image is not greater than the standard preset value,
the cloud server adopts the skin wrinkle evaluation method according to any one of claims 1 to 8, evaluates the skin wrinkles of the human face according to the human face image, and outputs the evaluation result to a user terminal remotely connected with the cloud server for the user to view.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810084748.2A CN108363964A (en) | 2018-01-29 | 2018-01-29 | A kind of pretreated wrinkle of skin appraisal procedure and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810084748.2A CN108363964A (en) | 2018-01-29 | 2018-01-29 | A kind of pretreated wrinkle of skin appraisal procedure and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108363964A true CN108363964A (en) | 2018-08-03 |
Family
ID=63007247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810084748.2A Pending CN108363964A (en) | 2018-01-29 | 2018-01-29 | A kind of pretreated wrinkle of skin appraisal procedure and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108363964A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109381165A (en) * | 2018-09-12 | 2019-02-26 | 维沃移动通信有限公司 | A kind of skin detecting method and mobile terminal |
| CN109948476A (en) * | 2019-03-06 | 2019-06-28 | 南京七奇智能科技有限公司 | A kind of face skin detection system based on computer vision and its implementation |
| CN109994206A (en) * | 2019-02-26 | 2019-07-09 | 华为技术有限公司 | A kind of appearance prediction technique and electronic equipment |
| WO2022184084A1 (en) * | 2021-03-02 | 2022-09-09 | 华为技术有限公司 | Skin test method and electronic device |
| WO2024021000A1 (en) * | 2022-07-29 | 2024-02-01 | L'oreal | System and method for evaluating dynamic wrinkles on keratin material of a user to be tested |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090195545A1 (en) * | 2008-01-31 | 2009-08-06 | University Fo Southern California | Facial Performance Synthesis Using Deformation Driven Polynomial Displacement Maps |
| CN102819864A (en) * | 2012-08-14 | 2012-12-12 | 北京航空航天大学 | Luminosity diagram-based geometry and surface material reconstructing method |
| CN103827916A (en) * | 2011-09-22 | 2014-05-28 | 富士胶片株式会社 | Wrinkle detection method, wrinkle detection device and wrinkle detection program, as well as wrinkle evaluation method, wrinkle evaluation device and wrinkle evaluation program |
| CN105139003A (en) * | 2015-09-17 | 2015-12-09 | 桂林远望智能通信科技有限公司 | Dynamic face identification system and method |
| CN105654451A (en) * | 2014-11-10 | 2016-06-08 | 中兴通讯股份有限公司 | Image processing method and device |
| CN106127250A (en) * | 2016-06-24 | 2016-11-16 | 深圳市唯特视科技有限公司 | A kind of face method for evaluating quality based on three dimensional point cloud |
| CN106469302A (en) * | 2016-09-07 | 2017-03-01 | 成都知识视觉科技有限公司 | A kind of face skin quality detection method based on artificial neural network |
| CN106524909A (en) * | 2016-10-20 | 2017-03-22 | 北京旷视科技有限公司 | Three-dimensional image acquisition method and apparatus |
| CN107392858A (en) * | 2017-06-16 | 2017-11-24 | 广东欧珀移动通信有限公司 | Image highlight area processing method, device and terminal device |
-
2018
- 2018-01-29 CN CN201810084748.2A patent/CN108363964A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090195545A1 (en) * | 2008-01-31 | 2009-08-06 | University Fo Southern California | Facial Performance Synthesis Using Deformation Driven Polynomial Displacement Maps |
| CN103827916A (en) * | 2011-09-22 | 2014-05-28 | 富士胶片株式会社 | Wrinkle detection method, wrinkle detection device and wrinkle detection program, as well as wrinkle evaluation method, wrinkle evaluation device and wrinkle evaluation program |
| CN102819864A (en) * | 2012-08-14 | 2012-12-12 | 北京航空航天大学 | Luminosity diagram-based geometry and surface material reconstructing method |
| CN105654451A (en) * | 2014-11-10 | 2016-06-08 | 中兴通讯股份有限公司 | Image processing method and device |
| CN105139003A (en) * | 2015-09-17 | 2015-12-09 | 桂林远望智能通信科技有限公司 | Dynamic face identification system and method |
| CN106127250A (en) * | 2016-06-24 | 2016-11-16 | 深圳市唯特视科技有限公司 | A kind of face method for evaluating quality based on three dimensional point cloud |
| CN106469302A (en) * | 2016-09-07 | 2017-03-01 | 成都知识视觉科技有限公司 | A kind of face skin quality detection method based on artificial neural network |
| CN106524909A (en) * | 2016-10-20 | 2017-03-22 | 北京旷视科技有限公司 | Three-dimensional image acquisition method and apparatus |
| CN107392858A (en) * | 2017-06-16 | 2017-11-24 | 广东欧珀移动通信有限公司 | Image highlight area processing method, device and terminal device |
Non-Patent Citations (7)
| Title |
|---|
| ALI SOHAIB: ""Use of Photometric Stereo for the Accurate Modelling of Three-Dimensional Skin Microrelief"", 《A THESIS OF THE UNIVERSITY OF THE WEST OF ENGLAND》 * |
| ALI SOHAIB等: ""3D In-Vivo Measurement of Skin Topography Using Photometric Stereo"", 《3RD INTERNATIONAL CONFERENCE ON 3D BODY SCANNING TECHNOLOGIES》 * |
| ALI SOHAIB等: ""In vivo measurement of skin microrelief using photometric stereo in the presence of interreflections"", 《JOURNAL OF THE OPTICAL SOCIETY OF AMERICA》 * |
| E. JULIASTUTI等: ""Prototype of Photometric Stereo System using 6 and 9 LED light source to reconstruct human skin texture"", 《APPLIED MECHANICS AND MATERIALS》 * |
| SHAHZAD ANWAR等: ""3D Skin Texture Analysis: A Neural Network and Photometric Stereo Perspective"", 《3RD INTERNATIONAL CONFERENCE ON 3D BODY SCANNING TECHNOLOGIES》 * |
| 张家林: ""基于立体视觉的皮肤表面检测技术的研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 许鹏翔: ""皮肤评价系统的研究"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109381165A (en) * | 2018-09-12 | 2019-02-26 | 维沃移动通信有限公司 | A kind of skin detecting method and mobile terminal |
| CN109994206A (en) * | 2019-02-26 | 2019-07-09 | 华为技术有限公司 | A kind of appearance prediction technique and electronic equipment |
| WO2020173152A1 (en) * | 2019-02-26 | 2020-09-03 | 华为技术有限公司 | Facial appearance prediction method and electronic device |
| CN109948476A (en) * | 2019-03-06 | 2019-06-28 | 南京七奇智能科技有限公司 | A kind of face skin detection system based on computer vision and its implementation |
| CN109948476B (en) * | 2019-03-06 | 2022-12-13 | 南京七奇智能科技有限公司 | Human face skin detection system based on computer vision and implementation method thereof |
| WO2022184084A1 (en) * | 2021-03-02 | 2022-09-09 | 华为技术有限公司 | Skin test method and electronic device |
| WO2024021000A1 (en) * | 2022-07-29 | 2024-02-01 | L'oreal | System and method for evaluating dynamic wrinkles on keratin material of a user to be tested |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108363964A (en) | A kind of pretreated wrinkle of skin appraisal procedure and system | |
| CN108921100B (en) | Face recognition method and system based on visible light image and infrared image fusion | |
| CN108764071B (en) | Real face detection method and device based on infrared and visible light images | |
| CN103810478B (en) | Sitting posture detection method and device | |
| Santo et al. | Deep photometric stereo network | |
| CN108324247B (en) | Method and system for evaluating skin wrinkles at specified positions | |
| CN108009531B (en) | Multi-strategy anti-fraud face recognition method | |
| CN107545302B (en) | Eye direction calculation method for combination of left eye image and right eye image of human eye | |
| US7321370B2 (en) | Method and apparatus for collating object | |
| CN101339607B (en) | Human face recognition method and system, human face recognition model training method and system | |
| CN110991266B (en) | Binocular face living body detection method and device | |
| US20150193649A1 (en) | Enhanced face detection using depth information | |
| CN104123543B (en) | A kind of eye movement recognition methods based on recognition of face | |
| CN109583304A (en) | A kind of quick 3D face point cloud generation method and device based on structure optical mode group | |
| CN107169405A (en) | Method and device based on binocular camera vivo identification | |
| JP2021522591A (en) | How to distinguish a 3D real object from a 2D spoof of a real object | |
| CN107798279B (en) | Face living body detection method and device | |
| CN106372629A (en) | Living body detection method and device | |
| CN111166290A (en) | Health state detection method, equipment and computer storage medium | |
| CN102982334B (en) | The sparse disparities acquisition methods of based target edge feature and grey similarity | |
| CN102609724B (en) | Method for prompting ambient environment information by using two cameras | |
| CN102609684A (en) | Human body posture detection method and device | |
| CN113850865A (en) | Human body posture positioning method and system based on binocular vision and storage medium | |
| CN109784230A (en) | A kind of facial video image quality optimization method, system and equipment | |
| CN107018323A (en) | Control method, control device and electronic installation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20210528 |