WO2023054967A1

WO2023054967A1 - Age estimating device

Info

Publication number: WO2023054967A1
Application number: PCT/KR2022/014057
Authority: WO
Inventors: 임상섭; 강명주; 조현수; 안성권; 최명제; 서현; 신중곤; 김윤관
Original assignee: 주식회사 엘지생활건강; 서울대학교 산학협력단
Priority date: 2021-09-29
Filing date: 2022-09-20
Publication date: 2023-04-06
Also published as: KR20230046210A

Abstract

The objective of the present disclosure is to provide an age estimating device capable of calculating a user's age by using only a partial face image. The age estimating device may comprise: an image input unit that receives a user's entire face image; an image segmentation unit that obtains segmented images by segmenting the entire face image; an individual age estimation unit that calculates an individual age on the basis of the segmented images; an overall age estimation unit that calculates an overall age on the basis of the entire face image; and a final age estimation unit that calculates a final age by integrating the individual age and the overall age.

Description

age estimation device

The present disclosure relates to an age estimation device.

As people become more interested in appearance, skin care, etc., devices for calculating skin age by measuring skin conditions have appeared. However, since this skin age is limited only to the skin itself, even if the skin age is calculated high, it is somewhat difficult to know that the skin is in a poor state unless the skin is observed closely, so that the calculated skin age is not trusted, or efforts for subsequent management are made. There is a limit that cannot be reached.

Accordingly, there is a demand for a new guide that is highly reliable and can arouse awareness in relation to people's appearance or skin, and accordingly, a device for estimating age based on a user's face image has been developed.

On the other hand, since an entire face image is required to estimate age, there is a limit to an error occurring when a part of the photo is cut off or a specific part of the face is covered. In particular, when there is a risk of infection such as the recent spread of the corona virus, it is impossible to take off the mask, resulting in an impossible age estimation.

An object of the present disclosure is to provide an age estimation device capable of calculating a user's age using only a partial face image.

An object of the present disclosure is to provide an age estimating device capable of estimating a user's age even when a part of the face is covered.

An object of the present disclosure is to provide an age estimating device capable of estimating a user's age through an image including only a specific part of the face.

An age estimation system according to an embodiment of the present disclosure is based on a server generating and storing an individual age estimation model, an overall age estimation model, or a final age estimation model, and the individual age estimation model, the overall age estimation model, or the final age estimation model. and a mobile terminal displaying the individual age, the total age, or the final age calculated by the above, wherein the server receives the entire face image of the user from the mobile terminal, and divides the received full face image and the entire face image into divisions. At least one of the images is applied to the individual age estimation model, the total age estimation model, or the final age estimation model to calculate the individual age, total age, or final age, and transmit the calculated individual age, total age, or final age to the mobile terminal. can transmit

The server may receive only the whole face image from the mobile terminal or receive both the whole face image and the divided images from the mobile terminal.

An age estimation system according to an embodiment of the present disclosure includes a server generating an individual age estimation model, an overall age estimation model, or a final age estimation model, and receiving the individual age estimation model, the total age estimation model, or the final age estimation model from the server. and a mobile terminal for storing and displaying the individual age, the total age, or the final age calculated based on the stored individual age estimation model, the total age estimation model, or the final age estimation model, wherein the mobile terminal includes an entire face image of the user. is input, and at least one of the input whole face image and the divided image obtained by dividing the whole face image is applied to the individual age estimation model, the total age estimation model, or the final age estimation model to determine the individual age, the total age, or the final age. can be calculated

The server or the mobile terminal acquires the divided image for each face part by dividing the whole face image by face part, and the individual age may be the age of each face part.

The server or the mobile terminal may extract feature points for segmentation for each face part and divide the entire face image based on the extracted feature points.

The server or the mobile terminal may calculate the average of the individual ages and the total ages as the final age.

The server or the mobile terminal may calculate a weighted average of the estimated age for each part as the final age after assigning a weight to each part of the face.

The average weight may be a value obtained by dividing the sum of values obtained by multiplying the weight for each part by the age of the corresponding part by the sum of the weights.

The server or the mobile terminal may determine the weight of the non-measurable part as 0.

The server or the mobile terminal may determine the weight as the reciprocal of the mean of the absolute error.

The server or the mobile terminal may use an ensemble model when estimating the individual age.

The age estimation system further includes a data providing server storing data necessary for obtaining an age estimation model, and the server or the mobile terminal generates an age estimation model to which the ensemble model is applied based on data received from the data providing server. can be obtained

The server or the mobile terminal may correct the angle or lighting environment of the face included in the input whole face image to correspond to a preset standard.

The server or the mobile terminal may assign a penalty to the weight according to the focus or quality of the input whole face image or the split image for each face part.

An age estimation system according to an embodiment of the present disclosure includes a server for generating an individual age estimation model, an overall age estimation model, or a final age estimation model, and a mobile terminal receiving an entire face image of a user, wherein the server generates the individual age estimation model. When a memory for storing an age estimation model, a total age estimation model, or a final age estimation model is included, the mobile terminal includes a transmission device that transmits the received entire face image to a server, and the server receives the information from the mobile terminal. An individual age, total age, or final age is calculated from the received whole face image through the individual estimation model, the total age estimation model, or the final age estimation model stored in the memory, and the calculated individual age and total age are calculated by the mobile terminal. or a transmission device that transmits the final age, and if the server includes a transmission device capable of transmitting the individual estimation model, the total age estimation model, or the final age estimation model to the mobile terminal, the mobile terminal may send the server and a memory for storing the individual estimation model, the total age estimation model, or the final age estimation model received from the mobile terminal, wherein the mobile terminal stores the individual estimation model, the total age estimation model, or An individual age, total age, or final age is calculated through a final age estimation model, and the mobile terminal includes a display device displaying the calculated individual age, total age, or final age, and the individual age estimation model is a whole face image. It is created using training data in which a split image obtained by dividing and an actual age are paired, and the full age learning model is created using training data in which a whole face image and an actual age are paired, and the final The age estimation model may be generated using learning data in which the individual age calculated by the individual age estimation model, the total age calculated by the total age learning model, and the actual age are paired.

According to an embodiment of the present disclosure, age can be estimated even for a user who is wearing a mask or whose face is partly covered, thereby minimizing inconvenience such as having to take off the mask.

According to an embodiment of the present disclosure, the final age can be calculated by reflecting the weight according to the performance of each part, and even if some parts are covered, the final age can be calculated by excluding the weight of parts not included in the image. There is an advantage.

According to an embodiment of the present disclosure, the age estimating device has an advantage of utilizing a cosmetic performance evaluation index by estimating the age before and after using the cosmetic and comparing the estimated age.

According to an embodiment of the present disclosure, since the age of a person viewing a billboard or the like can be estimated, there is an advantage in that the advertisement can be used as a means for outputting customized advertisements suitable for each user.

According to an embodiment of the present disclosure, age can be estimated under any conditions, and the accuracy is high, so it can be used for age-restricted services such as security enhancement and age-restricted content control, or applied to specific products such as human-computer interaction. .

According to an embodiment of the present disclosure, there is an advantage in that age can be estimated using some face images of a criminal suspect or the like, or used for simulation such as face change of a missing person.

1 is a control block diagram of an age estimation device according to an embodiment of the present disclosure.

2 is a flowchart illustrating an operating method of an age estimation apparatus according to an embodiment of the present disclosure.

3 is an exemplary view of an image divided by an image segmentation unit according to an embodiment of the present disclosure.

4 is a schematic diagram illustrating the use of an ensemble model in an age estimation device according to an embodiment of the present disclosure.

5 is a diagram illustrating an example of a convolution operation according to an embodiment of the present disclosure.

6 is a diagram illustrating an example of a method for visualizing an age analysis result by an age estimation device according to an embodiment of the present disclosure.

7 is a block diagram of an age estimation system according to an embodiment of the present disclosure.

8 is a flowchart illustrating an operating method of an age estimation system according to an embodiment of the present disclosure.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffix "part" for components used in the following description is given or used interchangeably in consideration of ease of writing the specification, and does not itself have a meaning or role distinct from each other. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

The 'estimator' referred to in the present specification means a step or device for estimating each age using an estimation model obtained through machine learning using learning data paired with an actual age and an input variable as an outcome variable. .

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

An age estimation device 1 according to an embodiment of the present disclosure includes an image input unit 11, an image segmentation unit 12, an individual age estimation unit 13, a total age estimation unit 14, and a final age estimation unit 15. , at least some or all of the age output unit 16 and the control unit 17.

The image input unit 11 may receive an input of a user's face image, which is a target of age estimation.

For example, the image input unit 11 may include an image capturing device such as a camera, and in this case, the user's face image may be obtained by capturing the user's face through the image capturing device.

As another example, the image input unit 11 may include a communication module capable of transmitting and receiving data with an external device such as a mobile terminal. In this case, the user's face image is received from the external device through the communication module to receive data related to the user's face image. can be obtained.

Meanwhile, the above-described examples are merely for convenience of explanation, and the image input unit 11 may include both an image capture device and a communication module, or may include other image input means.

The entire face image input to the image input unit 11 may be a face image in which a part of the face is covered or a face image in which a part of the face is not covered.

Also, the image input unit 11 may correct the input user's face image. In particular, the image input unit 11 may standardize the user's face image. Specifically, when the input user's face image is used as it is without a separate standardization process, since the possibility of error in the age estimation process increases depending on the angle of the face or the lighting environment, the image input unit 11 may perform image standardization. there is.

More specifically, the standard for the angle of the face or the standard for the lighting environment may be previously set in the image input unit 11 . Accordingly, the image input unit 11 may correct the angle of the face included in the entire face image of the user or the lighting environment to correspond to a preset standard.

The image division unit 12 may receive the user's face image from the image input unit 11 and divide the received user's face image into a plurality of pieces.

For example, the image division unit 12 may divide the user's face image by face part.

Also, the image segmentation unit 12 may perform image reduction or image cropping as well as image segmentation. For example, when the image input unit 11 acquires a user's face image, the size of the image may vary, and even if the size of the image is fixed, the size of the face in the image may vary. You can zoom out or crop the image. Here, the 'reduced image' refers to an image that has undergone a process of reducing the number of pixels of an input image. For example, an image having a long axis of 2,500 pixels is reduced by a reduction ratio of 1/2 to 1/10, For example, it may be an image obtained by reducing an image of 2,500*2,500 pixels at a reduction ratio of 1/2 to 1/10, but is not limited thereto. The 'reduced image' may provide an effect of estimating the age only by the contour of the face or the contour of individual parts of the face without considering blemishes or fine skin conditions of the face, for example.

The individual age estimation unit 13 may calculate at least one individual age based on the face image of the user. Here, the user's face image may include both an input image such as a normalized image, a segmented image, a reduced image, and the like, and a result image obtained by performing various tasks after being input.

In particular, the individual age estimation unit 13 may calculate the individual age based on each of the divided images divided by facial parts. The individual age may mean an estimated age for each face part. The individual age may be the age for each facial part. The individual age may be at least one. That is, there may be a plurality of individual ages.

The total age estimator 14 may calculate the total age based on the user's face image. Similarly, here, the user's face image may include both an input image such as a normalized image, a segmented image, a reduced image, and the like, and a result image obtained by performing various tasks after being input.

In particular, the total age estimator 14 may calculate the total age based on at least one of a standardized image and a reduced face image. The total age may refer to an age estimated based on the entire face of the user.

The final age estimating unit 15 is based on the age calculated by the individual age estimating unit 13 alone or the combination of the age calculated by the individual age estimating unit 13 and the age estimated by the total age estimating unit 14. to extract the final age.

The final age may refer to an age finally calculated based on the user's face image. The final age may be an age finally calculated by considering or integrating only individual ages or combinations of all ages. The final age may be an age in which individual ages and total ages are combined.

The age output unit 16 may output the age calculated by analyzing the user's face image. The age output unit 16 may output at least one of individual ages, total ages, and final ages.

The age output unit 16 may be implemented as a display that visually outputs the result of analyzing the user's face image or a speaker that audibly outputs the result of analyzing the user's face image, but this is only exemplary and is not limited thereto.

The controller 17 may control the operation of the age estimation device 1 .

The control unit 17 may include at least a part of the image input unit 11, the image segmentation unit 12, the individual age estimation unit 13, the total age unit 14, the final age estimation unit 15, and the age output unit 16. Or you can control all of them.

Next, an operating method of the age estimation device 1 according to an embodiment of the present disclosure will be described in detail.

The image input unit 11 may receive an entire face image of the user (S11).

The entire face image is a user's face image, and may mean an image including the entire user's face. Here, the image containing the entire face means an image that includes all or part of the face contour, and at least 10% or more, 20% or more, 30% or more, 40% or more, 50% or more, It may be an image that includes 60% or more, 70% or more, 80% or more, 90% or more, or all facial contours.

The image input unit 11 may perform image standardization on the input entire face image of the user (S13).

Image standardization may include a task of converting an image property (eg, brightness or contrast) to be identical to a preset property, and a task of adjusting an angle of a face to a preset angle.

The controller 17 may calculate individual ages and total ages after performing image standardization. The individual age estimation process and the overall age estimation process may be separately performed. The individual age estimation process and the overall age estimation process may be performed simultaneously. Meanwhile, in FIG. 2 , for convenience of description, it is described that an individual age estimation process is performed and then the entire age estimation process is performed, but it is reasonable that the age estimation process is not limited thereto.

The image segmentation unit 12 may extract feature points for image segmentation (S15).

The image segmentation unit 12 may extract feature points in order to divide the entire face image by facial parts. For example, the image segmentation unit 12 may extract facial feature points using openCV, Dlib (image processing library), etc., but any method known in the art may be used. The feature point may be at least one or more. For example, the image segmentation unit 12 may extract a first feature point corresponding to the eyes, a second feature point corresponding to the nose, and a third feature point corresponding to the mouth from the entire face image.

The image division unit 12 may divide the face image based on feature points (S17).

The image segmentation unit 12 may segment the face image based on at least one extracted feature point. The image segmentation unit 12 may obtain a plurality of divided images of the entire face image based on feature points.

The divided image may include images for each face part.

Meanwhile, when dividing the face image, the image divider 12 may further obtain a reduced image obtained by reducing the entire face image.

As in the example shown in FIG. 3 , the image segmentation unit 12 may obtain an entire reduced image obtained by reducing the entire face image.

In addition, the image segmentation unit 12 divides the entire face image based on feature points, such that a first divided image including the forehead, a second divided image including the left eye, and a third division including the right eye image, a fourth segmented image including a nose, a fifth segmented image including a left cheek, a sixth segmented image including a right cheek, and a seventh segmented image including a mouth.

On the other hand, since the reduced image and divided image shown in FIG. 3 are only examples, it is appropriate that they are not limited thereto.

Again, FIG. 2 is described.

The individual age estimation unit 13 may estimate individual ages through the divided images (S19).

That is, the individual age estimating unit 13 may estimate the individual age, which is the age of each facial part, based on at least one divided image. The individual age may be at least one. For example, the individual age estimation unit 13 calculates the age corresponding to the forehead, the age corresponding to the eyes, the age corresponding to the nose, the age corresponding to the cheeks, and the age corresponding to the lips, thereby estimating the individual age. can

Meanwhile, the total age estimation unit 14 may estimate the total age through the entire face image (S21).

The total age estimator 14 may calculate the total age based on at least one of a standardized whole face image or a normalized and reduced whole face image.

Meanwhile, each of the individual age estimating unit 13 and the total age estimating unit 14 may use an ensemble model when calculating the individual age and the total age.

An ensemble can be a model built on top of a machine learning paradigm where multiple models are trained to solve the same problem and get better results. The ensemble model may be a generalized model by combining several individually learned models. An ensemble model may be an estimation technique using multiple learning models.

The individual age estimator 13 may apply an ensemble model to estimate individual ages. That is, the individual age estimation unit 13 may use the ensemble model when calculating the age corresponding to the forehead, the age corresponding to the eyes, the age corresponding to the nose, the age corresponding to the cheeks, and the age corresponding to the lips. .

Also, the total age estimator 14 may use an ensemble model when calculating the total age.

Referring to the example of FIG. 4 , the overall age estimator 14 may finally calculate the average of ages estimated through 10 individual learning models for the entire face image as the overall age.

In addition, the individual age estimator 13 may finally calculate the average of the forehead ages estimated through 10 individual learning models for the divided images including the forehead as the age corresponding to the forehead. Similarly, the individual age estimator 13 converts the average of the eye ages estimated through 10 individual learning models for the divided images including the left eye part and the divided image including the right eye part into the final age corresponding to the eye. can be calculated The individual age estimator 13 may finally calculate the average of nose ages estimated through 10 individual learning models for the segmented images including the nose as the age corresponding to the nose. The individual age estimator 13 finally calculates the average of the cheek ages estimated through 10 individual learning models for the divided images including the left cheek and the divided images including the right cheek as the age corresponding to the cheek. can The individual age estimator 13 may finally calculate the average of the lip ages estimated through 10 individual learning models for the segmented image including the lip portion as the age corresponding to the lips.

The above individual learning model can be implemented as an AI structure, especially a MobileNet structure among CNNs. Convolutional Neural Network (CNN) is a major method of image-based deep learning, which utilizes the two-dimensional characteristics of images.

Deep learning is an algorithm that abstracts (extracts key information from a large amount of data) through a combination of various nonlinear calculations. Existing learning was judged after extracting features through PCA and pattern recognition. Because deep learning learns using a large amount of data, there are no a priori biases or assumptions.

Convolution is a calculation utilization method using a filter of adjacent pixel values, and the calculation method is illustrated in FIG. 5 as an example.

For example, in the case of performing a convolution operation on a black-and-white image, a pixel with a value of 0 may be added to a border for an adjacent pixel operation. Then, for each pixel, an addition operation after filtering and multiplication can be performed while pushing one space to the right. A result of vertical emphasis and a result of horizontal emphasis through the convolution operation are shown as an example.

MobileNet can be a method of reducing image-based deep learning model parameters and operations as a method of CNN. A general CNN is a 3*3*3 filter operation in which three RGB colors are superimposed on adjacent 3*3 pixels, but MobileNet is a 3*3+3 operation by separating the 3*3 filter operation and the RGB 3, which is 512*512 For images, about 4,800,000 parameters can be learned.

As in the present disclosure, when using 10 individual learning models, about 100 layers and about 4,800,000 or more parameters may be included.

When the ensemble model is used as described above, there is an advantage in that estimation stability and accuracy are improved in machine learning.

Again, FIG. 2 is described.

The final age estimating unit 15 may estimate the final age of the user by integrating division or division/whole image estimation values (S23).

The division image estimation value may refer to an individual age calculated through division images. The total image estimation value may mean the total age calculated through the entire face image.

The final age estimator 15 may calculate the final age by integrating the divided image estimation values and/or the entire image estimation values.

That is, the final age estimation unit 15 may calculate the final age by integrating individual ages and/or all ages.

According to the first embodiment, the final age estimation unit 15 may calculate an average of individual ages or an average of individual ages and total ages as the final age.

According to the second embodiment, the final age estimator 15 may calculate the average of the remaining values as the final age after excluding the highest value and the lowest value among individual ages and total ages.

In the case of the first and second embodiments, there is an advantage in that the calculation method is relatively simple, but age estimation accuracy is different for each part, and there may be a limitation in that such performance for each part is not reflected.

According to the third and fourth embodiments, the final age estimating unit 15 may calculate an average of individual ages and/or total ages as the final age after assigning weights. That is, according to the third and fourth embodiments, the final age estimator 15 reflects the weight when calculating the final age, but the weight calculation method may be different. Hereinafter, a weight calculation method will be described.

In the case of the third embodiment, the final age estimator 15 may calculate weights by learning a linear regression equation. In this case, there is an advantage in that the performance of each part is reflected, but there is a limitation in that weights of all parts are required. That is, in this case, it may be difficult to calculate the weighted final age if there is no weight for a specific part.

In the case of the fourth embodiment, the final age estimator 15 may assign 1/estimation error as a weight of individual models for each part and then calculate the weighted average of the estimated age for each part as the final age. Specifically, the final age estimation unit 15 may calculate the final age through Equation 1 below.

In Equation 1 above, A is the age, W is the weight, I is the final integration, T is the entire face, F is the forehead, E is the eye, N is the nose, C is the cheek, and L is the lips. That is, Equation 1 indicates that the weight average, which is the final age, is a value obtained by dividing the sum of values obtained by multiplying the weight for each part by the age of the corresponding part by the sum of the weights.

And, the weight W is determined by 1/MAE, and the weight of the non-measurable item (part) may be 0. The non-measurable item (part) may refer to a face part that is not recognized because it is covered by a mask or the like.

Optionally, the final age estimator 15 may apply a penalty to the weight when the input image or the segmented image for each face part is out of focus or according to the quality of the input or segmented image. Specifically, the final age estimator 15 may impose a penalty on weights according to the focus or quality of the input whole face image or divided images for each face part. For example, the final age estimator 15 may impose a penalty on weights when the entire face image or the divided images are out of focus. Alternatively, the final age estimator 15 may calculate the quality of the whole face image or the segmented image, and if the calculated quality is lower than a predetermined threshold value, a weight penalty may be applied. Here, the quality may be calculated as PSNR (Peak Signal-to-Noise Ratio) or SSIM (Structural Similarity Index Map), but since this is only an example, it is reasonable not to be limited thereto.

In addition, the final age estimator 15 may decrease the weight by a predetermined value or a predetermined ratio when a penalty is imposed. However, since this is merely an example, it is reasonable not to be limited thereto.

The final age estimator 15 may automatically determine a weighted value for the total age as 0 when there is an unmeasurable item (part).

That is, the final age estimator 15 may determine the weight W of each part as 1/MAE, and may determine the weight W of the non-measurable item as 0 based on the entire image and the divided images. The final age estimator 15 may determine the weight W for each part as the reciprocal of the MAE.

Here, MAE is an abbreviation of Mean Absolute Error, and may represent the average of absolute value errors. MAE learned a total of 10 models through original full face images of 2592*3888 size for 37,080 subjects, split images with original resolution of 512*215 or less, and full face images for training compressed to 512*512 size, 6 per data It can be obtained through the result of synthesizing the estimation results of the dogs, which can be shown in Table 1 below.

부위part	MAEMAE	SDSD	CA-3CA-3	CA-5CA-5
전체entire	2.462.46	2.162.16	68.568.5	89.989.9
이마forehead	4.994.99	4.014.01	38.138.1	58.858.8
눈 (평균)snow (average)	3.533.53	2.952.95	52.052.0	75.475.4
코nose	3.843.84	3.113.11	47.947.9	71.071.0
볼 (평균)ball (average)	4.464.46	3.683.68	42.242.2	64.464.4
입술Lips	3.703.70	3.083.08	49.949.9	73.173.1
통합 (1/MAE)Integration (1/MAE)	2.442.44	2.152.15	68.868.8	90.090.0
통합(가중치 수동 조정)Integration (manual adjustment of weights)	2.432.43	2.152.15	69.069.0	90.290.2

The table above is a model that is learned as a set and generated estimation results that are not included in the set. 4 1/2 data learning models (Model A, B, C, D), 2 estimation results for each data 6 1/3 data learning models (

models

0, 1, 2, 37, 46, 58), and 2 estimation results for each data were generated (37 is set3 and 7 data learning models, the intersection of other learning models and data In the table above, MAE (Mean Absolute Error) is the average difference between the estimated value and the actual value in the data not used in learning, SD (Standard deviation) is the standard deviation, CA-3 (Cumulative Accuracy 3) is the error is 3 The ratio (%) under age 5, CA-5 (Cumulative Accuracy 5) indicates the ratio (%) under 5 years of age with an error, and the average of both image-based age estimates was used for the eyes and cheeks. CA-k is the accuracy when it is assumed that the difference between the estimated value and the measured value is equal to or less than k, and may be a scale made to talk about age prediction accuracy in % units. In this way, according to the fourth embodiment, the final The age estimator 15 may assign a separate weight to each part, and calculate an average of the weighted individual age and overall age for each part as the final age. And, the final age estimator 15 may determine the weight of the non-measurable part as 0.

Next, a method for the final age estimation unit 15 to calculate the final age using Equation 1 above will be described.

for example,

,

are 6, 5, 4, 3, 2, 1 respectively,

,

are 20, 17, 20, 22, 19, and 34 respectively, the final age estimation unit 15

can be calculated as (6*20+5*17+4*20+3*22+2*19+1*34)/(6+5+4+3+2+1)=20.14.

As another example, when the weights for the nose and mouth are 0 because the user is wearing a mask, the final age estimator 15 calculates the weights for the eyes and nose.

and

is set to 0, so

(6 * 20 + 5 * 17 + 4 * 20 + 0 + 2 * 19 + 0) / (6 + 5 + 4 + 0 + 2 + 0) = 19 can be calculated.

In this way, according to the fourth embodiment, there is an advantage in that the final age can be calculated by reflecting the weight according to the performance of each part. In addition, even if some parts are covered, there is an advantage in that the final age can be calculated by excluding weights of parts not included in the image.

In addition, the final age estimating unit 15 may calculate the final age using weight adjustment and regression or a form of an all-age matching table. Through this, it may be possible to correct the tendency of predicting that young people are slightly older and older people are slightly younger.

Again, FIG. 2 is described.

The age output unit 16 may visualize the analysis result (S25).

That is, the age output unit 16 may display the final age. In addition, the age output unit 16 may display the individual age, which is the age for each part, together with the final age.

Alternatively, the age output unit 16 may also display the average individual age of people of the same age while displaying the individual age for each part.

As shown in FIG. 6 , the age output unit 16 may display the individual ages of people of the same age as a radial chart along with the individual ages of the user. The age output unit 16 may visualize the individual age of the user and the individual age of people of the same age in the form of a radar plot. That is, the age output unit 16 may display the age of each part along with the average age of people of a similar age other than the user's age.

If the output shown in FIG. 6 is interpreted, it can be seen that the more the yours line representing the user's age goes inward, the more 'younger' and the more outward it is, the more it is diagnosed as 'older'. As a result, it can be seen that 'looks young', and eyes and lips are relatively 'looks old'.

When the age output unit 16 outputs the age analysis result, it may further display a guide such as care/product recommendation. In the case of the example of FIG. 6 , the age output unit 16 may recommend intensive care for the eyes and lips, and may emphasize the eye area.

That is, the age output unit 16 may show the severely aged parts of the face and recommend suitable procedures or cosmetics.

Also, the final age estimation unit 15 may predict skin age. For example, the final age estimator 15 may calculate skin age by integrating ages of specific parts (eg, forehead + eyes + cheeks) among facial parts.

The age output unit 16 may diagnose the degree of aging (eg, severe, moderate, or good). And, accordingly, the age output unit 16 may classify the aging part reference type (eg, eye aging type, forehead aging type).

Meanwhile, the above-described age estimating device 1 may be a device manufactured separately to acquire and output an age, or may be a mobile terminal such as a smart phone. That is, the mobile terminal may be an age estimation device that obtains and outputs an age.

Also, the present disclosure may be implemented as an age estimation system that estimates and outputs an age through communication between a plurality of devices.

The age estimation system may include a mobile terminal 110, a server 120, and a data providing server 130. In the age estimation system according to the present invention, the mobile terminal 110 or the server 120 may be the above-mentioned age estimation device, may include some or all of the features of the above-mentioned age estimation device, and may include the terminal 110 and The server 120 may perform each function by dividing the function of the age estimation device described above. In the following, functions of the age estimating device are divided into exemplary descriptions of performing each function in the system.

The mobile terminal 110 may be a user terminal such as a smart phone, a tablet PC, or a laptop computer.

The server 120 may obtain an age estimation model. The age estimation model may include an individual age estimation model, an overall age estimation model, and a final age estimation model. The server 120 may obtain an age estimation model based on data received from the data providing server 130 .

The server 120 may directly obtain the individual age, total age, and final age based on the age estimation model, or use the mobile terminal 110 to obtain the individual age, total age, and final age from the mobile terminal 110. An estimation model may be transmitted.

The data providing server 130 may store and update data necessary for calculating an age estimation model. The data providing server 130 may store and update algorithms and user information necessary for calculating an age estimation model.

The data providing server 130 may store data necessary for calculating an age estimation model. The data providing server 130 may transmit data necessary for calculating the age estimation model to the server 120 .

The data providing server 130 may directly obtain an age estimation model based on data necessary to calculate the age estimation model. Alternatively, the data providing server 130 may transmit data necessary for calculating the age estimation model to the server 120 so that the server 20 may acquire the age estimation model.

Next, referring to FIG. 8 , a method of obtaining an age estimation model by the age estimation system, estimating an age based on the age estimation model, and visualizing the analyzed result will be described in detail.

The data providing server 130 may store data necessary for obtaining an age estimation model (S111).

That is, the data providing server 130 may store and manage data necessary for obtaining an age estimation model.

The data providing server 130 may transmit data necessary for acquiring the age estimation model to the server 120 (S113).

The server 120 may receive data necessary for obtaining an age estimation model from the data providing server 130 .

The server 120 may obtain an age estimation model based on data necessary for acquiring the age estimation model (S115).

The age estimation model may include at least one of an individual age estimation model, an overall age estimation model, and a final age estimation model.

The server 120 may include a memory (not shown) for storing an individual age estimation model, a total age estimation model, or a final age estimation model.

An individual age estimation model may be generated using training data in which a segmented image obtained by segmenting an entire face image and an actual age are paired. The full age learning model may be generated using training data in which a full face image and an actual age are paired. The final age estimation model may be generated using learning data in which the individual age calculated by the individual age estimation model, the total age calculated by the total age learning model, and the actual age are paired.

Meanwhile, the mobile terminal 110 may receive the user's entire face image (S117).

The mobile terminal 110 may transmit the entire face image of the user to the server 120 (S119). The server 120 may receive the entire face image of the user from the mobile terminal 110 .

Alternatively, the mobile terminal 110 may divide the entire face image of the user to obtain divided images, and transmit both the entire face image and the divided images to the server 120 . The mobile terminal 110 may further include a transmission device (not shown), and may transmit an entire face image or a divided image through the transmission device (not shown). The server 120 may receive only the entire face image, and in this case, it may obtain divided images by dividing the entire face image. The server 120 may receive both the entire face image and the divided images.

Upon receiving the user's entire face image from the mobile terminal 110, the server 120 may analyze the user's entire face image to obtain an individual age, total age, and final age (S121).

The server 120 may calculate an individual age, a total age, or a final age by applying at least one of the full face image and the segmented images to an individual age estimation model, an overall age estimation model, or a final age estimation model.

The server 120 may transmit individual ages, total ages, and final ages to the mobile terminal 110 (S123).

The server 120 may further include a transmission device (not shown), and may transmit the individual age, total age, and final age to the mobile terminal 110 through the transmission device (not shown).

The mobile terminal 110 may visualize the analysis result (S125).

The mobile terminal 110 may include a display device (not shown) displaying analysis results. The mobile terminal 110 may display individual ages, total ages, or final ages through a display device (not shown).

That is, the mobile terminal 110 may output at least one of individual ages, total ages, and final ages. In addition, the mobile terminal 110 may also display the average individual age of people of the same age while displaying the individual age for each part.

According to an embodiment of the present invention, the age estimation model acquired and stored in the server 120 is transmitted to the mobile terminal 110, and the mobile terminal 110 does not need to transmit the entire face image of the user to the server 120. The individual age, total age, and final age may be directly obtained by performing the above step (S121).

Specifically, the mobile terminal 110 may receive and store an individual age estimation model, an overall age estimation model, or a final age estimation model from the server. The mobile terminal 110 may include a memory (not shown) for storing an individual age estimation model, a total age estimation model, or a final age estimation model. The mobile terminal 110 may directly receive an entire face image and obtain divided images by dividing the received entire face image. The mobile terminal 110 may calculate an individual age, total age, or final age by applying at least one of the entire face image and the segmented images to an age estimation model, a total age estimation model, or a final age estimation model.

According to an embodiment of the present invention, the individual age, total age, and final age obtained by the server 120 or the mobile terminal 110 are transmitted and stored again to the data providing server 130, and the mobile terminal 110 The age estimation model may be updated by the server 120 again by being provided from and stored in the data providing server 130 as additional learning data together with the actual age stored in the data providing server 130.

The present disclosure described above can be implemented as computer readable codes in a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is

The above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of this specification should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of this specification are included in the scope of this specification.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments.

The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims

a server that creates and stores an individual age estimation model, an overall age estimation model, or a final age estimation model; and

A mobile terminal displaying an individual age, a total age, or a final age calculated based on the individual age estimation model, the total age estimation model, or the final age estimation model;

The server

A full face image of the user is received from the mobile terminal, and at least one of the received full face image and a split image obtained by dividing the entire face image is applied to the individual age estimation model, the total age estimation model, or the final age estimation model. Calculating an individual age, total age, or final age, and transmitting the calculated individual age, total age, or final age to the mobile terminal

age estimation system.
The method of claim 1,

The server

Receiving only the entire face image from the mobile terminal;

Receiving both the whole face image and the split image from the mobile terminal

age estimation system.
a server that generates an individual age estimation model, an overall age estimation model, or a final age estimation model; and

The individual age estimation model, the total age estimation model, or the final age estimation model is received and stored from the server, and the individual age, total age, or final age estimation model calculated based on the stored individual age estimation model, total age estimation model, or final age estimation model is stored. Including a mobile terminal displaying the age,

The mobile terminal

The user's entire face image is received as an input, and at least one of the received full face image and the divided image obtained by dividing the entire face image is applied to the individual age estimation model, the total age estimation model, or the final age estimation model, to calculate age or final age

age estimation system.
The method according to any one of claims 1 or 3,

The server or the mobile terminal

Obtaining the divided image for each face part by dividing the entire face image for each face part;

The individual age is the age for each facial part

age estimation system.
The method of claim 4,

The server or the mobile terminal

Extracting feature points for segmentation by face part, and dividing the entire face image based on the extracted feature points

age estimation system.
The method according to any one of claims 1 or 3,

The server or the mobile terminal

Calculating the average of the individual age and the total age as the final age

age estimation system.
The method according to any one of claims 1 or 3,

The server or the mobile terminal

After weighting each part of the face, calculating the weighted average of the estimated age for each part as the final age

age estimation system.
The method of claim 7,

The weighted average is

The value obtained by dividing the sum of the values multiplied by the weight for each part by the age of the corresponding part by the sum of the weights

age estimation system.
The method of claim 7,

The server or the mobile terminal

Determines the weight of the non-measurable part as 0

age estimation system.
The method of claim 5,

The server or the mobile terminal

Determining the weight as the reciprocal of the average of the absolute value errors

age estimation system.
The method according to any one of claims 1 or 3,

The server or the mobile terminal

When estimating the individual age, an ensemble model is used.

age estimation system.
The method of claim 11,

Further comprising a data providing server for storing data necessary for obtaining an age estimation model,

The server or the mobile terminal

Obtaining an age estimation model to which the ensemble model is applied based on the data received from the data providing server

age estimation system.
The method according to any one of claims 1 or 3,

The server or the mobile terminal

Correcting the angle or lighting environment of the face included in the input whole face image to correspond to a preset standard

age estimation system.
The method of claim 7,

The server or the mobile terminal

Applying a penalty to the weight according to the focus or quality of the input whole face image or the divided image for each face part

age estimation system.
a server that generates an individual age estimation model, an overall age estimation model, or a final age estimation model; and

Including a mobile terminal that receives a user's entire face image,

When the server includes a memory for storing the individual age estimation model, the entire age estimation model, or the final age estimation model, the mobile terminal includes a transmission device for transmitting the received entire face image to the server, and the server calculates an individual age, a total age, or a final age from the entire face image transmitted from the mobile terminal through the individual estimation model, total age estimation model, or final age estimation model stored in the memory, and calculates the calculated age with the mobile terminal. a transmission device for transmitting an individual age, total age or final age;

When the server includes a transmission device capable of transmitting the individual estimation model, the total age estimation model, or the final age estimation model to the mobile terminal, the mobile terminal receives the individual estimation model and the total age estimation model transmitted from the server. or a memory for storing a final age estimation model, wherein the mobile terminal determines the individual age, total age or final age estimation model stored in the memory from the received whole face image. Calculate the final age,

The mobile terminal includes a display device displaying the calculated individual age, total age, or final age,

The individual age estimation model is generated using training data in which a segmented image obtained by segmenting an entire face image and an actual age are paired,

The full age learning model is generated using learning data paired with a full face image and an actual age,

The final age estimation model is generated using learning data in which the individual age calculated by the individual age estimation model, the total age calculated by the total age learning model, and the actual age are paired.

age estimation system.