KR20200021733A - Software tool to rate wrinkled skinusing Deep Learning - Google Patents

Abstract

The present invention relates to a deep learning-based wrinkle evaluation apparatus. The deep learning-based wrinkle evaluation apparatus comprises: at least one grade evaluation model trained based on deep learning to output a result value of evaluating one of grades in a range of (N+1) grades (N is a natural number) according to a naked eye determination criterion for an input wrinkle image; and an evaluation grade output part determining and outputting a wrinkle image evaluation value according to the result value output from the at least one grade evaluation model.

Description

Deep learning based wrinkle evaluation device {Software tool to rate wrinkled skinusing Deep Learning}

The present invention relates to an apparatus for evaluating wrinkles, and more particularly, to an apparatus for evaluating wrinkles on a deep learning basis.

Wrinkle evaluation can be divided into image analysis and visual evaluation using a device, of which wrinkle can be divided into two types.

Recent advances in optical technology have resulted in more precise wrinkle evaluation results obtained through image analysis. However, in the wrinkle evaluation using the image analysis technology, the evaluation result may be different from the visual evaluation result. For example, when visually evaluating wrinkles around the eyes, the wrinkles are classified into large / medium / small or several stages based on a complex judgment based on the starting area, depth, number, and length of the wrinkles. On the other hand, the evaluation result by the image analysis technology evaluates the length, volume and number as parameters. Although the field of precision mechanics has evolved in medical engineering, visual evaluation is still a major parameter.

On the other hand, even if a trained expert is trained for visual evaluation, multi-point evaluation or different investigators may affect the evaluation result. That is, according to the criteria of the visual evaluation evaluator, even the same wrinkle pattern may be evaluated as a high grade or a low grade.

Accordingly, the present invention provides a deep learning based wrinkle evaluation apparatus capable of outputting a comprehensive result such as visually evaluating wrinkles through image analysis.

In addition, the present invention provides a deep learning-based wrinkle evaluation apparatus that can overcome the problem that the wrinkle pattern is evaluated differently according to the criteria of the evaluator at the time of visual evaluation.

The present invention is a deep learning-based wrinkle evaluation apparatus, deep learning-based learning to output the result of evaluating the input wrinkle image in the range of the N + 1 (N is a natural number) grades according to the visual judgment criteria And an evaluation grade output unit configured to determine and output a wrinkle image evaluation value according to a grade evaluation model and a result value output from the grade evaluation model.

According to another aspect, the rating model is a regressor that outputs one of a range of discrete real values and outputs N binary classifications as one of '0' and '1'. Binary Classifiers are included.

According to the first embodiment of the other aspect, the grading model outputs a value indicating that the wrinkle image is the first grade when the wrinkle image is less than or equal to the first grade among N + 1 grades, and when the wrinkle image is more than the second grade. It is driven by the first first binary classifier driving the second first binary classifier and the M-1 first binary classifier, so that the wrinkle image is generated when the wrinkle image is less than or equal to the Mth of the N + 1 grades. M-th (M is an integer of 2≤M≤N-1) first binary classification, outputting a value indicating the M-th grade, and driving the M + 1-th first binary classification unit when the wrinkle image is M + 1-th or higher. And, driven by the N-th first binary classification unit, output a value indicating that the wrinkle image is the N-th grade when the wrinkle image is less than or equal to the N-th grade among N + 1 grades, and the wrinkle image is N +. Wrinkle image is class N + 1 for 1st grade or higher N-th first binary value and outputting a classification indicating comprises parts.

According to a second embodiment of another aspect, the rating model outputs a value indicating that the wrinkle image is the N + 1th grade when the wrinkle image is greater than or equal to the N + 1th rank among the N + 1 grades, and the wrinkle image is N The first image is driven by the first second binary classifier that drives the second second binary classifier and the M-1 second binary classifier when the first class is less than the first grade, so that the wrinkle image is N + 1-among the N + 1 grades. If the (M-1) th rank or more, a value indicating that the wrinkle image is the N + 1- (M-1) th rank is output, and when the wrinkle image is the N + 1-M th rank or less, the M + 1 th binary classification M-th (M is an integer of 1≤M≤N) driving the negative, driven by the second binary classifier and the N-1th binary classifier, where the wrinkle image is the second or more of the N + 1 grades. Outputs a value indicating that the wrinkle image is of the 2nd grade, and if the wrinkle image is below the 1st grade, the wrinkle image is 1 The N-th second containing binary classification unit configured to output the value indicating that the second grade.

According to the third embodiment of the other aspect, the rating model is a value indicating that the wrinkle image is the N + 1 th grade when the wrinkle image is greater than or equal to the N + 1 th rank among the N + 1 grades in the components of the second embodiment. If the wrinkle image is less than the N-th grade, the first binary binary classification unit for driving the second second binary classification unit, and the M-1 second binary classification unit, the wrinkle image is N + 1 If the N + 1- (M-1) th rank or more among the ranks, a value indicating that the wrinkle image is the N + 1- (M-1) th rank is output, and the wrinkle image is below the N + 1-M th rank. In this case, the M-th (M is an integer of 1≤M≤N) driving the M + 1 th binary classification unit and the N-1 th binary classification unit are driven, whereby N + 1 wrinkle images are generated. If it is above the 2nd grade among the grades, it outputs a value indicating that the wrinkled image is the 2nd grade, Or less in the image further comprises folds parts N-th second binary classifier to output a value indicating that first grade.

According to a third exemplary embodiment of the present disclosure, the evaluation grade output unit outputs a first evaluation grade output unit for outputting wrinkle image evaluation values according to values output from the N first binary classifiers, and the N second binary classifiers. And a second evaluation grade output unit configured to output the wrinkle image evaluation value according to the value.

According to a third exemplary embodiment of the present disclosure, an evaluation grade of determining and outputting a final wrinkle evaluation grade based on a probability value corresponding to each of the wrinkle image evaluation values output from the first evaluation grade output unit and the second evaluation grade output unit, respectively. The final decision unit further includes.

According to another aspect, the apparatus may further include a cropping unit extracting a region of interest from which the wrinkle pattern is distributed from the input wrinkle image, and a scaling unit to scale the region of interest image to a predetermined size.

According to yet another aspect, the apparatus may further include a normalization unit configured to convert the input wrinkle image into grayscale and normalize and output the average / standard deviation of the luminance value distribution of the input image.

According to the present invention, since the wrinkles are evaluated using the deep learning-based wrinkle evaluation model learned using the visually evaluated images as the training data, the evaluation result according to the comprehensive judgment such as the naked eye evaluation can be derived.

In addition, since the deep learning-based wrinkle evaluation model trained using training data evaluated by a plurality of evaluators with different evaluation criteria is used, an evaluation result based on an average standard among a plurality of evaluators is derived. You can narrow the difference.

1 is a block diagram of a deep learning-based wrinkle evaluation apparatus according to an embodiment of the present invention.
2A to 2D are exemplary views of wrinkle evaluation target images and evaluation results thereof according to the present invention.
3 is a detailed block diagram of a wrinkle evaluation engine according to a first embodiment of the present invention.
4 is a detailed block diagram of a wrinkle evaluation engine according to a second embodiment of the present invention.
5 is a detailed block diagram of a wrinkle evaluation engine according to a third embodiment of the present invention.

Hereinafter, a deep learning based wrinkle evaluation apparatus according to a preferred embodiment will be described in detail with reference to the accompanying drawings. Here, the same reference numerals are used for the same configurations, and detailed descriptions of well-known functions and configurations that may unnecessarily obscure the repeated description and the subject matter of the invention will be omitted. Embodiments of the invention are provided to more completely describe the invention for those skilled in the art. Accordingly, the shape and size of elements in the drawings may be exaggerated for clarity.

Combinations of each block of the accompanying block diagrams and respective steps of the flowcharts may be performed by computer program instructions (execution engines), which may be executed on a processor of a general purpose computer, special purpose computer or other programmable data processing equipment. As such, instructions executed through a processor of a computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored therein to produce an article of manufacture containing instruction means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And computer program instructions can also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps are performed on the computer or other programmable data processing equipment to create a computer-implemented process that can be executed by the computer or other programmable data. Instructions for performing data processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing specific logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that the functions may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently, and the blocks or steps may also be performed in the reverse order of the corresponding function as required.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, embodiments of the present invention illustrated in the following may be modified in many different forms, the scope of the present invention is not limited to the embodiments described in the following. Embodiments of the invention are provided to more fully describe the invention to those skilled in the art.

1 is a block diagram of a deep learning-based wrinkle evaluation apparatus according to an embodiment of the present invention, Figure 2 is an illustration of a wrinkle image and the evaluation result according to the present invention.

Referring to FIG. 1, the deep learning based wrinkle evaluation apparatus 1 visually determines a wrinkle progression of a target area in a wrinkle image output from a wrinkle image input unit 10 and a wrinkle image input unit 10 obtaining a wrinkle image. And a wrinkle evaluation engine unit 100 for outputting an evaluation result as described above.

The wrinkle image input unit 1 is connected to the photographing device through an interface or a wired or wireless network. In addition, the wrinkle image input unit 1 may perform preprocessing by accessing a target image from a memory (not shown). Here, the memory may be a portable removable memory or a memory provided in the deep learning based wrinkle evaluation device 1. Here, the preprocessing process is for easily extracting the wrinkle pattern from the wrinkle image, and for example, wrinkle pattern extraction, scaling, normalization, and the like may be performed on the input image.

The wrinkle pattern detection unit 11 may crop and extract a partial image of the target region in which the wrinkle pattern is distributed from the input image. For example, when the input image is the entire face image, an area in which wrinkles are distributed, such as around the eyes and around the mouth, may be extracted as the partial image. The scaling unit 12 reduces the partial image of the target area extracted by the cropping unit 21 to a predetermined size. For example, it may be reduced to a size of 250x150. The normalizer 23 converts the scaled image to grayscale and normalizes the average / deviation normalization of the luminance value distribution of the image. As a result, the burden of the evaluation process of the wrinkle evaluation engine unit 100 may be reduced, and the learning data may be reduced when the wrinkle evaluation engine unit 100 is previously learned. That is, the larger the size of the input image or the greater the distribution of luminance values, the more learning data is required.

Recently, deep learning has been used in various fields of artificial intelligence such as image classification and speech recognition. This development is because deep neural networks can effectively learn complex probability distributions through backpropagation.

Accordingly, the wrinkle evaluation engine unit 100 may include at least one grade evaluation model trained on a deep learning basis to classify an input wrinkle image into one of N + 1 grades (N is a natural number) according to visual judgment criteria; The apparatus may include an evaluation grade output unit configured to output a wrinkle image evaluation value according to a value output from at least one grade evaluation model.

Here, the learning algorithm is a deep learning based algorithm, for example, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a sparse autoencoder, a deformable part model (DPM), and a support vector (SVM). Various algorithms can be used to generate a learning model that can evaluate wrinkled images, including Machine) and Vgg Net.

The evaluation result may be one of a plurality of evaluation levels in which the wrinkle progression of the target region in the wrinkle image is visually evaluated according to the depth, the number of wrinkles, and the like. For example, the evaluation grade may be composed of ten grades of '0' to '9', and as shown in (a) of FIG. 2, when the wrinkles around the eyes are hardly found, the grade is evaluated as '0'. As shown in (b) to (d) of FIG. 2, it may be evaluated as grade 2, grade 4, grade 6, or the like according to the distribution of the depth, number, etc. of the wrinkles. Here, in the illustrated wrinkle image, the target area is shown around the eye, but this is merely an example, and the present invention is not limited thereto. That is, the target area may include, for example, all areas where wrinkles may appear with age, such as forehead, mouth, forehead, and neck.

In addition, in training the wrinkle evaluation engine unit 100, wrinkle images used as a training data set are evaluations completed, and may be a plurality of wrinkle images in which evaluation values for which visual evaluation is completed by a person are labeled. In addition, since the evaluation value may be different for each person even for the same wrinkle image, a plurality of wrinkle image data labeled with different evaluation values may be used for the same wrinkle image. In addition, 10 to 100 wrinkle images may be used as a data set for each grade.

In addition, according to an aspect, the grade evaluation model of the wrinkle evaluation engine unit 100 includes a continuous real value of a predetermined range so as to classify the input wrinkle image into one of a predetermined range of grade values according to visual judgment criteria. It can be a regressor that outputs to one of the following. Then, when the real value is output from the grading model, the grading rating output unit may output a wrinkle image evaluation value of one of N + 1 ratings (N is a natural number) as an integer through a rounding criterion.

In addition, according to another aspect, the grading model of the wrinkle evaluation engine unit 100 is a plurality of binary classifications outputting one of the values of '0' and '1' as an evaluation value for evaluating an input wrinkle image according to visual criteria. Binary Classifiers may be included. That is, the wrinkle evaluation engine unit 100 may be configured by cascading the N binary classification units learned on the basis of a plurality of deep learning or machine learning. As a result, the accuracy can be improved rather than configuring the rating model as one learning model. At this time, the wrinkle evaluation engine unit 100 may be various embodiments.

According to the first embodiment, each of the first binary classifiers of the wrinkle evaluation engine unit 100 may be configured in a cascaded form in ascending order of grades classified by itself.

According to the second exemplary embodiment, the second binary classifiers of the wrinkle evaluation engine unit 100 may be configured to be cascaded in descending order of grades classified by themselves.

According to the third embodiment, the wrinkle evaluation engine unit 100 is configured in such a manner that the first binary classification units cascaded in ascending order and the second binary classification units cascaded in descending order are connected in parallel. It may be configured to determine the final wrinkle evaluation grade through a probability comparison of wrinkle image evaluation values output from each. This allows error checking between the two groups, further improving the accuracy of the evaluation results. Detailed description of each of the above-described first to third embodiments will be described later with reference to FIGS. 3 to 5.

3 is a detailed block diagram of a wrinkle evaluation engine according to a first embodiment of the present invention.

Referring to FIG. 3, N first binary classifiers 110-1 (N is a natural number) trained on a deep learning basis to classify an input wrinkle image into one of N + 1 grades according to visual criteria. 110-2, ..., 110-N) and the wrinkle image evaluation value according to the values output from the N first binary classification unit (110-1, 110-2, ..., 110-N) The first evaluation grade output unit 120 may be included.

Here, the N + 1 grades may have a first grade of grade '0' and the N + 1th grade of grade 'N'. For example, when the number of wrinkle image grades is 10, it may be made of 10 grades of '0' to '9', and the number of binary classification units 110-1, 110-2, ..., 110-N is It can be nine. However, this is only one embodiment, and the present invention is not limited thereto. That is, the N + 1 grades may be set to the first grade as the grade '1' and the N + 1th grade as the grade 'N + 1'.

Each of the first binary classifiers 110-1, 110-2,..., 110 -N is mapped to a rank of one of N + 1 grades so that an input wrinkle image is mapped to itself. It is determined whether or not it is below the grade of the given sequence and output as a result.

That is, the first first binary classification unit 110-1 outputs a value indicating that the wrinkle image is the first grade when the wrinkle image is less than the first grade among the N + 1 grades, and the wrinkle image is the second grade or more. In this case, the second first binary classification unit 110-2 is driven.

For example, when the number of wrinkle image grades is 10 and 10 grades of '0' to '9', the binary classification unit 1 (110-1) determines whether the input wrinkle image is less than the grade '0'. The pre-trained model, binary classifier 1 (110-1) is trained with a data set of wrinkled images labeled with a rating '0' or less, and images labeled with a rating '1' or more.

In addition, the binary classification units 110-1, 110-2,..., 110 -N may be output as one of values of '0' and '1' as evaluation values evaluated for the wrinkle image. Accordingly, the binary classifier 1 110-1 outputs '0' if the wrinkle image input is less than or equal to '0', and outputs to the first binary classifier 110-2 if the wrinkle image is greater than or equal to '1'. Print '1'.

M-th (M is an integer of 2≤M≤N-1). The first binary classification unit 110-2, 110-3, ... 110- (N-1) is the M-1th first binary. Driven by the classification unit 110-1, 110-2,..., 110-(N-2) to indicate that the wrinkle image is the M-th grade if the wrinkle image is less than or equal to the M-th grade among N + 1 grades. Outputs a value, and drives the M + 1st first binary classification unit 110-3, 110-4,.

For example, if the number of wrinkle image grades is 10 and is made of 10 grades of '0' to '9', the binary classification unit 2 110-2 may determine in advance whether or not the input wrinkle image is grade 1 or less. In the trained model, binary classifier 2 110-2 is trained with a data set of wrinkled images labeled with a grade '1' or less, and images labeled with a grade '2' or more.

Accordingly, the binary classifier 2 110-2 is driven as '1' is input from the binary classifier 1 110-1, and outputs a '0' when the input wrinkle image is less than or equal to grade 1, and the wrinkle image. If the '2' grade or more output '1' to the binary classification unit (110-3).

The binary classifier 3 110-3 is a model trained in advance to determine whether an input wrinkle image is a grade 2 or less, and the binary classifier 3 110-3 is a wrinkle labeled with a grade '2' or less. Trained with images and a data set of images labeled with a rating of '3' or higher.

Therefore, the binary classifier 3 (110-3) is driven as '1' is input from the binary classifier 2 (110-2), and outputs '0' when the input wrinkle image is less than or equal to the grade '2'. If the wrinkle image is a grade '3' or more, '1' is output to the binary classification unit 110-4.

Finally, the N-th first binary classifier 110-N is driven by the N-th first binary classifier 110- (N-1) so that the wrinkle image is N out of N + 1 grades. If it is less than the first grade, a value indicating that the wrinkle image is an N-th grade is output. If the wrinkle image is more than the N + 1th grade, a value indicating that the wrinkle image is a grade N + 1 is output.

For example, if the number of wrinkle image grades is 10 and 10 grades of '0' to '9', the binary classification unit N (110-N) determines whether the input wrinkle image is a grade '8' or less. The pre-trained model, the binary classifier N 110-N, is trained with a data set of wrinkled images labeled below grade '8' and images labeled above grade '9'.

Accordingly, the binary classifier N 110 -N is driven as '1' is input from the binary classifier N-1 110-(N-1), so that when the input wrinkle image is less than or equal to the grade '8', 0 ', and outputs' 1' to the binary classification unit 110-3 when the wrinkle image is greater than or equal to the grade '9'.

Accordingly, the evaluation grade determiner 120 may determine and output the grade of the input image according to values output from the binary classifiers 110-1, 110-2,..., 110 -N. For example, the grade of the input image is determined and output according to the classification reference grade mapped to the binary classification unit which outputs a value of '0' among the binary classification units 110-1, 110-2,..., 110 -N. do. For example, when the number of wrinkle image grades is 10 and 10 grades of '0' to '9' are outputted, '0' is output from the binary classification unit 2 (110-2), the evaluation grade determination unit 120 ) May determine a grade of the input wrinkle image as '1'. However, when '1' is output from the last binary classification unit 110 -N, the evaluation grade determiner 120 may determine and output the grade of the input wrinkle image as '9'.

However, '0' may be output from two or more of the binary classification units 110-1, 110-2,..., 110 -N. In this case, the evaluation grade determiner 120 may obtain probability values for the values output from the binary classifiers that output '0' and determine the wrinkle image evaluation value based on the corresponding probability values. For example, a value of '0' is output from the binary classification units corresponding to the grade '5' and the grade '6', the probability value corresponding to the grade '5' is 0.6, and the probability value corresponding to the grade '6' is 0.8. In this case, the evaluation grade determiner 120 may determine the wrinkle image evaluation value as a grade '6' having a higher probability. As another example, a value of '0' is outputted from binary classification units corresponding to grade '5', grade '6' and grade '7', and the probability value corresponding to grade '5' is 0.6, and the grade '6' If the corresponding probability value is 0.9 and the probability value corresponding to the grade '6' is 0.7, the evaluation grade determiner 120 may determine the wrinkle image evaluation value as the grade '6' having a higher probability, or use the probability value as a weight. The wrinkle image evaluation may be determined as the mean of the sum of the respective grades multiplied.

4 is a detailed block diagram of a wrinkle evaluation engine according to a second embodiment of the present invention.

Referring to FIG. 4, N second binary classifiers 130-1 (N is a natural number) trained on a deep learning basis to classify an input wrinkle image into one of N + 1 grades according to visual judgment criteria. 130-2, ..., 130-N) and the wrinkle image evaluation value according to the values output from the N second binary classification units 130-1, 130-2, ..., 130-N. The second evaluation grade output unit 140 may be included.

Here, the N + 1 grades may have a first grade of grade '0' and the N + 1th grade of grade 'N'. For example, when the number of wrinkle image grades is 10, it may be made of 10 grades of '0' to '9', and the number of binary classification units 130-1, 130-2, ..., 130-N is It can be nine. However, this is only one embodiment, and the present invention is not limited thereto. That is, the N + 1 grades may be set to the first grade as the grade '1' and the N + 1th grade as the grade 'N + 1'.

Each of the second binary classifiers 130-1, 130-2,..., 130 -N is mapped to a rank of one of N + 1 grades so that an input wrinkle image is mapped to itself. It determines whether or not it is above the grade of the sequence number and outputs the result.

That is, the first second binary classification unit 130-1 outputs a value indicating that the wrinkle image is the N + 1 th grade when the wrinkle image is the N + 1 th rank or higher among N + 1 grades, and the wrinkle image is displayed. If it is less than the N-th grade, the second second binary classification unit 130-2 is driven.

For example, when the number of wrinkle image grades is 10 and 10 grades of '0' to '9', the binary classification unit 1 130-1 determines whether an input wrinkle image is grade '9' or more. The pre-trained model, binary classifier 1 130-1, is trained with a data set of wrinkled images labeled grade '9' and images labeled below grade '8'.

In addition, the binary classification units 130-1, 130-2,..., 130 -N may be output as one of values of '0' and '1' as evaluation values evaluated for the wrinkle image. Accordingly, the binary classifier 1 130-1 outputs '0' when the input wrinkle image is greater than or equal to '9', and the second binary classifier 130-2 when the wrinkle image is less than or equal to '1'. Print '1'.

And, the M-th (M is an integer of 2≤M≤N-1) second binary classification unit (130-2, 130-3, ... 130- (N-1)) is M-1 second binary Driven by the classification unit 130-1, 130-2, ... 130- (N-2)), and the wrinkle image is equal to or greater than the N + 1- (M-1) th of N + 1 grades. Outputs a value indicating that the wrinkle image is the N + 1- (M-1) th rank, and when the wrinkle image is the N + 1-M th rank or less, the M + 1 th binary classification unit 130-3, 130-4 , ... 130-N).

For example, when the number of wrinkle image grades is 10 and 10 grades of '0' to '9', the binary classification unit 2 (130-2) determines whether the input wrinkle image is a grade '8' or more. The pre-trained model, binary classifier 2 (130-2) is trained with a data set of wrinkled images labeled with a rating '8' or higher and images labeled with a rating '7' or lower.

Therefore, the binary classifier 2 130-2 is driven as the '1' is input from the binary classifier 1 130-1, and outputs '0' when the input wrinkle image is greater than or equal to the grade '8'. If the wrinkle image is a grade '7' or less, '1' is output to the binary classification unit 130-3.

The binary classifier 3 130-3 is a model trained in advance to determine whether an input wrinkle image is a grade '7' or higher, and the binary classifier 3 130-3 is labeled as a grade '7' or higher. Trained wrinkled images and a data set of images labeled with a rating of '6' or lower.

Accordingly, the binary classifier 3 130-3 is driven by inputting '1' from the binary classifier 2 130-2, and outputs '0' when the wrinkled image input is a grade '7' or more. If the wrinkled image is a grade '7' or less, '1' is output to the binary classification unit 130-4.

Finally, the N-th second binary classifier 130-N is driven by the N-th second binary classifier 130- (N-1) so that the wrinkle image is 2 out of N + 1 grades. If it is more than the first grade, a value indicating that the wrinkle image is the second grade is output. If the wrinkles image is less than the first grade, the value representing the wrinkle image is output.

For example, when the number of wrinkle image grades is 10 and 10 grades of '0' to '9', the binary classification unit N 130 -N determines whether or not the input wrinkle image is below grade ' 0'. The pre-trained model, binary classifier N 130 -N, is trained with a data set of wrinkled images labeled with a rating of '1' or higher and images labeled with a rating of '0' or lower.

Therefore, the binary classifier N 130 -N is driven as '1' is input from the binary classifier N-1 130-(N-1), so that when the input wrinkle image is above grade '1', 0 'is output and' 1 'is output when the wrinkle image is below grade' 0 '.

Accordingly, the evaluation grade determiner 140 may determine and output the grade of the input image according to values output from the binary classifiers 130-1, 130-2,..., 130 -N. For example, the grade of the input image is determined and output according to the classification reference grade mapped to the binary classification unit which outputs a value of '0' among the binary classification units 130-1, 130-2,..., 130 -N. do. For example, in the case where the number of wrinkle image ratings is 10 and 10 grades of '0' to '9' are output, '0' is output from the binary classification unit 2 130-2, the evaluation grade determination unit 140 may determine the grade of the input wrinkle image as '1'. However, when '1' is output from the last binary classification unit 130 -N, the evaluation grade determiner 140 may determine and output the grade of the input wrinkle image as '0'.

However, '0' may be output from two or more of the binary classification units 120-1, 120-2,..., 120 -N. In this case, the evaluation grade determiner 140 may obtain probability values for the values output from the binary classifiers that output '0' and determine the wrinkle image evaluation value based on the corresponding probability values.

5 is a detailed block diagram of a wrinkle evaluation engine according to a third embodiment of the present invention.

Referring to FIG. 5, the wrinkle evaluation engine unit configures the components shown in FIGS. 3 and 4 in parallel, respectively, and is output from the first evaluation rating output unit 120 and the second evaluation rating output unit 130, respectively. The method may further include an evaluation grade final determiner 160 that determines and outputs a final wrinkle evaluation grade by comparing probabilities corresponding to each of the wrinkle image evaluation values.

That is, each of the first evaluation grade output unit 120 and the second evaluation grade output unit 140 may output different grade evaluation values. In this case, the evaluation grade final determination unit 160 determines the final grade.

To this end, the first rating class output unit 120 and the second rating class output unit 130 respectively calculate probability values for the evaluation results from the first binary classifier and the second binary classifier which output a value of '0'. It may be extracted and output to the evaluation grade final determination unit 160.

Then, the evaluation grade final determiner 160 determines the final evaluation grade based on the probability values output from the first evaluation grade output unit 120 and the second evaluation grade output unit 140. According to an aspect, the evaluation grade final determiner 160 determines the final grade by the grade output by the evaluation grade output unit that outputs a high probability value.

In addition, although the first evaluation rating output unit 120, the second evaluation rating output unit 140, and the evaluation rating final decision unit 160 are separately configured in FIG. 5, the present invention is not limited thereto. That is, the first evaluation grade output unit 120, the second evaluation grade output unit 140, and the evaluation grade final determiner 160 may be integrally configured as one evaluation grade determiner, or the first evaluation grade output unit Only 120 and the second evaluation grade output unit 140 may be integrally configured.

Claims (10)

A grading model trained on a deep learning basis to output a result of evaluating an input wrinkle image within a range of N + 1 grades (N is a natural number) according to visual judgment criteria;
Deep learning-based wrinkle evaluation apparatus including an evaluation grade output unit for determining and outputting the wrinkle image evaluation value according to the result value output from the grade evaluation model.
The method of claim 1 wherein the rating model
The rating model is a regressor that outputs one of a continuous range of discrete real values, and N binary classifiers output one of '0' and '1' as a result. Deep learning based wrinkle evaluation device comprising a.
The method of claim 1 wherein the rating model
A deep learning based wrinkle evaluation apparatus in which N binary classifiers, which are output as one of '0' and '1', are cascaded.
4. The rating model of claim 3, wherein
If the wrinkle image is less than the first grade among the N + 1 grades, a value indicating that the wrinkle image is the first grade is output, and if the wrinkle image is more than the second grade, the first first binary driving the second first binary classifier is performed. Classification Division,
Driven by the M-1 th first binary classification unit, if the wrinkle image is less than or equal to the Mth rank among the N + 1 grades, a value indicating that the wrinkle image is the Mth rank is output, and the wrinkle image is the M + 1st rank. M-th (M is an integer of 2≤M≤N-1) for driving the M + 1st first binary classification unit, if any, and the first binary classification unit;
It is driven by the N-th first binary classification unit, and outputs a value indicating that the wrinkle image is the N-th grade if the wrinkle image is less than or equal to the N-th grade among the N + 1 grades, and the wrinkle image is the N + 1-th grade. The deep learning-based wrinkle evaluation apparatus including an N-th first binary classification unit outputting a value indicating that the wrinkle image is a grade N + 1 when the error is abnormal.
4. The rating model of claim 3, wherein
1 to output a value indicating that the wrinkle image is the N + 1th grade when the wrinkle image is above the N + 1 rank among the N + 1 grades, and to drive the second second binary classification unit when the wrinkle image is below the Nth grade. Second binary classification section,
If the wrinkle image is driven by the M-1 second binary classification unit, and the wrinkle image is equal to or greater than the N + 1- (M-1) th rank among the N + 1 grades, the wrinkle image is N + 1- (M-1). A second binary classifier for outputting a value indicating the first rank, and driving the M + 1 th binary sorter when the wrinkle image is less than or equal to the N + 1-M th rank, wherein M is an integer of 1≤M≤N. ,
Driven by the N-1 th binary classification unit, it outputs a value indicating that the wrinkle image is the second grade when the wrinkle image is the second or more of N + 1 grades, and the wrinkle image when the wrinkle image is the first grade or less Deep learning based wrinkle evaluation apparatus including an N-th second binary classification unit for outputting a value indicating that the first grade.
The method of claim 4 wherein the rating model
1 to output a value indicating that the wrinkle image is the N + 1th grade when the wrinkle image is above the N + 1 rank among the N + 1 grades, and to drive the second second binary classification unit when the wrinkle image is below the Nth grade. Second binary classification section,
If the wrinkle image is driven by the M-1 second binary classification unit, and the wrinkle image is equal to or greater than the N + 1- (M-1) th rank among the N + 1 grades, the wrinkle image is N + 1- (M-1). A second binary classifier for outputting a value indicating the first rank, and driving the M + 1 th binary sorter when the wrinkle image is less than or equal to the N + 1-M th rank, wherein M is an integer of 1≤M≤N. ,
Driven by the N-1 th binary classification unit, it outputs a value indicating that the wrinkle image is the second grade when the wrinkle image is the second or more of N + 1 grades, and the wrinkle image when the wrinkle image is the first grade or less Deep learning-based wrinkle evaluation apparatus further comprises an N-th second binary classification unit for outputting a value indicating that the first grade.
The method of claim 6, wherein the evaluation rating output unit
A first evaluation grade output unit configured to output a wrinkle image evaluation value according to values output from the N first binary classification units;
And a second evaluation grade output unit configured to output a wrinkle image evaluation value according to values output from the N second binary classification units.
The method of claim 7, wherein
A deep learning based wrinkle further comprising an evaluation grade final decision unit configured to determine and output a final wrinkle evaluation grade based on a probability value corresponding to each of the wrinkle image evaluation values respectively output from the first and second evaluation grade output units. Evaluation device.
The method according to any one of claims 1 to 8,
A cropping unit for extracting a region of interest in which wrinkle patterns are distributed from an input wrinkle image;
Deep learning-based wrinkle evaluation apparatus further comprises a scaling unit for scaling the region of interest image to a predetermined size.
The method according to any one of claims 1 to 8,
A deep learning-based wrinkle evaluation apparatus further comprising a normalization unit converting an input wrinkle image into grayscale and normalizing and outputting an average / standard deviation of luminance value distribution of the input image.

Images