JP2020131026A - Skin analysis method, skin analysis system, and skin analysis program - Google Patents

Abstract

To provide a novel, noninvasive, and simple method of analyzing the physical properties of the skin, which puts a less burden on both a subject and an analyzing person.SOLUTION: The skin analysis method for analyzing the skin of a subject on the basis of the movement of the skin caused by change in facial expression calculates, on the basis of the correlation between the physical amount of a skin change caused by the change in facial expression and the viscoelasticity of the skin, the viscoelasticity of the skin of the subject from a measured value of the physical amount related to the subject.SELECTED DRAWING: Figure 3

Description

The present invention relates to a skin analysis method, a skin analysis system and a skin analysis program.

In recent years, research has been conducted on techniques for analyzing human skin using moving images. For example, in Patent Document 1, a plurality of tracking points are arranged in advance in an analysis area such as the outer corner of the eyes of a subject to shoot a moving image, and the amount of change in the tracking points due to a change in facial expression is tracked to determine the compression rate of the skin. The technology to analyze the skin condition of the subject by acquiring it is open to the public.

Patent Document 2 discloses a technique relating to a process of evaluating clinical signs such as wrinkles and fine wrinkles from a sequence of images of a person's face taken while producing a sound.

Further, Patent Document 3 discloses a technique for extracting a factor having a high correlation with the appearance impression of a face using dynamic information and a method for distinguishing the appearance impression of a face based on the factor. There is. Patent Document 3 discloses that the sites for determining the age impression are the cheeks and the periphery of the eyes.

Japanese Unexamined Patent Publication No. 2014-193197 Japanese Unexamined Patent Publication No. 2017-502732 Japanese Unexamined Patent Publication No. 2016-194901

As mentioned above, techniques for non-invasively evaluating skin by analyzing videos including human faces have been widely studied. However, the conventional technique merely evaluates the signs appearing on the skin surface such as wrinkles, and a specific method for non-invasively analyzing the physical properties such as the viscoelasticity and collagen structure of the skin is disclosed. Not. Therefore, in order to evaluate the physical properties of the skin, it is necessary to collect the tissue of the skin, apply force to the skin in contact with the skin, and measure the mechanical properties. Therefore, an object of the present invention is to provide a novel method for non-invasively analyzing the physical properties of skin.

In order to solve the above-mentioned problems, the skin analysis method according to the present invention is based on the correlation between the physical quantity of the skin change caused by the change in facial expression and the viscoelasticity of the skin, and is based on the measured value of the physical quantity of the subject. It is characterized by calculating the viscoelasticity of the skin.

In this way, the viscoelasticity of the skin is calculated by calculating the viscoelasticity of the skin from the measured value of the physical amount of the skin change based on the correlation between the physical amount of the skin change caused by the facial expression change and the viscoelasticity of the skin. Skin analysis, including calculations, can be performed non-invasively.

In a preferred embodiment of the present invention, the viscoelasticity is characterized by including at least one of dermal viscoelasticity and subcutaneous viscoelasticity.

In a preferred embodiment of the present invention, the viscoelasticity of the subject's skin is determined from the measured values of the physical quantity for the subject by using a viscoelasticity calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the viscoelasticity. It is characterized by calculating.
In this way, by calculating the viscoelasticity using the viscoelasticity calculation model created using the set of physical quantity and viscoelasticity measurement data, it is possible to calculate the viscoelasticity more accurately based on actual examples. it can.

A preferred embodiment of the present invention is characterized in that the collagen structure of the skin of a subject is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin.
In this way, skin analysis including estimation of the subcutaneous collagen structure by estimating the subcutaneous collagen structure based on the viscoelasticity of the skin calculated from the measured value of the physical quantity of the skin change caused by the facial expression change. Can be done non-invasively.

In a preferred embodiment of the present invention, the subject's skin is calculated from the viscoelasticity calculated for the subject using a collagen structure estimation model created by inputting a plurality of sets of measured values of the viscoelasticity and the collagen structure of the skin. It is characterized by estimating the collagen structure.

A preferred embodiment of the present invention is characterized in that one or a plurality of evaluation values of skin are calculated based on the viscoelasticity.
In this way, by calculating the evaluation value of the skin based on the viscoelasticity of the skin, it is possible to show a result that is easy for the subject to understand.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the calculation result of viscoelasticity is output.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the measured value of the physical quantity is output together with information based on the calculation result of viscoelasticity.
By outputting the calculated viscoelasticity and the information based on the measured physical quantity together in this way, it becomes easier for the user to understand the correspondence between the measured quantity and the analysis result.

The present invention is characterized in that the collagen structure of the skin of a subject is estimated from the measured value of the physical quantity of the subject based on the correlation between the physical quantity of the skin change caused by the change of facial expression and the collagen structure of the skin. ..
In this way, the collagen structure of the skin is estimated by estimating the collagen structure of the skin from the measured value of the physical amount of the skin change based on the correlation between the physical amount of the skin change caused by the facial expression change and the collagen structure of the skin. Skin analysis, including estimation, can be performed non-invasively.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the estimation result of the collagen structure is output.

A preferred embodiment of the present invention is characterized in that an analysis result including information based on the measurement value of the physical quantity is output together with information based on the estimation result of the collagen structure.

In a preferred embodiment of the present invention, the physical quantity is measured based on a moving image including the process of changing the facial expression.
In this way, by measuring the physical quantity using the moving image, the viscoelasticity of the skin can be calculated more easily without contacting the skin of the subject. As a result, the burden on both the subject and the person performing the analysis can be reduced.

A preferred embodiment of the present invention is characterized in that a guide for aligning the position of a facial component on the face of the subject is displayed on the screen for capturing the moving image.
In this way, on the screen for shooting a moving image, a guide for aligning the position of the face part is displayed, and by shooting according to this guide, the subject is captured at an appropriate position from an appropriate distance. It can be performed.

In a preferred embodiment of the present invention, the moving image is captured from a direction rotated in a range of 20 to 30 degrees with the front surface as 0 degrees around an axis in the vertical direction passing through the face of the subject.
In this way, by using the image of the face taken from an angle rotated in the range of 20 to 30 degrees from the front, the impression of the face such as the cheeks and the outer corners of the eyes can be left and right compared to the case of using the image taken from the front. It is possible to acquire an image that includes a wide range of skin in a part that is easy to make, and to perform a more accurate skin analysis on the skin in a part that easily affects the impression of the face.

In a preferred embodiment of the present invention, as the moving image, a moving image taken from a direction in which the front is rotated clockwise in the range of 20 to 30 degrees with the front as 0 degrees around the vertical axis passing through the subject's face, and the subject's moving image. It is characterized by using both a moving image taken from a direction in which the front is rotated counterclockwise in the range of 20 to 30 degrees with the front as 0 degrees around an axis in the vertical direction passing through the face.
In this way, by using moving images taken from different directions, it is possible to analyze the physical properties of the skin from the physical quantity that comprehensively considers the skin changes obtained from the left and right facial expression changes.

In a preferred embodiment of the invention, the moving image
The initial position is the state in which the subject is captured from the front, and the rotation angle of the imaging device from the initial position is acquired.
When the rotation angle is within a predetermined range, shooting of the moving image is permitted, and
It is characterized in that it is photographed based on the above permission.

In a preferred embodiment of the present invention, the physical quantity is measured based on at least one of the magnitude, direction of velocity, magnitude of acceleration, and direction of acceleration of the feature points contained in the face. It is characterized by that.

In a preferred embodiment of the present invention, the physical quantity is selected from skin followability, elasticity and deformability.

A preferred embodiment of the present invention is characterized in that the water content of the epidermis of a subject is calculated from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the water content of the epidermis.
In this way, the calculation of the water content of the epidermis is included by calculating the water content of the epidermis from the measured value of the physical quantity based on the correlation between the physical quantity of the skin change caused by the facial expression change and the water content of the epidermis. Skin analysis can be performed non-invasively.

In a preferred embodiment of the present invention, the water content of the subject is calculated from the measured values of the physical quantity for the subject by using the water content calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the water content. It is characterized by doing.

The present invention is a skin analysis system that analyzes the skin of a subject based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the viscoelasticity of the skin,
It is characterized by comprising a viscoelasticity calculating means for calculating the viscoelasticity of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the viscoelasticity of the skin.

In a preferred embodiment of the present invention, the viscoelasticity calculating means calculates at least one of dermis viscoelasticity and subcutaneous viscoelasticity.

In a preferred embodiment of the present invention, the storage means stores a viscoelasticity calculation model created by inputting a plurality of sets of physical quantities and measured values of viscoelasticity.
The viscoelasticity calculation means is characterized in that the viscoelasticity is calculated from the physical quantity measured for a subject by using the viscoelasticity calculation model.

In a preferred embodiment of the present invention, the storage means stores the correlation between the viscoelasticity and the collagen structure of the skin.
It is characterized by further comprising a collagen structure estimating means for estimating the collagen structure of the skin of the subject from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin.

In a preferred embodiment of the present invention, the storage means stores a collagen structure estimation model created by inputting a plurality of sets of measured values of viscoelasticity and collagen structure.
The collagen structure estimation means is characterized in that the collagen structure of the subject's skin is estimated from the viscoelasticity calculated for the subject using the collagen structure estimation model.

A preferred embodiment of the present invention is further provided with an evaluation means for calculating one or more evaluation values of the skin based on the viscoelasticity of the skin of the subject.

A preferred embodiment of the present invention is further provided with an output means for outputting an analysis result including information based on the calculation result of viscoelasticity.

In a preferred embodiment of the present invention, the output means outputs an analysis result including information based on the measured value of the physical quantity together with information based on the calculation result of the viscoelasticity.

The present invention is a skin analysis system that analyzes the skin of a subject based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the collagen structure of the skin,
It is characterized by comprising a collagen structure estimating means for estimating the collagen structure of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the collagen structure of the skin.

A preferred embodiment of the present invention is further provided with an output means for outputting information based on the estimation result of the collagen structure.

The output means is characterized in that it outputs an analysis result including information based on the measurement value of the physical quantity together with information based on the estimation result of the collagen structure.

In a preferred embodiment of the present invention, a moving image acquisition means for acquiring a moving image including the process of changing the facial expression is further provided.
The physical quantity measuring means is characterized in that the physical quantity is measured based on the moving image.

A preferred embodiment of the present invention is further provided with means for displaying a guide for aligning the positions of facial parts on the face of the subject on the screen for capturing the moving image.

In a preferred embodiment of the present invention, the moving image acquisition means captures a moving image of a subject's face from a direction rotated in a range of 20 to 30 degrees with the front surface as 0 degrees around a vertical axis passing through the subject's face. It is characterized by acquiring.

In a preferred embodiment of the present invention, as the moving image, a moving image taken from a direction in which the front is rotated clockwise in the range of 20 to 30 degrees with the front as 0 degrees around the vertical axis passing through the subject's face, and the subject's moving image. It is characterized by using both a moving image taken from a direction in which the front is rotated counterclockwise in the range of 20 to 30 degrees with the front as 0 degrees around an axis in the vertical direction passing through the face.

In a preferred embodiment of the present invention, the moving image acquisition means acquires the moving image taken by a terminal device for taking a picture of a subject under predetermined conditions.
The terminal device is
A means for acquiring the rotation angle of the terminal device from the initial position, with the state in which the subject is captured from the front as the initial position,
A means for permitting the shooting of the moving image when the rotation angle is within the range of 20 to 30 degrees, and
It is characterized by comprising means for shooting the moving image based on the permission.

In a preferred embodiment of the present invention, the physical quantity measuring means is based on at least one of the magnitude, direction of velocity, magnitude of acceleration, and direction of acceleration of movement of feature points contained in the face. It is characterized by measuring a physical quantity.

In a preferred embodiment of the present invention, the physical quantity measuring means measures, as the physical quantity, any of skin followability, elasticity, and deformability.

In a preferred embodiment of the present invention, the storage means stores the correlation between the physical quantity and the water content of the epidermis.
It is further provided with a water content calculating means for calculating the water content of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the water content of the epidermis.

In a preferred embodiment of the present invention, the storage means stores a water content calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the water content.
The water content calculation means is characterized in that the water content is calculated from the physical quantity measured for a subject by using the water content calculation model.

The present invention is a skin analysis program for analyzing the skin of a subject based on the movement of the skin when the facial expression changes, and uses a computer.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the viscoelasticity of the skin,
It is characterized in that it functions as a viscoelasticity calculating means for calculating the viscoelasticity of the skin of a subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the viscoelasticity of the skin.

The present invention is a skin analysis program for analyzing the skin of a subject based on the movement of the skin when the facial expression changes, and uses a computer.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the collagen structure of the skin,
It is characterized in that it functions as a collagen structure estimation means for estimating the collagen structure of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the collagen structure of the skin.

It is possible to analyze the physical properties of the skin non-invasively based on the changes in the skin when the facial expressions change.

It is a schematic diagram of the facial expression reproduced by the subject in one embodiment of the present invention. It is a functional block diagram of the skin analysis system which concerns on one Embodiment of this invention. It is a flowchart which shows the skin analysis method which concerns on one Embodiment of this invention. It is a flowchart which shows the process of the moving image acquisition process in one Embodiment of this invention. It is a figure which shows the imaging direction of the subject in one Embodiment of this invention. It is a figure which shows the face part coordinates in one Embodiment of this invention. It is a figure which shows an example of the reference distance in one Embodiment of this invention. It is a flowchart which shows the process of the analysis target extraction process in one Embodiment of this invention. It is a flowchart which shows the process of the movement amount measurement process in one Embodiment of this invention. It is a figure which shows the fractionation pattern of the analysis area in one Embodiment of this invention. It is a flowchart which shows the process flow of the viscoelasticity calculation process in one Embodiment of this invention. It is a flowchart which shows the process flow of the water content calculation process in one Embodiment of this invention. It is a flowchart which shows the process flow of the calculation of the subcutaneous collagen structure based on the subcutaneous viscoelasticity in one Embodiment of this invention. It is a figure which shows the appearance of the terminal apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the flow of shooting a moving image in one Embodiment of this invention. It is a figure which shows the shooting angle of the moving image in one Embodiment of this invention. It is a figure which shows an example of the shooting preparation screen of a moving image in one Embodiment of this invention. It is a figure which shows an example of the shooting screen of the moving image in one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, a moving image showing the process of facial expression change is acquired, a frame at which the facial expression change starts and a frame at which the facial expression change ends are specified, and analysis is performed on a frame shot between the two. Do.

In the present invention, "calculating viscoelasticity" refers to calculating a value for quantitatively evaluating viscosity and elasticity. In the present embodiment, the viscoelasticity of the skin refers to the viscoelasticity of the dermis and the viscoelasticity of the skin. Since such viscoelasticity of the skin causes changes in appearance such as wrinkles and sagging, it is an index for confirming the aging phenomenon of the skin with aging.

The collagen structure of the skin refers to the collagen structure of the subcutaneous or dermis, including the fibrotic level of the fibrotic structure surrounding the subcutaneous adipocytes and the degree of binding of the collagen fibers of the dermis.

The skin is formed of three layers: epidermis, dermis and subcutaneous tissue. Most of the subcutaneous tissue that supports the epidermis and dermis is subcutaneous fat composed of fat leaflets in which adipocytes form agglomerates, and has a heat retaining effect and a buffering action against external force. The fat leaflets are surrounded by connective tissues such as collagen fibers and elastin fibers in a mesh pattern to form a fibrous structure that encloses adipocytes.

As a result of diligent studies by the present inventors, it has been clarified that collagen fibers surrounding subcutaneous adipocytes become fibrotic with aging. In the present invention, an index representing the fibrosis level of such a subcutaneous fiber structure is estimated as "subcutaneous collagen structure".

It is also known that the connective tissue of the dermis gradually loses its flexibility and elasticity and becomes harder with aging. It is believed that this is due to the fact that collagen fibers, which are the main components of connective tissue, crosslink and bind with age. Therefore, in the present invention, an index indicating the degree of binding of collagen fibers in the dermis is estimated as "dermis collagen structure". In addition to this, any index for evaluating the structure of collagen may be used.

In the present embodiment, a value indicating at least one of the magnitude of the movement of the feature points included in the face, the direction of the velocity, the magnitude of the acceleration, and the direction of the acceleration in the process of changing the facial expression (hereinafter, The amount of movement) is measured. The amount of movement may be obtained as a value in each frame, or may be obtained as an average value through one facial expression change. Alternatively, a region such as a cheek or a chin may be set, and an integrated value, an average value, or the like of values in the region may be obtained. Further, in the present embodiment, the physical quantity of the skin change caused by the facial expression change refers to a value measured based on the movement amount.

In the following, the facial component coordinates are coordinates indicating the positions of the components on the face for tracking the movement of the face and evaluating the change in facial expression. The facial component refers to any feature such as eyes, nose, facial contours, etc. For example, points at the upper and lower ends of the lip contour, points at the left and right ends, and the like can be detected and used as facial component coordinates.

The analysis target is a set of images to be analyzed for skin. In the present embodiment, the skin analysis is performed by specifying the frame at which the facial expression change starts and the frame at which the facial expression change ends, and analyzing the movement using the image during the facial expression change included between them.

In the present embodiment, a moving image showing the process of the vertical facial expression change in which the skin of the cheek portion is expanded and contracted in the vertical direction and the process of the lateral facial expression change in which the skin of the cheek portion is expanded and contracted in the horizontal direction are shown. A moving image is acquired, and the physical quantity of the skin change caused by each facial expression change is measured based on the moving image to perform skin analysis. In this way, by using a moving image showing the process of facial expression change that stretches and contracts the skin in a specific direction, it becomes possible to analyze the movement characteristics of the skin when the skin is stretched and contracted in a specific direction. In particular, by analyzing both the vertical facial expression change and the horizontal facial expression change, a wider range of skin motor characteristics can be investigated.

FIG. 1 is a diagram showing a facial expression reproduced by a subject in the present embodiment. In the present embodiment, as shown here, the subject has an expressionless expression (FIG. 1 (a)), a facial expression that opens the mouth vertically (hereinafter, “a” facial expression, FIG. 1 (b)), and a lateral mouth. It takes four facial expressions: a facial expression that opens to the front (hereinafter, "i" facial expression, FIG. 1 (c)) and a facial expression that purses the mouth (hereinafter, "u" facial expression, FIG. 1 (d)).

In the present embodiment, the subject reproduces facial expressions in the order of no expression, "a" expression, "u" expression, "a" expression, "u" expression, and no expression as vertical facial expression changes. .. That is, the vertical facial expression change from no expression to the "a" facial expression, the vertical facial expression change from the "a" facial expression to the "u" facial expression, and the "u" facial expression to the "a" facial expression. Skin analysis is performed by acquiring moving images including four types of vertical facial expression changes: a vertical facial expression change and a vertical facial expression change from a "u" facial expression to no facial expression. Of the four types of facial expressions for skin analysis, only two types are used: a vertical facial expression change from expressionless to "a" and a vertical facial expression change from "u" to expressionless. You may.

In addition, as the lateral facial expression change, the subject reproduces the facial expression in the order of no expression, "i" expression, "u" expression, "i" expression, "u" expression, and no expression. That is, the lateral expression change from expressionless to "i" expression, the lateral expression change from "i" expression to "u" expression, and the "u" expression to "i" expression. Skin analysis is performed by acquiring moving images including four types of lateral facial expression changes, that is, a lateral facial expression change and a lateral facial expression change from a "u" facial expression to no facial expression. Of the four types of facial expressions for skin analysis, only two types are used: a lateral facial expression change from expressionless to "i" and a lateral expression change from "u" to expressionless. You may. Here, in the facial expression change from the expressionless expression to the facial expression of "U", since the skin of the cheek portion expands and contracts both vertically and horizontally, it is treated as both a vertical facial expression change and a horizontal facial expression change.

As described above, in the present embodiment, a moving image including a plurality of types of facial expression changes that expand and contract the skin of the cheek portion in the same direction is acquired, and various frames at which various facial expression changes start and end are specified. The image from the frame at which the facial expression change starts to the frame at which the facial expression changes ends is analyzed. In this way, by identifying and analyzing each of the multiple types of facial expression changes that stretch and contract the skin in the same direction, it is possible to analyze in more detail the movement characteristics of the skin that accompany the stretching of the skin in a specific direction. Can be done.

In the present embodiment, analysis targets are extracted for each of the four types of vertical facial expression changes and four types of horizontal facial expression changes, and the amount of movement is measured for each analysis target. As the physical quantity, any value derived from the movement amount can be used, such as the average value, the total value, and the value calculated by weighting each movement amount measured for each analysis target.

FIG. 2 is a functional block diagram of the skin analysis system according to the present embodiment. As shown here, the skin analysis system according to the present embodiment is configured so that the skin analysis device 10 and the terminal device 20 can communicate with each other. The skin analysis device 10 includes a computing device such as a CPU (Central Processing Unit), a main storage device such as a RAM (Random Access Memory), an auxiliary storage such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory. A general computer device such as a server device provided with various input / output devices including a device and a means for communicating with the terminal device 20 can be used. More specifically, programs for operating a computer are stored in the auxiliary storage device in advance or by copying from a recording medium as various means described later, and these programs are expanded on the main storage device. The computer device can be used as the skin analysis device 10 in the present embodiment by performing calculations by the arithmetic unit and controlling the input / output means and the like.

The terminal device 20 includes a photographing means 201 and a display unit 202. The terminal device 20 includes a general computer device provided with an arithmetic unit, a main storage device, an auxiliary storage device, a communication means with the skin analysis device 10, a photographing means such as a digital camera, various input / output devices, and the like. It can be used. For example, a tablet terminal or the like having a camera function may be used. Specifically, for example, by installing dedicated application software for operating the computer device as each means described later, the computer device can be used as the terminal device 20 in the skin analysis system according to the present embodiment. ..

The skin analysis device 10 in the present embodiment identifies the timing of the facial expression change from the moving image acquisition means 1 for acquiring the moving image of the facial expression change process and the moving image acquired by the moving image acquisition means 1 and extracts the analysis target. Correlation data is stored with the analysis target extraction means 2 to be analyzed, the physical quantity measuring means 3 that measures the movement amount in the analysis target extracted by the analysis target extraction means 2 and measures the physical amount of skin change based on the movement amount. The evaluation value of the skin of the subject is calculated based on the memory means 4, the analysis unit 5 that calculates the viscoelasticity of the skin, the water content of the epidermis, and the estimation of the subcutaneous collagen structure, and the analysis unit 5. The evaluation means 6 is provided.

In the present embodiment, the physical quantity measuring means 3 includes the feature point extracting means 31 for extracting the feature points of the face from the moving image, the magnitude of the movement speed of the feature points, the direction of the speed, the magnitude of the acceleration, the direction of the acceleration, and the like. A movement amount measuring means 32 for specifying a movement vector indicating the above and the like and measuring the movement amount from the movement vector, and a movement amount analysis means 33 for measuring a physical quantity based on the movement amount are provided.

The storage means 4 stores the correlation between the physical quantity and the viscoelasticity. In the present embodiment, the correlation between the physical quantity and the viscoelasticity stored by the storage means 4 includes the correlation between the physical quantity and the viscoelasticity under the skin, and the correlation between the physical quantity and the viscoelasticity of the dermis. In the present embodiment, the memory means 4 also has a correlation between the physical amount and the water content of the epidermis, a correlation between the viscoelasticity of the dermis and the dermis collagen structure, a correlation between the subcutaneous viscoelasticity and the subcutaneous collagen structure, and the like. Data, information on the moving image acquired by the moving image acquisition means 1, information on the analysis target extracted by the analysis target extracting means 2, analysis results, and the like are stored.

Here, as various correlation data in the present embodiment, for example, a function representing the mutual relationship of various values, a model trained by giving various data as a set, and the like are stored as a calculation model. Can be kept. In addition to this, information representing the correlation may be stored by any method and used for calculation.

In the present embodiment, the analysis unit 5 is based on the viscoelasticity calculating means 51 that calculates the viscoelasticity from the measured value of the physical quantity based on the correlation between the physical quantity and the viscoelasticity, and the correlation between the viscoelasticity and the subcutaneous collagen structure. The collagen structure estimation means 52 that estimates the skin collagen structure from the calculated viscoelasticity, and the water content calculation means that calculates the water content of the epidermis from the measured value of the physical quantity based on the correlation between the physical quantity and the water content of the epidermis. 53 and. The viscoelasticity calculating means 51 includes a subcutaneous viscoelasticity calculating means 511 for calculating the subcutaneous viscoelasticity and a dermis viscoelasticity calculating means 512 for calculating the viscoelasticity of the dermis. Further, the collagen structure estimating means 52 includes a subcutaneous collagen structure estimating means 521 for estimating the subcutaneous collagen structure and a dermis collagen structure estimating means 522 for estimating the dermis collagen structure.

In the present embodiment, the evaluation means 6 includes the dermis viscoelasticity evaluation means 61 for calculating the evaluation value regarding the viscous elasticity of the dermis calculated by the dermis viscoelasticity calculation means 512 and the subcutaneous viscoelasticity calculated by the subcutaneous viscoelasticity calculation means 511. The subcutaneous viscoelasticity evaluation means 62 for calculating the evaluation value for the dermal collagen structure, the dermis collagen structure evaluation means 63 for calculating the evaluation value for the dermis collagen structure estimated by the dermis collagen structure estimation means 522, and the subcutaneous collagen structure estimation means 521 estimated for The subcutaneous collagen structure evaluation means 64 for calculating the evaluation value regarding the collagen structure and the water content evaluation means 65 for calculating the evaluation value regarding the water content of the dermis calculated by the water content calculation means 53 are provided.

FIG. 3 is a flowchart showing the skin analysis method in the present embodiment. In the present embodiment, steps S11 and S12 in FIG. 3 are performed for each of a moving image including a vertical facial expression change and a moving image including a horizontal facial expression change, and there are two types in steps S13 to S16. Each step is processed based on a plurality of movement amounts measured from the moving image of. In this way, the amount of movement is measured for a plurality of facial changes that stretch and contract the skin in different directions, and the viscoelasticity of the skin is calculated based on the physical quantities measured from the plurality of movement amounts, thereby analyzing the skin in more detail. It can be performed.

<Extraction of analysis target>
First, in the moving image acquisition step of step S11, the moving image is acquired and the analysis target is extracted. FIG. 4 shows a flowchart of processing in the moving image acquisition process. In the present embodiment, a moving image including a vertical facial expression change and a moving image including a horizontal facial expression change are acquired, the processing shown in FIG. 4 is executed for each of the facial expression changes included in each, and the analysis target is extracted. To do. That is, the process shown in FIG. 4 is executed for each of the two types of moving images. Here, all the steps shown in FIG. 4 move to the next step after the processing for one entire moving image is completed. That is, for example, the facial expression motion evaluation step in step S24 starts after the calculation of the distances between the face component coordinates for all the frames constituting the moving image in the distance calculation step in step S23 is completed.

In step S21, the moving image acquisition means 1 displays information prompting the shooting of the moving image on the display unit 202 of the terminal device 20, and the shooting means 201 shoots the moving image. The moving image acquisition means 1 acquires the information of the moving image taken by the shooting means 201 and records it in the storage means 4.

FIG. 5 is a diagram showing the imaging direction of the subject in the present embodiment. In the present embodiment, the moving image acquisition means 1 acquires information on a moving image showing the process of facial expression change taken from a direction rotated by 25 degrees with the front as 0 degrees around a vertical axis passing through the subject's face. To do. At this time, the photographing means 201 may acquire the rotation angle from the front and allow photography when the rotation angle is within a predetermined range. The direction of rotation may be clockwise or counterclockwise toward the subject, or both may be analyzed.

In this way, it is desirable to shoot from the direction rotated around the vertical axis passing through the subject's face. More specifically, the rotation angle at this time is more preferably in the range of 20 to 30 degrees with the front surface as 0 degrees. As a result, it is possible to acquire an image that includes the skin around the cheeks and eyes, which tends to affect the impression of the face, more accurately, so that the skin can be analyzed more accurately. ..

At this time, a guide for aligning the positions of facial parts such as eyes and contours may be displayed on the shooting screen. For example, a guide for aligning the face may be displayed first, and when the alignment of the face is completed, the guide may be displayed so that the shooting angle is 20 to 30 degrees based on the angle of the shooting means 201. ..

Further, in the present embodiment, the skin analysis device 10 executes a process of correcting the deviation of the position of the face in the acquired moving image. Specifically, it is possible to correct the movement of the face position due to the facial expression movement and the camera shake during shooting, for example, with the face component such as the nose as a reference. At this time, it is preferable that the reference face component is a portion where the relative position change with respect to the analysis region is small.

It should be noted that the processing for displaying the information and the guide for prompting the shooting of the moving image, the correction of the displacement of the face position, and the like may be executed by either the skin analysis device 10 or the terminal device 20. For example, the moving image shooting process is completed in the terminal device 20, and the moving image acquisition means 1 may function as a means for simply receiving the moving image taken and temporarily stored in the terminal device 20.

Next, in the face component coordinate detection step of step S22, the analysis target extraction means 2 detects a plurality of face component coordinates of the face included in the moving image acquired in step S21 for each frame. Here, at least one set of two points whose distance changes in conjunction with the change in facial expression included in the moving image is detected. In this way, by detecting as a set two points whose distance changes in conjunction with the facial expression change, the image in which the facial expression change starts and the image in which the facial expression change ends based on the change in the distance between the two points can be obtained. Can be identified accurately.

When the detection of the face component coordinates for the entire moving image is completed in the face component coordinate detection step of step S22, the analysis target extraction means 2 calculates the distance between the face component coordinates in step S23. The calculation of the distance is performed for each frame of the entire moving image, and the calculation result is stored in the storage means 4.

FIG. 6 is a diagram showing the coordinates of the face parts used in the present embodiment. FIG. 6 (a) shows the coordinates of the face parts used when extracting the analysis target in the vertical facial expression change, and FIG. 6 (b) shows the coordinates of the face parts used in extracting the analysis target in the horizontal facial expression change. Shown. As shown here, in the present embodiment, when extracting the analysis target in the vertical facial expression change, in addition to the two points between the upper and lower ends of the lips, a plurality of points on the left and right cheeks and a point on the chin, respectively. Detect the coordinates as face part coordinates. Then, the distance calculation means 13 calculates the distance between the upper and lower ends of the lip contour and the distance between each point on the left and right cheeks and the point on the chin, and the analysis target is based on the calculation result. Be extracted.

Further, when extracting the analysis target in the lateral facial expression change, two points at the upper and lower ends of the lip contour and two points at the left and right ends of the lip contour are detected as facial component coordinates, and the distance calculation means 13 sets each of the lips as a set. The distance between the two points at the upper and lower ends of the contour and the distance between the two points at the left and right ends of the lip contour are calculated. In this way, by setting the facial component coordinates to be used according to the series of facial expression changes to be analyzed, it is possible to accurately identify the start and end of the facial expression change with a small amount of information.

Here, the two points on the upper and lower ends of the lips and the two points on the cheeks and chin both increase the distance when the skin on the cheeks is stretched in the vertical direction, and the skin on the cheeks is shortened in the vertical direction. The distance becomes smaller when Thereby, the timing of various vertical facial expression changes can be specified. In addition, the distance between the two points at the left and right ends of the lip contour decreases when the skin on the cheek portion is stretched laterally, and the distance increases when the skin on the cheek portion is contracted laterally. Furthermore, by using the distance between the upper and lower ends of the lip contour, for example, when the facial expression changes in the lateral direction, the facial expression from the "U" expression to the expressionless expression where the distance change between the two points at the left and right ends of the lip contour is small. As for the change, the image at which the facial expression change starts and the image at which the facial expression change ends can be specified, and the timing of various lateral facial expression changes can be specified.

As described above, in the face component coordinate detection step, it is preferable to detect a plurality of pairs of two points whose distance changes in conjunction with the expansion and contraction of the skin on the cheek portion. As a result, for a moving image containing a plurality of types of facial expression changes, the frame at which the various facial expression changes start and the frame at which the facial expression changes end are specified by a series of processes without performing analysis target extraction processing for each type. be able to.

Any generally known method can be used to detect the coordinates of the face component. For example, application software developed using a development kit that is generally sold may be used. As a development kit, for example, Face Sensing Engine (registered trademark) (https://www.oki.com/jp/fse/function/#fp, last viewed date: December 19, 2018) is used. be able to.

Further, in the present embodiment, in the distance calculation step S23, a reference distance calculation step for calculating the reference distance for standardizing the distance between the face component coordinates is executed. Then, in the facial expression motion evaluation step S24 to the analysis target extraction step S26, which will be described later, the analysis target is extracted using the value obtained by standardizing the distance between the facial component coordinates by the reference distance. By doing so, it is possible to reduce the influence of individual differences in the face and the magnification at the time of shooting, and to extract the analysis target more accurately.

FIG. 7 is a diagram showing an example of a reference distance that can be used in the present embodiment. In the present embodiment, the distance between the left and right eye (distance of "A" in the illustrated example) is used as a reference distance to standardize the distance between the coordinates of the face parts. Specifically, for example, the distance between the face component coordinates can be standardized by calculating the distance of A and dividing all the distances between the face component coordinates by A.

In addition, any distance that can correct the influence of individual differences and magnification at the time of shooting, such as the length of the perpendicular line drawn from the line connecting the left and right eyes to the chin (distance of "B" in the illustrated example). Is preferably used as the reference distance. Further, a plurality of reference distances may be combined to standardize the distance between the coordinates of the face parts. As the reference distance, a distance related to a point showing an arbitrary feature on the face may be used, and there is no particular limitation.

In the facial expression motion evaluation step of step S24, the magnitude of the facial expression change is evaluated based on the change in the distance between the facial component coordinates calculated in step S23. Specifically, for example, the distance between the facial component coordinates when the subject reproduces the expressionlessness and the facial expressions other than the expressionlessness ("a" expression, "i" expression, "u" expression) are reproduced. The ratio between the distance between the facial component coordinates and the facial expression change can be calculated as the evaluation value of the facial expression change. By using the ratio in this way, it is possible to reduce the influence of individual differences such as the size of the face and the magnification at the time of shooting.

Next, in step S25, the evaluation value calculated in step S24 is compared with the reference value, and if it exceeds the reference value, the process proceeds to the analysis target extraction step of step S26. If the value does not reach the reference value, the process proceeds to step S27, and the moving image acquisition means 1 transmits information for prompting the shooting of the moving image to the terminal device 20 again. When step S27 is completed, the process returns to step S21, and the moving image is acquired again. For the video acquired again, in steps S22 to S25, each process of the face component coordinate detection step, the distance calculation step, and the facial expression motion evaluation step is performed.

In the present embodiment, since the acquired moving image includes a plurality of facial expression changes, in the facial expression motion evaluation process, all the facial expression changes included in the moving image are compared with the reference values, and all the facial expression changes. When the evaluation value of is higher than the reference value, the process proceeds to the analysis target extraction step.

In this way, by evaluating the magnitude of the facial expression change, and if it does not meet the predetermined criteria, re-shooting is performed and the moving image is acquired again, the magnitude of the facial expression change is sufficient for skin analysis. You can perform skin analysis on the video. This makes it possible to improve the accuracy of calculating the viscoelasticity of the skin.

In step S26, the analysis target extraction means 2 extracts the analysis target. FIG. 8 is a flowchart showing an example of processing in the analysis target extraction process. Here, in step S26, the processing shown in FIG. 8 is executed for each of the analysis targets in various facial expression changes included in the moving image, and the analysis target is extracted. In the present embodiment, a total of eight types of facial expression changes, four types of vertical facial expression changes of a moving image including a vertical facial expression change and four types of horizontal facial expression changes of a moving image including a horizontal facial expression change, are analyzed. Is extracted.

The process shown in FIG. 8 is a process performed for each of the distances between the coordinates of the two face parts detected as a set. In the present embodiment, in order to extract the analysis target by using a plurality of distances between the coordinates of the face parts, this processing is performed for each of the distances between the coordinates of the two face parts detected as a set, and the combination of the results is used. Extract the analysis target. Specifically, the analysis target is extracted using the distance between the coordinates of two face parts detected as a set, and the distance between the coordinates of the other two points for the facial expression change that could not be extracted at that time. The analysis target can be extracted by performing the process shown in FIG. 8 using the distance of.

First, in step S31, the analysis target extraction means 2 calculates the absolute value of the difference (change amount) of the distance between the face component coordinates between consecutive frames. Next, in step S32, a vertex (maximum point) in which the difference in distance between the face component coordinates is larger than the reference value is detected. That is, in each process of the plurality of facial expression changes included in the moving image, the point where the distance between the coordinates of the face parts changes most is detected.

In step S33, a frame in which the difference in distance between the face component coordinates is less than the reference value before and after the vertex detected in step S32 is specified. As a result, the frame in which the difference in the distance between the face component coordinates is less than the reference value immediately before the vertex detected in step S32 is specified as the frame in which the facial expression change has started, and similarly, the frame between the face component coordinates immediately after the vertex is specified. A frame in which the difference in distance is less than the reference value is specified as a frame in which the facial expression change is completed, and is recorded in the storage means 4. As a result, the analysis target extraction means 2 extracts the frame included between the frame at which the facial expression change has started and the frame at which the facial expression change has ended as the analysis target.

When performing the process shown in FIG. 8, filtering or the like may be appropriately performed. For example, in step S31, a low-pass filter (LPF: Low Pass Filter) is applied to remove noise by applying a low-pass filter (LPF: Low Pass Filter) to the time change of the distance between the face component coordinates (change in the distance between the face component coordinates between each frame). After that, the difference between the frames may be calculated. In addition to this, in order to remove noise, the absolute value of the difference in the distance between the face component coordinates between the frames calculated in step S31 may be set to 0 for a portion less than a predetermined value. it can.

As described above, the facial component coordinates are detected from a plurality of images, the image in which the facial expression change starts and the image in which the facial expression change ends are specified based on the distance between the facial component coordinates, and the images included in the interval are analyzed. By extracting as a target, it is possible to accurately identify a frame containing a predetermined facial expression change. As a result, the movement characteristics of the skin when a predetermined facial expression changes can be analyzed more accurately, and the accuracy of the skin analysis can be improved.

<Measurement of physical quantity>
When the step S11 in FIG. 3 is completed as described above, the movement amount measuring step of the step S12 is started. FIG. 9 is a flowchart showing a processing flow in the movement amount measuring process. Here, in step S12, the processing shown in FIG. 9 is executed for each of the analysis targets in the various facial expression changes extracted in step S11, and the movement amount is measured. In the present embodiment, the amount of movement is measured for each of the analysis targets in the four types of vertical facial expression changes and the four types of horizontal facial expression changes, that is, the eight types of facial expression changes.

First, in step S41, feature points are extracted from each frame in the analysis target. Here, a method of determining the unevenness of the skin from the brightness value of each pixel in the frame and setting the feature points, or a method of setting the feature points in a grid shape in the moving image, that is, arranging the feature points in a grid pattern. Any method may be used, such as a method of

In this embodiment, the amount of movement is measured by the optical flow method. Specifically, one side of the analysis area of the cheek is divided into a predetermined number, the divided square area is regarded as the basic analysis unit, the central point in each basic analysis unit is regarded as the feature point, and the amount of movement is calculated. Measure. FIG. 10 is a diagram showing an example of a fractionation pattern of the analysis region in the present embodiment. The shaded area in FIG. 10 is one basic analysis unit in each fractionation pattern. In the present embodiment, one side is first divided into 12, and motion vectors in analysis units of various sizes are specified (step S42). The motion vector can be obtained by the change in brightness between frames and the like.

After that, in step S43, the motion vector is analyzed and the movement amount is measured. Examples of the amount of movement here include the amount of displacement of the coordinates of the feature points mentioned above, the movement speed, acceleration, the movement direction, and the wave motion parameters of the plane movement, such as the period and the circumference, and the movement of the skin of the face. The amount of is mentioned.

In addition, each such movement amount may be obtained as a value in each frame, or may be obtained as an average value through one facial expression change. Alternatively, a region such as a cheek or a chin may be set, and an integrated value or an average value of values in the region may be obtained.

In step S43, in the present embodiment, the analysis area is first divided into 12 × 12 144 basic analysis units. Next, by measuring the amount of movement of each basic analysis unit and taking the average value between adjacent basic analysis units, 6 × 6, 4 × 4, 3 × 3, 2 × 2, 1 × 1, respectively. The amount of movement of each basic analysis unit in the fractionation pattern of can be specified. As a result, in the present embodiment, a total of 210 vector data can be used as the movement amount for each frame.

Further, in the present embodiment, a value based on the comparison of the movement amount between the basic analysis units in each fractionation pattern and a comparison of the movement amount in the same region between different fractionation patterns (basic analysis of different sizes) are further performed. A value based on (comparison between units) and a value are calculated for each frame, and these are used as the movement amount in addition to the above vector data. More specifically, the difference in the amount of movement between the basic analysis units in each fractionation pattern is brute-forced, and the movement of the corresponding region (basic analysis unit indicating the same position) between different fractionation patterns is performed. The difference in quantity can be brute-forced and used as the amount of movement for each frame.

In this way, by analyzing the amount of movement from various viewpoints for each position (basic analysis unit) in the analysis region, effects such as observing changes in motor characteristics in each region and reducing individual differences can be expected.

As described above, by executing the process shown in FIG. 9 in step S12 for each of the analysis targets in the various facial expression changes extracted in step S11, four types of vertical facial expression changes and four types of horizontal facial expressions The amount of movement can be obtained for each of the changes, that is, the analysis targets in the eight types of facial expression changes.

In the physical quantity measuring step of step S13, the physical quantity is measured based on the movement amount corresponding to these eight facial expression changes. In the present embodiment, various movement amounts including the movement direction and speed "for each frame" as described above, that is, a two-dimensional waveform having a time dimension is converted into a scalar amount by an arbitrary method. For example, conversion is possible by using analysis methods such as frequency analysis, autoregressive model analysis, and wavelet analysis. As a result, the converted scalar quantity can be used as a physical quantity. At this time, it is expected that a huge physical quantity can be obtained, but the physical quantity may be reduced by any method such as principal component analysis.

Further, any value derived from the movement amount may be used as the physical quantity, such as the average value and total of the movement amount and the physical quantity in each facial expression change, and the value calculated by weighting each movement amount.

<Calculation of viscoelasticity>
In the viscoelasticity calculation step of step S14, the viscoelasticity of the skin of the subject is calculated from the physical quantity of the subject measured in step S13 based on the correlation between the physical quantity and the viscoelasticity of the skin.

FIG. 11 is a flowchart showing a processing flow in the viscoelasticity calculation process. In this embodiment, in step S14, the viscoelasticity and evaluation value of the dermis and the viscoelasticity and evaluation value of the subcutaneous skin are calculated, respectively.

First, in step S51, the physical quantity-viscoelastic correlation data stored in the storage means 4 is acquired. In the present embodiment, the physical quantity-viscoelasticity correlation data includes the correlation between the physical quantity and the subcutaneous viscoelasticity and the correlation between the physical quantity and the viscoelasticity of the dermis.

Next, in step S52, the viscoelasticity of the skin is calculated based on the correlation acquired in step S51 and the physical quantity measured by the physical quantity measuring means 3. Specifically, for example, a function representing the relationship between the physical quantity and the viscoelasticity of the skin is registered in advance in the storage means 4 as a viscoelasticity calculation model, and the physical quantity measured by the physical quantity measuring means 3 is stored in the function. There is a method such as calculating the viscoelasticity of the skin by substituting.

The function representing the relationship between the physical quantity and the viscoelasticity of the skin can be obtained by a statistical method from the data of the set of the measured values of the physical quantity and the viscoelasticity. More specifically, a plurality of sets of measured values of the physical quantity measured for the subject by the above method and the viscoelasticity of the skin measured for the same subject are input to the computer, for example, linear regression or decision tree regression. , Support vector regression, Gaussian process regression, etc. can be used to obtain functions (regression models). Further, the function representing the relationship between the physical quantity and the water content and the function expressing the relationship between the viscoelasticity and the collagen structure, which will be described later, can be obtained in the same manner. In the present embodiment, such a viscoelasticity calculation model is created for each of the dermis viscoelasticity and the subcutaneous viscoelasticity, and each viscoelasticity is calculated in step S52.

When the calculation of viscoelasticity is completed, the evaluation value of viscoelasticity is calculated in step S53. In the calculation of the evaluation value, the viscoelasticity calculated in step S52 is used to calculate the evaluation value of each of the dermis and the subcutaneous viscoelasticity. For example, the calculated viscoelasticity can be subjected to a predetermined process, and a standardized value or the like can be used as the evaluation value so that the subject can easily understand it sensuously.

Here, steps S51 and S52 are performed by the subcutaneous viscoelasticity calculating means 511 when calculating the subcutaneous viscoelasticity, and by the dermis viscoelasticity calculating means 512 when calculating the dermal viscoelasticity. Further, step S53 is performed by the subcutaneous viscoelasticity evaluation means 62 when calculating the evaluation value of the subcutaneous viscoelasticity, and by the dermis viscoelasticity evaluation means 61 when calculating the evaluation value of the dermis viscoelasticity.

<Calculation of water content>
When the step S14 is completed, the water content calculation step is started in the step S15. In step S15, the water content of the epidermis of the subject is calculated from the physical quantity of the subject measured in step S13 based on the correlation between the physical quantity and the water content of the epidermis.

FIG. 12 is a flowchart showing the flow of processing in the water content calculation process. First, in step S61, the water content calculating means 53 acquires the physical quantity-water content correlation data stored in the storage means 4. Next, in step S62, the water content calculating means 53 calculates the water content of the epidermis based on the correlation acquired in step S61 and the physical quantity measured in step S13. Specifically, for example, a function representing the relationship between the physical quantity and the water content of the epidermis is registered in the storage means 4 as a water content calculation model, and the physical quantity measured by the physical quantity measuring means 3 is substituted into the function. There is a method such as calculating the water content of the epidermis by doing so. The function that expresses the relationship between the physical quantity and the water content of the epidermis is obtained by a statistical method from the data of the set of the measured values of the physical quantity and the water content, like the function that expresses the relationship between the physical quantity and the viscoelasticity. be able to.

When the calculation of the water content is completed, in step S63, the water content evaluation means 65 calculates the evaluation value of the water content of the epidermis using the water content calculated in step S62. As the evaluation value, for example, a value obtained by subjecting the water content calculated in step S62 to a predetermined treatment and standardized can be used.

<Estimation of collagen structure>
In the collagen structure estimation step of step S16, the collagen structure of the skin of the subject is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. In the present embodiment, in step S16, the dermis collagen structure and the evaluation value and the subcutaneous collagen structure and the evaluation value are calculated, respectively. FIG. 13 is a flowchart showing a processing flow in the collagen structure estimation step.

In the collagen structure estimation step of the present embodiment, for the subcutaneous collagen structure, the skewness of the histogram derived by the subcutaneous collagen structure estimation means 521 analyzing the ultrasonic image of the subcutaneous adipocyte is estimated as the subcutaneous collagen structure. , Subcutaneous collagen structure evaluation means 64 estimates the collagen structure by dividing the skewness into a plurality of stages and determining the evaluation value. That is, the evaluation value of skewness is estimated as the collagen structure.

The histogram used here is obtained as a result of cutting out the subcutaneous fat portion from the ultrasonic image of the subject and using the image analysis software as an image for analysis. When this histogram is analyzed and the skewness is calculated, the lower the degree of fibrosis, the higher the skewness, and the higher the degree of fibrosis, the lower the skewness. Depending on the skewness, the fibers of the collagen fiber structure that wraps the subcutaneous fat You can evaluate the level of conversion. Therefore, in the present embodiment, the skewness of the histogram obtained as described above is used as an index of the collagen structure under the skin.

As for the dermal collagen structure, the microscopic image of the dermal collagen fibers is subjected to image processing, and the value obtained by quantifying the degree of cohesion of the collagen fibers is estimated as the "dermis collagen structure".

First, in step S71, the subcutaneous viscoelasticity calculated by the subcutaneous viscoelasticity calculating means 511 is acquired. Next, in step S72, the viscoelasticity-collagen structure correlation data stored by the storage means 4 is acquired. In this embodiment, the viscoelasticity-collagen structure correlation data includes the correlation between dermal viscoelasticity and dermal collagen structure and the correlation between subcutaneous viscoelasticity and subcutaneous collagen structure.

In the present embodiment, a function representing the relationship between the subcutaneous viscoelasticity and the skewness of the histogram obtained by analyzing the ultrasonic image of the subcutaneous fat layer is registered in the storage means 4 as a subcutaneous collagen structure estimation model. The skewness is obtained by substituting the viscoelasticity calculated by the subcutaneous viscoelasticity calculation means 511 into the function. The function representing the relationship between the subcutaneous viscoelasticity and the skewness can be obtained by a statistical method from the data of the set of the measured values of the viscoelasticity and the skewness.

In addition, the dermis collagen structure is a function that expresses the relationship between the viscoelasticity of the dermis and the quantified value of the degree of cohesion of the collagen fibers obtained by performing image processing on the microscopic image of the dermis collagen fibers. The degree of cohesion of the dermal collagen fibers is obtained by registering it in the storage means 4 as an estimation model and substituting the viscoelasticity calculated by the dermal viscoelasticity calculation means 512 into the function. The function representing the relationship between the dermal viscoelasticity and the degree of cohesion can be obtained by a statistical method from the data of the set of the measured values of the viscoelasticity and the degree of cohesion.

The order of steps S71 and S72 may be arbitrarily changed. Further, as the correlation data, a calculation model trained by the combination of the viscoelasticity and the measured value of the collagen structure (skewness or cohesion) may be used.

In step S73, the collagen structure estimating means 52 estimates the dermal collagen structure and the subcutaneous collagen structure of the subject based on the viscoelasticity and viscoelasticity-collagen structure correlation data of the skin acquired in steps S71 and S72. At this time, the storage means 4 may store the analysis result.

When the estimation of the collagen structure is completed, the evaluation value of the collagen structure is calculated in step S74. In the calculation of the evaluation value, the evaluation value of each of the dermis and subcutaneous collagen structures is calculated using the collagen structure calculated in step S73. For example, a value obtained by subjecting the calculated collagen structure (skewness or cohesion) to a standardized value, a value for evaluating the value of the collagen structure in a plurality of stages, and the like can be used as the evaluation value.

As described above, according to the present embodiment, it is possible to easily analyze non-invasive skin by having the subject reproduce specific facial expressions in order and analyzing a moving image of the state. Specifically, the physical quantity of the skin change caused by the change in facial expression is measured from the moving image, and the viscoelasticity of the subject's skin is calculated from the measured value of the physical quantity for the subject based on the correlation between the physical quantity and the viscoelasticity of the skin. be able to.

Further, as the viscoelasticity of the skin, the viscoelasticity of the dermis and the viscoelasticity of the subcutaneous skin can be calculated, respectively, and the evaluation values thereof can be calculated. Further, the water content of the epidermis and its evaluation value can be calculated. In addition, the collagen structure of the skin can be estimated based on the viscoelasticity of the skin, and the evaluation value thereof can be calculated. Thereby, detailed results can be presented by the combination of these evaluation values.

In the present embodiment, the storage means 4 stores a function representing each relationship obtained by a statistical method from the data of a set of various measured values, and according to this function, the viscoelasticity and water content of the skin. The morphology for estimating the collagen structure is shown. However, in addition to this, for example, by giving a set of measured values of physical quantity and viscoelasticity (or water content) as teacher data and training it in a neural network, a viscoelasticity calculation model, a water content calculation model, and a collagen structure estimation model can be obtained. It may be a configuration to be created. In addition, the configuration may be arbitrarily changed without departing from the spirit of the present invention, such as changing the order of various processes.

Hereinafter, the results of calculation experiments on the viscoelasticity of the skin based on the measured values of physical quantities and the estimation results of the collagen structure based on the viscoelasticity will be described by the present inventors. Although the present invention has been made based on the following experimental results, the values and correlations used in the present invention are not limited to the following contents.

<Experimental result 1>
(1) Measurement of physical quantity The physical quantity was measured by acquiring moving images showing the process of facial expression change for 100 people in their 20s and 60s by the same method as in the above embodiment. Specifically, the analysis region of the cheek was divided into basic analysis units according to a plurality of fractionation patterns, and the amount of movement of each region was measured. Then, such a movement amount is further analyzed for each frame by comparing the movement amount of the basic analysis unit in each fractionation pattern and the movement amount of the corresponding region between the fractionation patterns, and has a time dimension. The physical quantity was measured by converting the amount of movement into a scalar quantity. Measurement of such physical quantities was performed for each facial expression change.

(2) Measurement of viscoelasticity and water content The same subjects as in (1) were measured for viscoelasticity and water content. The viscoelasticity of the dermis was measured using a device capable of measuring the viscoelasticity by sucking the skin surface by negative pressure and monitoring the displacement of the skin when it was opened. The subcutaneous viscoelasticity was measured by acquiring an elastographic image of the inside of the skin using elastography. The water content of the epidermis was measured using a device that measures the water content of the stratum corneum of the epidermis by a high-frequency sinusoidal current.

(3) Construction of various models Statistical analysis was performed using the results of (1) and (2) as a set for each subject, and a viscoelasticity calculation model for each of the dermis and subcutaneous and a water content calculation model for the epidermis were created. .. Specifically, regression analysis was performed by an arbitrary method to obtain a function that describes the relationship between the physical quantity, viscoelasticity, and water content, and used as a viscoelasticity calculation model and a water content calculation model.

(4) Verification of accuracy The physical quantity, viscoelasticity, and water content of subjects other than those in (1) to (3) were measured in the same manner, and the accuracy of the model was verified. Specifically, using the model (3), viscoelasticity and water content were calculated from physical quantities and compared with actual measurement results. For viscoelasticity and water content, evaluation values in a plurality of stages were determined based on the values, and it was confirmed whether or not the evaluation values match. As a result, the estimation accuracy when a one-step deviation of the evaluation value is allowed is as follows.
Epidermis water content: 95%
Dermal viscoelasticity: 90%
Subcutaneous viscoelasticity: 90%

From the result of (4), it was shown that the viscoelasticity and the water content can be estimated with high accuracy by the measured value of the physical quantity of the skin change acquired based on the moving image showing the process of the facial expression change.

<Experimental result 2>
(1) Measurement of viscoelasticity and evaluation of collagen structure Subcutaneous viscoelasticity was measured by the same method as in (2) in <Experimental Result 1>. Then, a subcutaneous fat portion was cut out from the ultrasonic image of the same subject, and a histogram was created using this as an image for analysis using image analysis software. When this histogram was analyzed and the skewness was calculated, the skewness increased as the degree of fibrosis decreased, and the skewness decreased as the degree of fibrosis increased. That is, the fibrosis level of the collagen fiber structure surrounding the subcutaneous fat can be evaluated by the skewness.

(2) Regression analysis A regression analysis was performed on the measurement results of viscoelasticity in (1) and the skewness indicating the fibrosis level of subcutaneous adipocytes. Then, it was found that a positive correlation was established between the viscoelasticity under the skin and the skewness of the histogram of the ultrasonic image of the subcutaneous fat layer.

From the result of (2), it was shown that the fibrosis level (subcutaneous collagen structure) of subcutaneous adipocytes can be estimated from the subcutaneous viscoelasticity. Furthermore, since the subcutaneous viscoelasticity can be estimated from the measured value of the physical quantity of the skin change acquired based on the moving image showing the process of facial expression change, the collagen structure under the skin is indirectly based on the measured value of the physical quantity. It was shown that it is possible to estimate.

The following is an example of a detailed embodiment relating to the shooting of moving images and the output of analysis results.

<Video shooting>
As described above, when shooting a moving image, a guide for aligning the positions of facial parts such as eyes and contours can be displayed on the shooting screen. That is, the skin analysis device 10 or the terminal device 20 includes means for displaying and processing a guide for aligning the positions of facial parts on the subject's face. Here, a more suitable method for shooting a moving image using the terminal device 20 as a shooting device will be described in detail. Although the case where the tablet terminal is used as the terminal device 20 will be described below, any computer device including a camera, a gyro sensor, an acceleration sensor and the like can be used as the terminal device.

FIG. 14 is a diagram showing the appearance of the terminal device 20. FIG. 14A is a diagram showing the back side of the terminal device 20. As shown here, a camera C is provided on the back side of the terminal device 20, and the photographing means 201 uses this to photograph a moving image. FIG. 14B is a diagram showing the front side of the terminal device 20. As shown here, on the front side of the terminal device 20, various displays for the photographer, a touch panel T for receiving operations from the operator, and the like are provided.

In the present embodiment, as shown in FIG. 14, a tablet-type terminal which is substantially rectangular when viewed from the front side or the back side has an x-axis in the long side direction, a y-axis in the short side direction, and The z-axis is taken in the direction from the front side to the back side, respectively, and the holding angle and the rotation angle of the terminal device 20 are expressed. However, the present invention is not limited to this, and may be configured to take an axis in another manner, or may have a configuration in which the axis is taken with respect to the subject instead of the terminal device 20.

FIG. 15 is a flowchart showing the flow of shooting a moving image in the present embodiment. This is an example of a case where the photographer uses the terminal device 20 to shoot a moving image including a change in the facial expression of the subject's face. Before performing each of the processes shown here, it is preferable for the subject to explain in advance what kind of facial expression changes during imaging. In this explanation, the terminal device 20 may be configured to include a means for displaying a moving image for explaining the photographed content on the touch panel T, and may be shown to the photographer or the subject.

First, in step S81, the angle is reset. This is a process for determining the initial position of the terminal device 20. In the present embodiment, the initial state of the terminal device 20 is that the y-axis is parallel to the vertical direction, the x-axis is substantially parallel to the subject's face, and the z-axis is substantially perpendicular to the subject's face. The position. That is, when viewed from above, it is in a state as shown in FIG. 16A. More specifically, on the screen as shown in FIG. 17, a guideline for aligning the subject's eye height, chin height, and nose position is provided as a guide for alignment on the touch panel T of the terminal device 20. By superimposing and displaying on the face of the subject photographed by the camera C, the photographer is urged to align the terminal device 20. Further, by displaying the pointer P that moves according to the inclination of the z-axis, the alignment becomes easier. Then, the angle is reset by accepting an operation by the touch panel T (selection of the "reference value set" button in FIG. 17), and the initial position of the terminal device 20 is determined.

Here, it is preferable that the rotation of the terminal device 20 around the x-axis, that is, the inclination in the front-rear direction and the rotation around the z-axis, that is, the inclination in the left-right direction can be reduced. .. For this purpose, for example, the vertical direction and the horizontal direction orthogonal to the gravitational acceleration are specified by a gyro sensor, an acceleration sensor, or the like provided in the terminal device 20, and the angle between the x-axis and the z-axis is the horizontal direction. The difference may be displayed as an inclination around each axis, these may be brought close to 0, and the photographer may be prompted to adjust the angle of the terminal device 20 so as to be within a predetermined allowable range.

After resetting the angle in this way and determining the initial position of the terminal device 20, the terminal device 20 is moved in step S82. In the present embodiment, the terminal device 20 is rotated 25 degrees, and the face of the subject is photographed from an angle of 25 degrees diagonally forward to the left. Here, by moving the terminal device 20 to the right toward the subject so as to be in the state shown in FIG. 16B when viewed from above, the face of the subject is photographed diagonally from the front left. , Adjust the position. That is, the terminal device 20 may be moved so as to draw an arc centered on the subject and having a central angle of 25 degrees while keeping the state in which the camera C of the terminal device 20 is directed at the subject. When such a movement is performed, as shown in FIG. 16B, the terminal device 20 rotates 25 degrees counterclockwise around the y-axis. The shooting direction may be either left or right, or both left and right may be shot.

As a more specific adjustment method here, as shown in FIG. 18A, a guideline for aligning the subject's eye height, chin height, and left eye position is displayed on the touch panel T, and the photographer. It encourages alignment. At this time, the touch panel T moves in accordance with the rotation of the terminal device 20 around the y-axis, and when it is rotated 25 degrees, a pointer P that overlaps with the center guideline is further displayed. When it is determined that is appropriate, the display form such as the guideline and the color of the pointer is changed.

Since the terminal device 20 is in a state of being held by the photographer, the terminal device 20 is not configured to allow only the case where the temperature is exactly 25 degrees, but is allowed in advance, for example, 25 degrees ± 5 degrees. It is preferable that the configuration is such that the angle range is defined.

In this way, by displaying the guideline and aligning the terminal device 20 according to the guideline, even if the terminal device 20 does not accurately move in the above-mentioned arc drawing, as long as the subject does not change the direction. , It is possible to reliably shoot from a predetermined angle. As a result, the shooting angle can be easily and surely adjusted even in a configuration in which the photographer holding the terminal device 20 moves instead of fixing the terminal device 20 (or the subject) with an instrument and moving it. ..

In the present embodiment, as described above, a configuration is shown in which three guidelines for adjusting the eye height, the chin height, and the left eye position of the subject are displayed, but the present invention is limited to this. is not. By displaying points and lines for aligning the position of the subject's face parts as guides and adjusting the position of the terminal device 20 accordingly, it is possible to take an image of the subject in an appropriate position from an appropriate distance. It will be possible.

At this point, even if the rotation angle is included in the permissible range and it is determined to be appropriate, the focus and exposure adjustments have not been completed, so recording cannot be started. Therefore, it is preferable to show the photographer that the recording cannot be started yet, such as displaying or not displaying the button for starting recording in a state in which the button cannot be pressed.

If it is determined in step S83 that the rotation angle of the terminal device 20 is within a predetermined range as a result of the movement of the terminal device 20 in step S82, the process proceeds to step S84, and the focus and exposure are automatically adjusted. These adjustments may be performed by any method, and particularly when a tablet terminal is used as in the terminal device 20 according to the present embodiment, the adjustment is provided by an OS (Operating System) vendor or the like running on the terminal. The configuration may be such that the API (Application Programming Interface) related to the control of the camera is used. Further, not only the automatic adjustment but also the manual adjustment may be possible. Here, there is a configuration provided with means for automatically adjusting not only focus and exposure but also other parameters related to image shooting such as ISO sensitivity, and an LED (Light Emitting Diode) that detects the illuminance of the shooting environment and when the illuminance is low. ) Or the like to provide a means for illuminating the subject.

After the automatic focus and exposure adjustment is completed in step S84, the focus and exposure lock (AFAE lock) is performed. Then, when it is determined in step S85 that the adjustment of focus and exposure is completed, the process proceeds to step S86, and the state is such that the photographer can press the recording start button.

Then, after receiving the instruction to start recording from the photographer, recording by the camera C is started in step S87. Even after the start of recording, it is preferable to continue measuring the rotation angle of the terminal device 20 and display it to the photographer as shown in FIG. 18 (c). If the rotation angle deviates from the allowable range adjusted in step S86 during recording, the recording may be stopped immediately, or the photographer may be prompted to adjust the rotation angle and the recording may be continued. May be.

In this embodiment, since a moving image including a change in the facial expression of the subject's face is taken, for example, as shown in FIG. 18C, when the subject opens his / her mouth wide during recording of the moving image, the chin In some cases, the position of is below the guideline, and the position is deviated from the state in which the alignment was performed in step S82. Even in such a case, since the AFAE lock is performed, there is no problem as long as the position of the subject and the position of the terminal device 20 do not move. Alternatively, during recording, the photographer adjusts the position of the terminal device 20 so as to maintain the position of the left eye in accordance with the guideline, and it is considered that there is little movement due to changes in facial expressions. You may take a method that keeps matching.

Further, the measurement of the rotation angle of the terminal device 20 during recording is performed not only for the rotation around the y-axis but also for the rotation around the x-axis and the z-axis, that is, the inclination of the terminal device 20 in the front-rear direction and the left-right direction. Is preferable. Here, too, the recording may be stopped immediately when the inclination of the terminal device 20 exceeds a predetermined allowable range, or the photographer may be prompted to adjust the rotation angle and the recording may be continued. May be good. Alternatively, the terminal device 20 may be configured to perform correction according to the inclination of the terminal device 20, or the inclination of the terminal device 20 at each time point during recording may be recorded, and the correction may be made at the time of later analysis by the skin analysis device 10. It may be configured so that it can be performed. As a correction method here, for example, if it is for tilting in the left-right direction, each frame image in the recorded moving image may be rotated, and if it is for tilting in the front-back direction, each frame image may be rotated. A method such as applying keystone correction can be mentioned.

Then, when it is determined in step S88 that the recording is finished, the recording is finished and the video file is recorded in step S89. Here, the determination of the end of recording may be based on the operation of the photographer, or the recording may be terminated after a predetermined time has elapsed. The moving image file saved in this way is transmitted to the skin analysis device 10.

As described above, the terminal device 20 according to the present embodiment is a program dedicated to a general-purpose tablet terminal provided with a sensor such as a gyro sensor or an acceleration sensor and a camera C without preparing special equipment or equipment. It can be realized by deploying. As a result, it is possible to easily shoot a moving image by specifying conditions such as a shooting angle for analysis.

In the above embodiment, an example of shooting from diagonally left front of the subject has been described. For example, moving images are shot from the same rotation angle on both the left and right sides, and the physical quantity is calculated from the result of synthesizing the moving images. You may. When the subject is photographed from both the left and right directions, in the present embodiment, the amount of movement can be obtained for the analysis target of eight types of facial expression changes, that is, a total of 16 types of facial expression changes on both the left and right sides. In step S13 of FIG. 3, by calculating the physical quantity using the movement amount obtained from the analysis target in these 16 types of facial expression changes, from the physical quantity that comprehensively considers the skin changes obtained from the left and right facial expression changes. , The physical properties of the skin can be analyzed.

<Specific examples of physical quantities>
Examples of the physical quantity used in the analysis include skin followability, elasticity, and deformability. Here, the "followability of the facial skin in the facial expression change" is the degree of delay in the movement of the facial skin that changes in accordance with the facial expression change. When a change in facial expression occurs, the skin on the face changes with a delay in the movement, but the smaller the degree of the delay, the better the followability.

The followability can be quantitatively evaluated by observing any two points on the face when the facial expression changes and measuring the degree of deviation in the timing of movement of these two points. For example, a method is conceivable in which the motion velocity in a specific analysis region is specified based on the amount of movement, and the difference in time at which the motion velocity is maximized between the analysis regions is quantitatively measured. At this time, using the analysis area corresponding to the position of the face that moves most remarkably in the facial expression change and the analysis area corresponding to the position of the other face, the difference in the time at which the movement speed is maximized between them. It is preferable to measure.

Further, "the elasticity of the skin of the face due to the change of facial expression" refers to the ease of expansion and contraction of the skin when the change of facial expression occurs. For example, when there is a change in facial expression that stretches the skin of the face, the higher the ratio of the increase in the distance in the extension direction to the increase in the distance in the entire extension direction, the more "excellent in elasticity". Can be evaluated. For example, it is possible to evaluate the rate of increase in the distance in the stretching direction in a specific analysis region as elasticity from the amount of movement such as the displacement of the coordinates of the feature points along the stretching direction of the skin when the facial expression changes. it can.

Further, as the deformability, it is conceivable to measure the deformation method (distortion method) of the analysis region corresponding to an arbitrary position of the face. The method for measuring the deformability is not limited, but for example, it may be calculated using the displacement amount of the feature point, the movement amount such as the movement direction, and the like.

In particular, it is known that followability and elasticity each have a negative correlation with age, and both tend to decrease with aging. It is also known that a positive correlation is established between followability and subcutaneous viscoelasticity, and it is possible to calculate viscoelasticity based on the correlation between followability and viscoelasticity. Conceivable.

<Output of analysis result>
The skin analysis device 10 may further include an output means for outputting the analysis result. As an analysis result, the output means outputs information based on the viscoelasticity of the dermis and the subcutaneous, the water content of the epidermis, the calculation result of the collagen structure of the dermis and the subcutaneous, and their evaluation values. The form of output is not limited, and the terminal device 20 may display it as a static or dynamic image, or may output it as sound, vibration, or characters.

Furthermore, not only the calculation result of viscoelasticity and water content and the estimation result of collagen structure, but also the physical quantity used for the analysis, that is, the information based on the physical quantity having a correlation with any of viscoelasticity, water content and collagen structure is output. You may. This makes it possible to clearly show how the measured values and the values obtained from the captured moving images affect the analysis results.

<Example of modification related to estimation of collagen structure>
In the above embodiment, the collagen structure estimating means 52 shows a form in which the collagen structure is estimated by using the calculation result by the viscoelasticity calculating means 51 and the viscoelasticity-collagen structure correlation data. Not limited. As described above, since the correlation between the physical quantity and the viscoelasticity is recognized, and further the correlation between the viscoelasticity and the collagen structure is recognized, there is a direct correlation between the physical quantity and the collagen structure. Probability is high.

Therefore, for example, the collagen structure estimating means 52 may directly estimate the collagen structure of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the collagen structure of the skin.

In this case, the storage means 4 stores a function representing the relationship between the physical quantity and the index related to the collagen structure as the physical quantity-collagen structure correlation data, and the physical quantity measured by the physical quantity measuring means 3 is stored in this function. By substituting, the index of collagen structure can be calculated.

Even when the collagen structure is estimated directly from the physical quantity without calculating the viscoelasticity, the subcutaneous collagen structure, the collagen structure of the dermis, and the evaluation value thereof should be estimated as in the above-described embodiment. Is preferable. That is, the storage means 4 has a correlation between the physical quantity and the collagen structure under the skin (for example, the skewness of the histogram obtained by analyzing the ultrasonic image of the subcutaneous fat layer), and the physical quantity and the collagen structure of the dermis (for example, dermis collagen). The correlation with (a value obtained by quantifying the degree of cohesion of collagen fibers obtained from the histogram image of the fibers) is memorized.

By estimating the collagen structure based on the correlation between the physical quantity and the collagen structure in this way, an effect of improving the estimation accuracy can be expected as compared with the case of estimating the collagen structure via viscoelasticity.

10 Skin analysis device 1 Video acquisition means 2 Analysis target extraction means 3 Physical quantity measurement means 31 Feature point extraction means 32 Movement amount measurement means 33 Movement amount analysis means 4 Storage means 5 Analysis unit 51 Viscoelasticity calculation means 511 Subcutaneous viscoelasticity calculation means 512 Dermal Viscoelasticity Estimating Means 52 Collagen Structure Estimating Means 521 Subcutaneous Collagen Structure Estimating Means 522 Dermal Collagen Structure Estimating Means 53 Moisture Amount Calculation Means 6 Evaluation Means 61 Dermal Viscoelasticity Evaluating Means 62 Subcutaneous Viscoelasticity Evaluation Means 63 Dermal Collagen Structure Evaluating Means 64 Subcutaneous Collagen structure evaluation means 65 Moisture content evaluation means 20 Terminal device 201 Imaging means 202 Display unit C Camera T Touch panel P Pointer

Claims (42)

A skin analysis method characterized in that the viscoelasticity of a subject's skin is calculated from the measured value of the physical quantity of the subject based on the correlation between the physical quantity of the skin change caused by the change in facial expression and the viscoelasticity of the skin. .. The skin analysis method according to claim 1, wherein the viscoelasticity includes at least one of dermis viscoelasticity and subcutaneous viscoelasticity. It is characterized in that the viscoelasticity of the subject's skin is calculated from the measured values of the physical quantity for the subject by using the viscoelasticity calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the viscoelasticity. , The skin analysis method according to claim 1 or 2. The invention according to any one of claims 1 to 3, wherein the collagen structure of the skin of the subject is estimated from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. Skin analysis method. Using a collagen structure estimation model created by inputting a plurality of sets of measured values of viscoelasticity and skin collagen structure, it is possible to estimate the skin collagen structure of a subject from the viscoelasticity calculated for the subject. The skin analysis method according to claim 4, which is characterized. The skin analysis method according to any one of claims 1 to 5, wherein one or a plurality of evaluation values of the skin are calculated based on the viscoelasticity. The skin analysis method according to any one of claims 1 to 6, wherein an analysis result including information based on the calculation result of viscoelasticity is output. The skin analysis method according to claim 7, wherein an analysis result including information based on the measured value of the physical quantity is output together with information based on the calculation result of viscoelasticity. A skin analysis method, which comprises estimating the collagen structure of a subject's skin from the measured value of the physical quantity of the subject based on the correlation between the physical quantity of the skin change caused by the change of facial expression and the collagen structure of the skin. .. The skin analysis method according to claim 9, wherein an analysis result including information based on the estimation result of the collagen structure is output. The skin analysis method according to claim 10, wherein an analysis result including information based on the measured value of the physical quantity is output together with information based on the estimation result of the collagen structure. The skin analysis method according to any one of claims 1 to 11, wherein the physical quantity is measured based on a moving image including the process of changing the facial expression. The skin analysis method according to claim 12, wherein a guide for aligning the positions of facial parts on the face of the subject is displayed on the screen for capturing the moving image. The moving image according to claim 12 or 13, wherein the moving image is taken from a direction rotated in a range of 20 to 30 degrees with the front surface as 0 degrees around a vertical axis passing through the subject's face. Skin analysis method. The moving images include a moving image taken from a direction in which the front is rotated clockwise in the range of 20 to 30 degrees with the front as 0 degrees around a vertical axis passing through the subject's face, and a vertical axis passing through the subject's face. The skin analysis method according to claim 14, wherein both a moving image taken from a direction in which the front surface is rotated counterclockwise in the range of 20 to 30 degrees and a moving image taken are used. The video is
The initial position is the state in which the subject is captured from the front, and the rotation angle of the imaging device from the initial position is acquired.
When the rotation angle is within the range of 20 to 30 degrees, shooting of the moving image is permitted.
The skin analysis method according to claim 14 or 15, wherein the image is taken based on the permission. The physical quantity is measured based on at least one of the magnitude, direction of velocity, magnitude of acceleration, and direction of acceleration of the feature points contained in the face. Item 4. The skin analysis method according to any one of Items 1 to 16. The skin analysis method according to any one of claims 1 to 17, wherein the physical quantity is selected from the followability, elasticity, and deformability of the skin. The method according to any one of claims 1 to 18, wherein the water content of the epidermis of the subject is calculated from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the water content of the epidermis. Skin analysis method. It is characterized in that the water content of a subject is calculated from the measured value of the physical quantity for the subject by using a water content calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the water content. The skin analysis method according to claim 19. A skin analysis system that analyzes the subject's skin based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the viscoelasticity of the skin,
A skin analysis system comprising: a viscoelasticity calculating means for calculating the viscoelasticity of a subject's skin from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the viscoelasticity of the skin. The skin analysis system according to claim 21, wherein the viscoelasticity calculating means calculates at least one of dermis viscoelasticity and subcutaneous viscoelasticity. The storage means stores a viscoelasticity calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the viscoelasticity.
The skin analysis system according to claim 21 or 22, wherein the viscoelasticity calculation means calculates the viscoelasticity from the physical quantity measured for a subject using the viscoelasticity calculation model. The storage means memorizes the correlation between the viscoelasticity and the collagen structure of the skin.
21. Claim 21 is further provided with a collagen structure estimating means for estimating the collagen structure of the skin of the subject from the viscoelasticity calculated for the subject based on the correlation between the viscoelasticity and the collagen structure of the skin. The skin analysis system according to any one of ~ 23. The storage means stores a collagen structure estimation model created by inputting a plurality of sets of measured values of viscoelasticity and skin collagen structure.
The skin analysis system according to claim 24, wherein the collagen structure estimation means estimates the collagen structure of the subject's skin from the viscoelasticity calculated for the subject using the collagen structure estimation model. The skin analysis system according to any one of claims 21 to 25, further comprising an evaluation means for calculating one or a plurality of evaluation values of the skin based on the viscoelasticity of the skin of the subject. The skin analysis system according to any one of claims 21 to 26, further comprising an output means for outputting an analysis result including information based on the calculation result of viscoelasticity. The skin analysis system according to claim 27, wherein the output means outputs an analysis result including information based on the measured value of the physical quantity together with information based on the calculation result of viscoelasticity. A skin analysis system that analyzes the subject's skin based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the collagen structure of the skin,
A skin analysis system comprising: a collagen structure estimating means for estimating a collagen structure of a subject's skin from a measured value of the physical quantity of the subject based on a correlation between the physical quantity and a collagen structure of the skin. The skin analysis system according to claim 29, further comprising an output means for outputting information based on the estimation result of the collagen structure. The skin analysis system according to claim 30, wherein the output means outputs an analysis result including information based on the measurement value of the physical quantity together with information based on the estimation result of the collagen structure. Further provided with a moving image acquisition means for acquiring a moving image including the process of the facial expression change,
The skin analysis system according to any one of claims 21 to 31, wherein the physical quantity measuring means measures the physical quantity based on the moving image. The skin analysis system according to claim 32, further comprising means for displaying and processing a guide for aligning the positions of facial parts on the face of the subject on the screen for capturing the moving image. The moving image acquisition means is characterized in that a moving image of a subject's face is acquired from a direction rotated in a range of 20 to 30 degrees with the front surface as 0 degrees around a vertical axis passing through the subject's face. The skin analysis system according to claim 32 or 33. The moving image acquisition means includes a moving image taken from a direction in which the front is rotated clockwise in the range of 20 to 30 degrees with the front as 0 degrees around an axis in the vertical direction passing through the subject's face, and a vertical direction passing through the subject's face. 34. The skin analysis system according to claim 34, wherein a moving image taken from a direction in which the front surface is rotated counterclockwise in a range of 20 to 30 degrees with the front surface as 0 degrees is acquired. The moving image acquisition means acquires the moving image taken by a terminal device for taking a picture of a subject under a predetermined condition, and obtains the moving image.
The terminal device is
A means for acquiring the rotation angle of the terminal device from the initial position, with the state in which the subject is captured from the front as the initial position, and
A means for permitting the shooting of the moving image when the rotation angle is within the range of 20 to 30 degrees, and
The skin analysis system according to claim 34 or 35, which comprises means for photographing the moving image based on the permission. The physical quantity measuring means measures the physical quantity based on at least one of the magnitude, direction of velocity, magnitude of acceleration, and direction of acceleration of feature points included in the face. The skin analysis system according to any one of claims 21 to 36. The skin analysis system according to any one of claims 21 to 37, wherein the physical quantity measuring means measures any of skin followability, elasticity, and deformability as the physical quantity. The storage means stores the correlation between the physical quantity and the water content of the epidermis.
21 to 21 to claim, further comprising a water content calculating means for calculating the water content of the skin of the subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the water content of the epidermis. The skin analysis system according to any one of 38. The storage means stores a water content calculation model created by inputting a plurality of sets of the measured values of the physical quantity and the water content.
The skin analysis system according to claim 39, wherein the water content calculation means calculates the water content from the physical quantity measured for a subject by using the water content calculation model. A skin analysis program for analyzing the subject's skin based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the viscoelasticity of the skin,
A skin analysis program characterized by functioning as a viscoelasticity calculation means for calculating the viscoelasticity of a subject's skin from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the viscoelasticity of the skin. .. A skin analysis program for analyzing the subject's skin based on the movement of the skin when the facial expression changes.
A physical quantity measuring means for measuring the physical quantity of skin changes caused by the facial expression change,
A memory means for storing the correlation between the physical quantity and the collagen structure of the skin,
A skin analysis program characterized by functioning as a collagen structure estimating means for estimating the collagen structure of the skin of a subject from the measured value of the physical quantity of the subject based on the correlation between the physical quantity and the collagen structure of the skin. ..