JP7446000B2 - Beauty promotion device, beauty promotion method, and beauty promotion program - Google Patents

Description

The present invention relates to a beauty promotion device, a beauty promotion method, and a beauty promotion program.

2. Description of the Related Art Conventionally, systems are known that use image processing techniques to promote beauty to users.
For example, in Patent Document 1, the strain distribution of the facial skin is determined using images taken before and after a facial beauty treatment such as a facial massage, and the effect of the beauty treatment is quantitatively evaluated using this strain distribution as an evaluation index. A system for analyzing this is disclosed.

JP 2017-9598 Publication

However, although conventional beauty promotion systems have been able to evaluate changes in the physical form of a user's skin, they have not been able to evaluate pigmentation such as age spots and pigment spots. These pigmentations are symptoms that tend to become more obvious with age, for example, and there is a strong desire to improve them among users who are conscious of beauty on a daily basis.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a beauty promotion device that can quantitatively evaluate the degree of pigmentation such as age spots and pigment spots and can be used for beauty treatments.

In order to solve the above problems, the beauty promotion device according to the present invention uses image data obtained by capturing an image of the user's skin to detect the condition of the user's skin at any location on the user's skin based on its hue value. A skin color evaluation unit divides the user into multiple preset stages for classification, and a pigment abnormality area including spots due to pigmentation is detected on any part of the user's skin based on the stages classified by the skin color evaluation unit. and a pigmentation identifying section that identifies the area where pigmentation has occurred.

Furthermore, an evaluation result display section may be provided that displays the location of the imaged data for each stage divided by the skin color evaluation section and marks the pigment abnormality region identified by the pigmentation identification section.

Furthermore, the evaluation result display section may display the position and marking for each stage of a plurality of pieces of imaging data taken at different times for the same user, and display the data side by side.

Further, the image forming apparatus may include a depth estimating section that estimates the depth of the pigment abnormality part of the user's skin from the skin surface based on the hue value of the pigment abnormality part specified by the pigmentation identification part.

Furthermore, a measure proposing unit may be provided that proposes a measure to promote improvement of pigmentation based on the depth of the pigment abnormality area estimated by the depth estimating unit.

Furthermore, an improvement data generation unit may be provided that assumes a state in which pigmentation in the pigment abnormality area has been improved based on the imaging data, changes the hue of the pigment abnormality area, and displays the changed hue to the user.

Further, in the beauty promotion method according to the present invention, the computer classifies the condition of the user's skin based on the hue value of an arbitrary part of the user's skin using image data obtained by capturing an image of the user's skin. A skin color evaluation step that divides the user into a plurality of stages set in advance, and a pigment abnormality including spots due to pigmentation occurring in any part of the user's skin based on the stages divided by the skin color evaluation step. performing a pigmentation identification step of identifying the portion;

Furthermore, the beauty promotion program of the present invention is configured to cause a computer to classify the condition of the user's skin based on the hue value of any part of the user's skin using image data obtained by capturing an image of the user's skin. A skin color evaluation function that divides the skin color into multiple preset stages, and an area where pigment abnormalities, including spots due to pigmentation, have occurred on any part of the user's skin based on the stages classified by the skin color evaluation function. Pigmentation identification function to identify and achieve.

In the beauty promotion device of the present invention, the skin color evaluation section uses image data obtained by capturing an image of the user's skin to classify an arbitrary part of the user's skin into a plurality of preset stages according to its hue value. . Then, the pigmentation identification section identifies the position of the pigment abnormality on the user's skin based on the stages divided by the skin color evaluation section. Thereby, the degree of pigmentation such as age spots and pigment spots can be quantitatively evaluated and used for beauty treatments.

1 is a schematic diagram showing a configuration example of a beauty promotion system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration example of the beauty promotion device shown in FIG. 1. FIG. This is an example of an evaluation table that serves as an evaluation standard for the skin color evaluation section. (a) A diagram showing an example of a processing result in the beauty promotion system, and (b) a partially enlarged diagram of FIG. 4(a). FIG. 2 is a block diagram showing a configuration example of the mobile terminal shown in FIG. 1. FIG. It is a figure showing the processing flow in a beauty promotion device. It is a figure showing other examples of processing results in a beauty promotion system.

A beauty promotion system 1 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 7.
FIG. 1 is a schematic diagram showing a configuration example of a beauty promotion system 1 according to an embodiment of the present invention. The beauty promotion system 1 is a system for aging management that performs data processing on imaged data of the user's 50 face and quantitatively evaluates the skin condition of the user's 50 face in order to contribute to the beauty promotion of the user 50. This is the system.
In addition, in the following description, the contents of evaluating the skin condition of the user's 50 face will be explained as an example, but the beauty promotion system 1 is not limited to evaluating the skin condition of parts other than the user's 50 face. May be used.

As shown in FIG. 1, the beauty promotion system 1 includes an imaging device 10 and a beauty promotion device 30 that are connected to each other via a network 20. Further, in the illustrated example, a mobile terminal 40 of a user 50 is connected to the network 20. The mobile terminal 40 is connected to the network 20 by wireless communication.

Here, the beauty promotion system 1 of the present invention shows the results of quantitatively evaluating images of the user's 50's face in a store that provides beauty-related services such as a beauty salon. It is used to suggest future measures to improve the condition of your skin.
In addition, when using the beauty promotion system 1, as shown in FIG. 1, it may be performed under the operation of the store operator 60, or the user 50 himself may perform the operation.

The imaging device 10 is not particularly limited as long as it is a device that can image the face of the user 50 and acquire imaging data. The imaging device 10 includes an imaging section having an imaging element such as a CMOS or a CCD. Further, the imaging device 10 includes a transmitting section that transmits image data captured by the imaging section to the beauty promotion device 30.

The imaging data acquired by the imaging device 10 may be 2D data or 3D data. In this embodiment, a configuration in which the imaging unit acquires imaging data as 2D data will be described.
Note that the imaging device 10 may be, for example, a 3D camera in which a plurality of imaging units are arranged at intervals, or may be configured to include one imaging unit and a distance sensor.

The network 20 is a network for interconnecting the imaging device 10, the beauty promotion device 30, and the mobile terminal 40, and is, for example, a wireless network or a wired network.
Specifically, the network 20 includes wireless LAN (WLAN), wide area network (WAN), ISDNs (integrated service digital networks), wireless LANs, and LTE (long term network). evolution), LTE-Advanced, These include fourth generation (4G), fifth generation (5G), CDMA (code division multiple access), WCDMA (registered trademark), and Ethernet (registered trademark).

Further, the network 20 is not limited to these examples, and includes, for example, the Public Switched Telephone Network (PSTN), Bluetooth (registered trademark), Bluetooth Low Energy, optical line, and ADSL. (Asymmetric Digital Subscriber Line) It may be a line, a satellite communication network, or any other network.

Further, the network 20 may be, for example, NB-IoT (Narrow Band IoT) or eMTC (enhanced Machine Type Communication). Note that NB-IoT and eMTC are wireless communication systems for IoT, and are networks capable of long-distance communication at low cost and low power consumption.

Moreover, the network 20 may be a combination of these. Further, the network 20 may include a plurality of different networks combining these examples.
For example, the network 20 may include a wireless network using LTE and a wired network such as an intranet that is a closed network.

Next, the configuration of the beauty promotion device 30 will be explained using FIG. 2. FIG. 2 is a block diagram showing a configuration example of the beauty promotion device 30.
The beauty promotion device 30 includes a device-side communication section 31, a data storage section 32, a data processing section 33, and a device-side display section 34. The beauty promotion device 30 is an information processing device that analyzes the skin condition of the user's 50 face from image data of the user's 50 face, and as an example, a personal computer is used in this embodiment.

The device-side communication unit 31 is a communication interface that transmits and receives various data via the network 20. The various data include imaging data, processing data, and improvement data. That is, the device-side communication unit 31 functions as a receiving unit that receives imaging data transmitted from the transmitting unit of the imaging device 10.
Here, the imaging data refers to data obtained by imaging the user's 50 face by the imaging device 10. Processed data refers to data obtained by specifying and marking the position of a stain on imaging data by an evaluation result display section 33C, which will be described later. The improvement data is data that is displayed to the user 50 by the improvement data generation unit 33F (described later), which changes the hue of the pigment abnormality area based on imaging data, assuming that the pigmentation in the pigment abnormality area has been improved. refers to

The data storage unit 32 has a function of storing various control programs necessary for the operation of the data processing unit 33 and various data received by the device side communication unit 31 from the outside. Further, the data storage unit 32 has an evaluation table that is used as a reference when a skin color evaluation unit 33A, which will be described later, evaluates the skin color of the user 50.
The data storage unit 32 is realized by various storage media such as an HDD, an SSD, and a flash memory.

By executing the control program stored in the data storage section 32, the data processing section 33 realizes each function that should be realized as the beauty promotion system 1. Each function here includes a skin color evaluation function, a pigmentation identification function, an evaluation result display function, a depth estimation function, a measure proposal function, and an improvement data generation function.
The device-side display unit 34 is a monitor device that displays the operation details and processing results of the beauty promotion device 30.

The data processing unit 33 is a computer that controls each unit of the beauty promotion device 30, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like.
Note that the data processing section 33 is not limited to these examples, and may be any computer as long as it controls each section of the beauty promotion device 30.

The data processing section 33 includes a skin color evaluation section 33A, a pigmentation identification section 33B, an evaluation result display section 33C, a depth estimation section 33D, a measure proposal section 33E, and an improvement data generation section 33F.
The skin color evaluation unit 33A classifies any part of the skin of the user 50 into a plurality of stages using the image data obtained by imaging the skin of the user 50.

The multiple stages are preset hierarchies for classifying the skin condition of the user 50, and are represented, for example, from stage 1 to stage 4, and the higher the stage value, the more severe the symptoms of pigment abnormality. means.
The skin color evaluation unit 33A classifies the skin color of the user 50 into a plurality of stages based on skin hue values (RGB values as an example). This content will be explained in detail using FIG. 3. FIG. 3 is an example of an evaluation table that serves as an evaluation standard for the skin color evaluation section 33A. Note that the hue value is not limited to RGB values, and may be CMYK values or other index values.

In an example of the evaluation table shown in FIG. 3, classifications of hue and stage are described for types of pigment spots among pigment abnormalities. Hue can be expressed by RGB values, for example. That is, although hue is expressed by color in this figure, it may also include data of corresponding RGB values.

For example, as shown in FIG. 3, in the case of senile pigment spots, pigment abnormalities tend to occur on the cheekbones, and if the color ranges from brown to brown, it is determined to be stage 1. Furthermore, if the pigment abnormality area is light brown to light brown, it is determined to be stage 2. Further, if the pigment abnormality is pale brown to light brown, it is determined to be stage 3.

Next, in the case of post-inflammatory pigmentation, it tends to occur on the entire face, and if the pigment abnormality is brown to brown, it is determined to be stage 1. Furthermore, if the pigment abnormality is light brown to light brown, it is determined to be stage 2. Furthermore, if the color is from light brown to pale brown, it is determined to be stage 3. Furthermore, if the color ranges from blue to gray, it is determined to be stage 4. Here, the type of pigment spot is determined by the operator 60 based on the position and appearance of the pigment abnormality.
Note that this evaluation table is just an example, and another evaluation table may be used.

The pigmentation specifying unit 33B specifies an arbitrary part of the user's 50 skin where a pigment abnormality including a stain due to pigmentation has occurred based on the stages classified by the skin color evaluation unit 33A. Here, pigmentation refers to skin pigment abnormalities caused by accumulation of black melanin in the epidermis.

Here, to explain the mechanism by which pigment abnormalities such as age spots (pigment spots) occur on the skin, when the skin receives stimulation such as ultraviolet rays, melanocytes (melanin-forming cells), which are internal tissues of the skin, produce black melanin. generate. This black melanin has the role of protecting the skin, but in the case of healthy skin, it is excreted from the body over time.

On the other hand, when the skin's metabolic cycle is disrupted or when a large amount of black melanin is produced, some of the black melanin is not excreted from the body and remains and accumulates in the epidermis. Among these pigment abnormalities, those caused by inflammation or ultraviolet rays are called pigmentation or pigment spots.

The evaluation result display section 33C displays the position of each stage of the imaging data divided by the skin color evaluation section 33A. Furthermore, the evaluation result display section 33C marks the pigment abnormality identified by the pigmentation identification section 33B. As a result, processed data in which a stain position is displayed and a marking is added to the imaged data is generated. Marking is performed by attaching a mark of a color set for each stage to a portion whose hue corresponds to one of stages 1 to 4.
Furthermore, the evaluation result display section 33C can display the position and marking for each stage of a plurality of pieces of image data taken at different times for the same user 50, and display them side by side.

Here, an example of processed data will be explained using FIG. 4. FIG. 4(a) is a diagram showing an example of a processing result in the beauty promotion system 1, and FIG. 4(b) is a partially enlarged view of FIG. 4(a). As shown in FIGS. 4(a) and 4(b), the positions of a plurality of stains are specified for arbitrary positions on the skin of the user's 50 face. At each position, pigment abnormalities are represented by markings M1 to M3. Some of these markings cannot be visually recognized.
In FIGS. 4A and 4B, the thinnest mark M1 indicates stage 1, and the darkest mark M3 indicates stage 3. A mark M2 of intermediate density indicates stage 2. In this figure, the stage 4 mark is not confirmed.

The depth estimating unit 33D estimates the depth of the pigment abnormality in the skin of the user 50 from the skin surface based on the hue value of the pigment abnormality identified by the pigmentation identifying unit 33B. It is generally known that the depth from the skin surface varies depending on the hue of the stain or pigment spot.

For example, in the evaluation table for age spots (pigment spots) shown in Figure 3, pigment abnormalities corresponding to stage 1 occur in the upper layer of the epidermis, and pigment abnormalities corresponding to stage 2 occur in the middle layer of the epidermis. It is determined that there is. It is also determined that the pigment abnormality corresponding to stage 3 occurs in the lower layer of the epidermis, and the pigment abnormality corresponding to stage 4 occurs from the lower layer of the epidermis to the dermis layer.
Here, the upper side refers to the side inside the skin that faces the surface, and the lower side refers to the side inside the skin that faces into the body. Criteria for determining color and depth can be set arbitrarily.

The measure proposing unit 33E proposes measures to promote improvement of pigmentation based on the depth of the pigment abnormality area estimated by the depth estimating unit 33D. The measures include the use of facial beauty devices, use of introduced beauty serums, use of carbonated packs, and UV protection. Which of these is appropriate is determined depending on the depth of the pigment abnormality. In addition, for pigment abnormalities formed in deep layers, it may be suggested that the patient be examined by a medical institution.

The improvement data generation unit 33F assumes a state in which the pigmentation in the pigment abnormality area has been improved with respect to the imaging data, changes the hue of the pigment abnormality area, and displays the changed hue to the user 50. In other words, it has the function of visually expressing what kind of effects will be obtained when the proposed measures are implemented for a certain period of time.

The improvement data generation unit 33F uses, for example, past history data to estimate how the pigment abnormality area will change if the measure is taken for a certain period of time from a similar state. Based on this estimation, the improvement data generation unit 33F generates improvement data from the imaging data.

Next, the configuration of the mobile terminal 40 will be explained using FIG. 5. FIG. 5 is a block diagram showing a configuration example of the mobile terminal 40. As shown in FIG.
The mobile terminal 40 includes a terminal side communication section 41, a terminal storage section 42, a terminal processing section 43, a camera 44, and a terminal side display section 45.

The terminal side communication unit 41 is a communication interface that transmits and receives various data via the network 20. The various data include imaging data and data indicating comparison results. That is, the terminal side communication unit 41 receives various information from the beauty promotion device 30.

The terminal storage section 42 has a function of storing various control programs and various data necessary for the terminal processing section 43 to operate. The terminal storage unit 42 is realized by various storage media such as, for example, an HDD, an SSD, and a flash memory.
By executing the control program stored in the terminal storage section 42, the terminal processing section 43 may realize at least a part of each function that should be realized as the beauty promotion system 1.

The terminal processing section 43 is a computer that controls each section of the mobile terminal 40, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like. Note that the terminal processing section 43 is not limited to these examples, and may be any computer as long as it controls each section of the mobile terminal 40.

The terminal processing section 43 includes a receiving section 43A. The reception unit 43A receives the imaging data and comparison results transmitted from the beauty promotion device 30, and displays them on the terminal display unit 45.
The camera 44 can take an image when operated by the user 50. Instead of the imaging device 10 according to this embodiment, imaging data may be acquired by the camera 44 of the mobile terminal 40 and transmitted to the beauty promotion device 30.
The terminal-side display unit 45 is a monitor device that displays information indicating the comparison results processed by the beauty promotion device 30. The terminal-side display unit 45 can display the imaging data along with the comparison results.

Next, the contents of the processing by the beauty promotion system 1 will be explained using FIG. 6. FIG. 6 is a diagram showing a processing flow in the beauty promotion device 30.
As shown in FIG. 6, first, the face of the user 50 is captured by the imaging device 10 (imaging step: S501).
Next, the skin color evaluation unit 33A classifies the skin of the user 50 into a plurality of stages while referring to the evaluation table (skin color evaluation step: S502).

Next, the pigmentation identification unit 33B identifies a portion of the skin of the user 50 where an abnormal pigmentation area occurs (pigmentation identification step: S503). Then, the evaluation result display section 33C identifies spots and pigment spots by marking the pigment abnormal area (evaluation result display step: S504).

Next, the depth estimation unit 33D estimates the depth of the pigment abnormality part from the skin surface (depth estimation step: S505). At this time, the depth estimation unit 33D refers to pre-stored correspondence data between the color of the pigment abnormality and the depth from the skin surface.

Next, the measure proposing unit 33E proposes a measure to improve pigmentation to the user 50 (measure proposing step: S506).
Finally, the improvement data generation unit 33F generates and displays improvement data (improvement data generation step: S507). The improvement data generation unit 33F assumes a state in which the pigmentation in the pigment abnormality area has been improved with respect to the imaging data, changes the hue of the pigment abnormality area, and displays the changed hue to the user 50. Thereby, the user 50 can visually understand what kind of effect will be obtained when implementing the measures, and can be motivated to continue implementing the measures.

Next, another example of the processing result will be described using FIG. 7. FIG. 7 is a diagram showing another example of the processing results in the beauty promotion system 1. As shown in FIG. 7, even if there are more spots than in FIG. 4 described above, markings are applied to pigment abnormalities in a state where they are divided into a plurality of stages. The number of confirmed stains can then be evaluated for each applicable stage. Furthermore, by counting the number of stains and comparing them with an ideal value or an average value, the current state of the user 50 can be objectively displayed.

As explained above, according to the beauty promotion system 1 according to the present embodiment, the skin color evaluation unit 33A uses the image data obtained by capturing the skin of the user 50 to evaluate the hue of any part of the skin of the user 50. According to the value of , it is divided into multiple preset stages. Then, the pigmentation identifying section 33B identifies the position of the pigment abnormality on the skin of the user 50 based on the stages classified by the skin color evaluating section 33A. Thereby, the degree of pigmentation such as age spots and pigment spots can be quantitatively evaluated and used for beauty treatments.

Further, the evaluation result display section 33C displays the position of the location for each stage divided by the skin color evaluation section 33A on the imaging data, and marks the pigment abnormality region identified by the pigmentation identification section 33B. . This allows the user 50 to visually understand the evaluation results, thereby ensuring convenience for the user 50.

Furthermore, the evaluation result display unit 33C displays the position and marking for each stage of a plurality of image data captured at different times for the same user 50, and displays them side by side. Thereby, it is possible to visually grasp the change over time in the condition of the user's 50 skin.

Further, since the depth estimating unit 33D estimates the depth of the pigment abnormality part from the skin surface of the skin of the user 50 based on the hue value of the pigment abnormality part, measures for improving the pigment abnormality part are considered. This can provide material for making decisions.

Further, since the measure proposing unit 33E proposes a measure to promote improvement of pigmentation based on the depth of the pigment abnormality area, it is possible to propose an appropriate measure according to the condition of the pigment abnormality area.

In addition, the improvement data generation unit 33F assumes that the pigmentation in the pigment abnormality area has been improved based on the imaging data, changes the hue of the pigment abnormality area, and displays the changed hue to the user 50. By visually conveying the effects that can be expected from the measures taken, the user 50 can be motivated to continue taking the measures.

It goes without saying that the beauty promotion system 1 is not limited to the above embodiment, and may be realized by other methods. Various modifications will be described below.
For example, the control program of the above embodiment may be provided in a state stored in a computer-readable storage medium. The storage medium can store the control program in a "non-transitory tangible medium". The storage medium may include any suitable storage medium such as an HDD or an SDD, or any suitable combination of two or more thereof. A storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile. Note that the storage medium is not limited to these examples, and may be any device or medium as long as it can store the control program.

Moreover, the beauty promotion system 1 can realize each function shown in the embodiment by, for example, reading a control program stored in a storage medium and executing the read control program.
Further, the control program may be provided to the beauty promotion system 1 via any transmission medium (communication network, broadcast waves, etc.). The beauty promotion system 1 realizes the functions of the plurality of functional units shown in each embodiment by, for example, executing a control program downloaded via the Internet or the like.

Further, the control program may be implemented using, for example, a script language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5. Good too.

At least a part of the processing in the beauty promotion system 1 may be realized by cloud computing configured by one or more computers. Further, each functional unit of the beauty promotion system 1 may be realized by one or more circuits that realize the functions shown in the above embodiments, and it is assumed that the functions of the plurality of functional units are realized by one circuit. Good too.

Furthermore, in the embodiment described above, an example has been described in which the beauty promotion device 30 and the mobile terminal 40 are separate devices, but the present invention is not limited to such an embodiment. The mobile terminal 40 may have some or all of the functions of the beauty promotion device 30.

Further, although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included within the scope of the present disclosure. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically contradictory, and it is possible to combine multiple means, steps, etc. into one, or to divide them. . Further, the configurations shown in each embodiment may be combined as appropriate.

1 Beauty promotion system 10 Imaging device 20 Network 30 Beauty promotion device 40 Mobile terminal

Claims (2)

The computer is
A skin color evaluation step of dividing an arbitrary part of the user's skin into a plurality of preset stages for classifying the user's skin condition based on the hue value using image data obtained by capturing the user's skin. and,
a pigmentation identification step of identifying a part of the user's skin where a pigment abnormality including a stain due to pigmentation has occurred, based on the stage classified by the skin color evaluation step; and the pigmentation identification step. The beauty promotion comprises: estimating the depth of the pigment abnormality part from the skin surface in the user's skin based on the hue value of the pigment abnormality part specified by , and executing a selection step in which the operator selects the type of the pigment abnormality part. Method. to the computer,
A skin color evaluation function that uses image data of the user's skin to classify any part of the user's skin into multiple stages preset to classify the user's skin condition based on its hue value. and,
a pigmentation identification function that identifies a part of the user's skin where a pigment abnormality including spots due to pigmentation has occurred based on the stages classified by the skin color evaluation function; and the pigmentation identification function. Beauty promotion that realizes a selection function in which the operator selects the type of the pigment abnormality by estimating the depth from the skin surface of the pigment abnormality in the user's skin based on the hue value of the pigment abnormality identified by program.

Images