JP7173989B2 - Information processing device, program, and information processing method - Google Patents

Description

The present invention relates to an information processing device and a program.

In recent years, the number of consumers who choose cosmetics according to their skin conditions is increasing. In order to recommend products suitable for such consumers, it is effective to recommend cosmetics suitable for individual skin conditions. Therefore, there is a need for a technique for quantifying the skin condition of consumers and estimating the skin condition after using cosmetics.

For example, Japanese Patent No. 3727807 discloses a model for estimating the amounts of components (melanin and hemoglobin) in the skin from the colorimetric values of the skin.

Japanese Patent No. 3798550 discloses a model for estimating components in the skin (melanin, oxygenated hemoglobin, reduced hemoglobin, and apparent absorbance of the dermis) from the reflectance spectrum of the skin.

To encourage consumers to purchase cosmetics by presenting changes in ingredients in the skin after using recommended cosmetics and skin tone resulting from the changes to consumers who choose cosmetics. can be made In particular, in order to encourage consumers to purchase, it is effective to present a simulation image that reproduces the skin color corresponding to changes in skin ingredients after using recommended cosmetics.

However, in general, colorimetric values are represented by at least three parameters (for example, color component values in the XYZ color system). Since the model of Japanese Patent No. 3727807 has two types of parameters (melanin and hemoglobin) corresponding to components in the skin, colorimetric values cannot be calculated from the components in the skin. Therefore, the simulation image cannot be generated.

The model in Japanese Patent No. 3798550 does not define the relationship between colorimetric values and components in the skin. Therefore, the simulation image cannot be generated.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a simulation image that reproduces the skin color according to the components in the skin.

One aspect of the present invention is
means for acquiring colorimetric data including first to third color component values representing the pigment of the subject's skin;
A means for acquiring image data of the subject's skin from a camera,
comprising means for generating a display image to be displayed on a display based on the image data;
means for converting the first to third color component values, respectively, into first to third pigment parameters relating to substance quantities representing light characteristics of skin;
Means for changing at least one of the first dye parameter to the third dye parameter according to an operator's instruction;
means for converting the modified first to third dye parameters into the first to third color component values, respectively;
means for modifying the displayed image based on the converted first to third color component values;
It is an information processing device.

According to the present invention, it is possible to provide a simulation image in which the skin color is reproduced according to the components in the skin.

1 is a block diagram showing the configuration of an information processing system according to an embodiment; FIG. 2 is a block diagram showing the configuration of a client device 10; FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of the outline of this embodiment; 4 is a flowchart of information processing according to the embodiment; It is a figure which shows the outline|summary of the data handled in the information processing of this embodiment. It is a figure which shows the example of a screen displayed in the information processing of this embodiment. FIG. 11 is a diagram showing an example of a screen displayed in information processing of modification 2;

An embodiment of the present invention will be described in detail below with reference to the drawings. In the drawings for describing the embodiments, in principle, the same constituent elements are denoted by the same reference numerals, and repeated description thereof will be omitted.

(1) Configuration of information processing system The configuration of the information processing system will be described. FIG. 1 is a block diagram showing the configuration of the information processing system of this embodiment.

As shown in FIG. 1 , the information processing system 1 includes a client device 10 and a colorimetric device 20 .
The client device 10 and the colorimetric device 20 are connected to each other by wire or wirelessly.

The client device 10 is an example of an information processing device that executes information processing using the measurement result of the colorimetry device 20 . The client device 10 is, for example, a smart phone, a tablet terminal, or a personal computer.

The colorimetric device 20 is configured to measure skin color.

(1-1) Configuration of Client Apparatus The configuration of the client apparatus 10 will be described. FIG. 2 is a block diagram showing the configuration of the client device 10. As shown in FIG.

As shown in FIG. 2, the client device 10 includes a storage device 11, a processor 12, an input/output interface 13, a communication interface 14, a camera 15, and a display 16.

Storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of ROM (Read Only Memory), RAM (Random Access Memory), and storage (eg, flash memory or hard disk).

Programs include, for example, the following programs.
・OS (Operating System) program ・Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
・Databases referenced in information processing ・Data obtained by executing information processing (that is, execution results of information processing)

The processor 12 is configured to implement the functions of the client device 10 by activating programs stored in the storage device 11 . Processor 12 is an example of a computer.

The input/output interface 13 is configured to acquire a user's instruction from an input device connected to the client device 10 and output information to an output device connected to the client device 10 .
Input devices are, for example, keyboards, pointing devices, touch panels, or combinations thereof.

Communication interface 14 is configured to control communication between client device 10 and colorimetric device 20 .

The camera 15 is configured to acquire image data (for example, image data corresponding to an image of a subject subject to colorimetry by the colorimetry device 20).

Display 16 is configured to display display images generated by processor 12 . The display 16 is, for example, an LCD (Liquid-Crystal-Display) or an OEL (Organic-Electro-Luminescence) display.

(2) Overview of this embodiment An overview of this embodiment will be described. FIG. 3 is an explanatory diagram of the outline of this embodiment.

As shown in FIG. 3, the colorimetric device 20 obtains colorimetric data by measuring the skin color of the subject OBJ. The colorimetric data includes first to third color component values representing the skin pigment of the subject OBJ.

The client device 10 acquires the image data of the skin image of the subject OBJ and the colorimetric data obtained by the colorimetric device 20 .
The client device 10 displays a display image IMG0 of the skin of the subject OBJ based on the image data.

The client device 10 determines the light characteristics of the skin (for example, at least one of light reflection characteristics and light absorption characteristics) based on the first to third color component values included in the colorimetric data. Calculate the first to third dye parameters for the amount of substance to represent.

An operator of the client device 10 gives an instruction to the client device 10 to change at least one of the first to third dye parameters.
The client device 10 changes at least one of the first to third dye parameters according to the operator's instructions. The client device 10 converts the changed first to third colorant parameters into first to third color component values, respectively. Based on the converted first to third color component values, the client device 10 displays a display image IMG1 corresponding to the converted first to third color component values.

In this way, the operator of the client device 10 can change the skin pigment of the subject OBJ simply by giving the client device 10 an instruction to change at least one of the first to third pigment parameters. A display image IMG1 corresponding to is obtained.

(3) Information processing Information processing according to the present embodiment will be described. FIG. 4 is a flow chart of information processing in this embodiment. FIG. 5 is a diagram showing an overview of data handled in information processing according to this embodiment. FIG. 6 is a diagram showing an example of a screen displayed during information processing according to this embodiment.

As shown in FIG. 4, the client device 10 acquires colorimetric data (S100).
Specifically, the processor 12 acquires colorimetric data from the colorimetric device 20 . The colorimetric data includes first to third color component values representing the skin pigment of the subject OBJ. The first to third color component values are, for example, color component values (X0, Y0, Z0) of the XYZ color system (FIG. 5).
The processor 12 stores the acquired colorimetric data in the storage device 11 .

After step S100, the client device 10 acquires image data (S101).
Specifically, the camera 15 generates image data of the skin of the subject OBJ by imaging the skin of the subject OBJ. The image data includes, for example, color component values in the RGB color space (hereinafter referred to as "display color information").
Processor 12 stores image data generated by camera 15 in storage device 11 .

After step S101, the client device 10 generates a display image (S102).
Specifically, the processor 12 converts the first to third color component values (X0, Y0, Z0) included in the colorimetric data stored in the storage device 11 in step S101 into the RGB color space. Convert to color information (R0, G0, B0) (FIG. 5).

・ｋ_ａｘ、ｋ_ｂｘ、ｋ_ｃｘ：コラーゲン繊維の密度及び変性に基づく係数
・ｋ_ａｙ、ｋ_ｂｙ、ｋ_ｃｙ：皮膚のメラニン量に基づく係数
・ｋ_ａｚ、ｋ_ｂｚ、ｋ_ｃｚ：皮膚のヘモグロビン量に基づく係数
・ｊ_ａ、ｊ_ｂ、ｊ_ｃ：定数The processor 12 converts the first to third color component values (X0, Y0, Z0) included in the colorimetric data into the first to third colorant parameters (Pa0, Pb0, Pc0) (Fig. 5).

・_kax , _kbx , _kcx : Coefficients based on the density and denaturation of collagen fibers ・_kay , _{kby, kcy: coefficients based on the amount of melanin in the skin ・kaz, kbz, kcz : The amount of} hemoglobin in the skin Coefficients based on j _a , j _b , j _c : constant

The first pigment parameter Pa is a parameter relating to the light scattering level of the dermis of the subject OBJ's skin.
The second pigment parameter Pb is a parameter relating to the light absorption level of the epidermis of the skin of the subject OBJ.
The third pigment parameter Pc is a parameter relating to the light absorption level of the dermis of the skin of the subject OBJ.

An image (that is, an image of the skin of the subject OBJ) based on the image data stored in the storage device 11 in step S101 and the converted display color information (R0, G0, B0) of the RGB color space is generated ( Figure 5).
Processor 12 displays screen P100 ( FIG. 6 ) on display 16 .

As shown in FIG. 6, the screen P100 includes a display object A100, button objects B100a to B100c, and an image object IMG100.
The display object A100 displays the first to third color component values (X0, Y0, Z0) included in the colorimetric data. The values displayed in the display object A100 correspond to the display color information (R0, G0, B0) in the RGB color space converted in step S102.
The button object B100a is an object that receives a user instruction for changing the first pigment parameter Pa regarding the light scattering level of the dermis. The value of the first pigment parameter Pa is determined according to the position of the button object B100a.
The button object B100b is an object that receives a user instruction for changing the second pigment parameter Pb regarding the light absorption level of the epidermis. The value of the second pigment parameter Pb is determined according to the position of the button object B100b.
The button object B100c is an object that receives a user instruction to change the third pigment parameter Pc regarding the light absorption level of the dermis. The value of the third pigment parameter Pc is determined according to the position of the button object B100c.
Image object IMG100 displays the image of subject OBJ's skin generated in step S102. The color of the skin image of subject OBJ is determined by the display color information (R0, G0, B0) in the RGB color space converted in step S102.

After step S102, the client device 10 changes the parameters (S103).
Specifically, when the operator operates the button object B100a, the processor 12 changes the first pigment parameter Pa to a value according to the position of the button object B100a.
When the operator operates the button object B100b, the processor 12 changes the second pigment parameter Pb to a value according to the position of the button object B100b.
When the operator operates the button object B100c, the processor 12 changes the third pigment parameter Pc to a value according to the position of the button object B100c.
As a result, the values of the first to third dye parameters are corrected from (Pa0, Pb0, Pc0) to (Pa1, Pb1, Pc1) (FIG. 5).

After step S103, the client device 10 changes the display image (S104).
Specifically, using Equation 1, the processor 12 converts the first to third colorant parameters changed in step S103 to the first to third color component values (X1, Y1, Z1). Convert (Fig. 5).
The processor 12 converts the first to third color component values (X1, Y1, Z1) into display color information (R1, G1, B1) in the RGB color space (FIG. 5).
An image based on the image data stored in the storage device 11 in step S101 and the converted display color information (R1, G1, B1) in the RGB color space (that is, an image corresponding to the changed dye parameter) is generated. (Fig. 5).
The processor 12 displays the screen P102 (FIG. 6) on the display 16. FIG.

As shown in FIG. 6, the screen P102 includes a display object A102, button objects B100a to B100c, and an image object IMG102.
The display object A102 displays the first to third color component values (X1, Y1, Z1) corresponding to the pigment parameter changed in step S103. The values displayed in the display object A102 correspond to the display color information (R1, G1, B1) in the RGB color space converted in step S104.
Image object IMG102 corresponds to the dye parameters changed in step S103.

According to the present embodiment, three types of color component values (for example, color component values of the XYZ color system) and three types of pigment parameters (for example, skin reflection characteristics (for example, a parameter related to the light scattering level of the dermis) ) and the absorption characteristics of the skin (for example, a combination of a parameter related to the light absorption level of the epidermis and a parameter related to the light absorption level of the dermis)), and a model (formula 1) that defines the relationship between When the operator only gives an instruction to change at least one of the three types of dye parameters, color component values corresponding to the changed dye parameters are obtained (steps S103 and S104). As a result, it is possible to provide a simulation image (for example, the image object IMG102 in FIG. 6) in which the skin color is reproduced according to the components in the skin.

(4) Modification A modification of the present embodiment will be described. Note that the same description as in the present embodiment will be omitted.

(4-1) Modification 1
Modification 1 of the present embodiment will be described.

In Modified Example 1, in step S103 (FIG. 4), the operator issues an instruction to change at least one of the first to third color component values (X0, Y0, Z0) displayed on the display object A100. Then, processor 12 uses Equation 1 to calculate the first to third colorant parameters Pa to Pc corresponding to the modified first to third color component values (X0,Y0,Z0) do.

According to Modification 1, when the operator of the client device 10 gives an instruction to change at least one of the first to third color component values, the color of the skin of the subject OBJ is changed. A display object A102 of the first colorant parameter Pa to the third colorant parameter Pc can be provided.

(4-2) Modification 2
Modification 2 of this embodiment will be described. Modification 2 is a modification of the first to third pigment parameters relating to the amount of substances representing the absorption characteristics of skin. FIG. 7 is a diagram showing an example of a screen displayed in information processing according to modification 2. As shown in FIG.

The first pigment parameter Pa of Modification 2 is a parameter relating to the amount of oxygenated hemoglobin in the skin of subject OBJ.
The second pigment parameter Pb is a parameter relating to the light absorption level of the epidermis of the skin of the subject OBJ.
The third pigment parameter Pc is a parameter relating to the amount of reduced hemoglobin in the skin of subject OBJ.

Processor 12 displays screen P110 (FIG. 7) on display 16 in step S102 (FIG. 4).

As shown in FIG. 7, screen P110 includes display object A100, button objects B110a to B110c, and image object IMG100.
The button object B110a is an object that receives a user instruction to change the first pigment parameter Pa regarding the amount of oxyhemoglobin. The value of the first pigment parameter Pa is determined according to the position of the button object B110a.
The button object B110b is an object that receives a user instruction for changing the second pigment parameter Pb regarding the light absorption level of the epidermis. The value of the second pigment parameter Pb is determined according to the position of the button object B110b.
The button object B110c is an object that receives a user instruction to change the third pigment parameter Pc regarding the amount of reduced hemoglobin. The value of the third pigment parameter Pc is determined according to the position of the button object B110c.

According to Modification 2, three types of color component values (eg, XYZ color system color component values) and three types of pigment parameters (eg, parameters related to the amount of oxyhemoglobin, parameters related to the light absorption level of the epidermis, and , a parameter related to the amount of reduced hemoglobin), and a model in which the relationship between the . As a result, when the operator gives an instruction to change at least one of the three types of dye parameters, the color component values corresponding to the changed dye parameters are obtained (steps S103 and S104). As a result, it is possible to provide a simulation image (for example, the image object IMG102 in FIG. 7) in which the skin color is reproduced according to the components in the skin.

(5) Summary of this embodiment This embodiment will be summarized.

The first aspect of this embodiment is
means for acquiring colorimetric data including first to third color component values representing skin pigments of subject OBJ (for example, processor 12 for executing the process of step S100);
A means for acquiring image data of the skin of the subject OBJ from the camera 15 (for example, the processor 12 that executes the processing of step S101);
Based on the image data, means for generating a display image to be displayed on the display 16 (for example, the processor 12 that executes the process of step S102);
The first color component value to the third color component value are respectively a first pigment parameter Pa and a second pigment parameter Pb relating to the light characteristics of skin (for example, at least one of light reflection characteristics or light absorption characteristics). , and means for converting to the third dye parameter Pc (for example, a processor 12 that performs the process of step S102),
According to an operator's instruction, means for changing at least one of the first dye parameter Pa to the third dye parameter Pc (for example, a processor 12 that executes the processing of step S103);
Means for converting the changed first pigment parameter Pa to third pigment parameter Pc into first to third color component values, respectively (for example, a processor 12 for executing the process of step S104);
Means for changing the displayed image based on the converted first to third color component values (for example, a processor 12 for executing the processing of step S104);
An information processing device (for example, the client device 10).

According to the first aspect, three types of color component values (first color component value to third color component value) are combined with three types of pigment parameters (first pigment parameter Pa to third A model is used that transforms into three dye parameters Pc). Only by giving an instruction to change at least one of the three types of dye parameters by the operator, color component values corresponding to the changed dye parameters are obtained. As a result, it is possible to provide a simulation image in which the skin color is reproduced according to the components in the skin.

A second aspect of the present embodiment is
The first pigment parameter is a parameter related to the light scattering level of the dermis,
The second pigment parameter is a parameter related to the light absorption level of the epidermis,
The third pigment parameter is a parameter related to the light absorption level of the dermis,
It is an information processing device.

According to the second aspect, the operator selects at least one of the reflective properties of the skin (e.g., the light scattering level of the dermis) and the absorption properties of the skin (e.g., the light absorption level of the epidermis and the light absorption level of the dermis). By simply giving an instruction to change one, the color component values corresponding to the changed pigment parameters can be obtained.

The third aspect of this embodiment is
The first pigment parameter Pa includes a factor based on the density and denaturation of collagen fibers,
It is an information processing device.

According to the third aspect, when the operator gives an instruction to change the optical properties, a color component value corresponding to the density and degeneration of collagen fibers is obtained.

The fourth aspect of this embodiment is
The first pigment parameter is a parameter related to the amount of oxygenated hemoglobin in the skin,
The second pigment parameter is a parameter related to the light absorption level of the epidermis,
The third pigment parameter is an information processing device that is a parameter related to the amount of reduced hemoglobin in the skin.

According to the fourth aspect, the operator instructs to change the absorption characteristics of the skin (for example, at least one of the amount of oxygenated hemoglobin in the skin, the light absorption level of the epidermis, and the amount of reduced hemoglobin in the skin). By simply giving it, we get the color component values corresponding to the changed colorant parameters.

The fifth aspect of this embodiment is
The first to third color component values each include an X color component value, a Y color component value, and a Z color component value of the XYZ color system,
It is an information processing device.

According to the fifth aspect, when the operator gives an instruction to change the light characteristics, the X color component value, the Y color component value, and the Z color component value of the XYZ color system corresponding to the instruction are obtained.

A sixth aspect of the present embodiment is
The modifying means modifies the display image by converting the X color component values, the Y color component values, and the Z color component values into image signals in the RGB color space;
It is an information processing device.

According to the sixth aspect, when the operator gives an instruction to change the optical characteristics, a display image corresponding to the instruction can be presented.

The seventh aspect of this embodiment is
The means for acquiring the colorimetric data acquires the colorimetric data from a colorimetric device 20 that measures the skin color of the subject OBJ.
It is an information processing device.

According to the seventh aspect, the color component values corresponding to the operator's instruction for the colorimetric data acquired from the colorimetric device 20 are obtained.

The eighth aspect of the present embodiment is
It is a program for causing a computer (for example, processor 12) to function as each means described above.

(6) Other modified examples

The storage device 11 may be connected to the client device 10 via a network (not shown).

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above embodiments. Also, the above embodiments can be modified and modified in various ways without departing from the gist of the present invention. Also, the above embodiments and modifications can be combined.

Reference Signs List 1: information processing system 10: client device 11: storage device 12: processor 13: input/output interface 14: communication interface 15: camera 16: display 20: colorimetric device

Claims (10)

means for acquiring colorimetric data including first to third color component values representing the pigment of the subject's skin;
A means for acquiring image data of the subject's skin from a camera,
comprising means for generating a display image to be displayed on a display based on the image data;
means for converting the first to third color component values, respectively, into first to third pigment parameters relating to substance quantities representing light characteristics of skin;
When the operator gives an instruction to change at least one of the first dye parameter to the third dye parameter, the means for altering at least one of said first dye parameter through said third dye parameter according to instructions;
means for converting the modified first to third dye parameters into the first to third color component values, respectively;
means for changing the display image based on the converted first to third color component values;,
When the operator gives an instruction to change at least one of the first to third color component values, at least one of the first to third color component values is changed according to the instruction. with means for modifying
means for converting the modified first to third color component values into the first to third dye parameters, respectively;
means for changing the display image based on the converted first to third dye parameters;Ru
Information processing equipment. The first pigment parameter is a parameter related to the light scattering level of the dermis,
The second pigment parameter is a parameter related to the light absorption level of the epidermis,
The third pigment parameter is a parameter related to the light absorption level of the dermis,
The information processing device according to claim 1 . the first pigment parameter includes a factor based on collagen fiber density and denaturation;
The information processing apparatus according to claim 2. The first pigment parameter is a parameter related to the amount of oxygenated hemoglobin in the skin,
The second pigment parameter is a parameter related to the light absorption level of the epidermis,
The information processing apparatus according to claim 1, wherein said third pigment parameter is a parameter relating to the amount of reduced hemoglobin in skin. The first to third color component values each include an X color component value, a Y color component value, and a Z color component value of an XYZ color system. 1. The information processing device according to claim 1. 6. The changing means according to claim 5, wherein said changing means changes said display image by converting said X color component value, said Y color component value, and said Z color component value into an image signal of RGB color space. information processing equipment. 7. The information processing apparatus according to claim 1, wherein said means for acquiring said colorimetric data acquires said colorimetric data from a colorimetric device for measuring skin of said subject. comprising means for displaying dye parameters and color component values;
The means for displaying
when the operator gives an instruction to change at least one of the first to third dye parameters, displaying the converted first to third color component values;
when the operator gives an instruction to change at least one of the first to third color component values, displaying the converted first to third color component parameters;
The information processing apparatus according to any one of claims 1 to 7. A program for causing a computer to function as each means according to any one of claims 1 to 8 . An information processing method for generating an image that reproduces the skin color of a subject using a computer,
obtaining colorimetric data including first to third color component values representing the pigment of the subject's skin;
Acquiring image data of the subject's skin from a camera;
generating a display image to be displayed on a display based on the image data;
converting the first to third color component values into first to third colorant parameters, respectively, relating to material quantities representing light characteristics of skin;
When the operator gives an instruction to change at least one of the first dye parameter to the third dye parameter, changing at least one of the first dye parameter to the third dye parameter according to the instruction. with
converting the modified first to third colorant parameters into the first to third color component values, respectively;
modifying the display image based on the converted first to third color component values;
When the operator gives an instruction to change at least one of the first to third color component values, at least one of the first to third color component values is changed according to the instruction. a step of changing
converting the modified first to third color component values into the first to third dye parameters, respectively;
modifying the displayed image based on the transformed first to third dye parameters;
Information processing methods.

Images