JP6323097B2 - Color measuring device, color measuring system, color measuring method, and program - Google Patents

Description

  The present invention relates to a color measurement device, a color measurement system including the color measurement device, a color measurement method, and a program.

  Conventionally, an attempt has been made to quantitatively grasp the skin condition and use it for skin care and makeup. For example, an apparatus that proposes a makeup cosmetic that produces the most natural makeup skin color based on user's skin color information is disclosed (see, for example, Patent Document 1). However, in the conventional invention disclosed in Patent Document 1, based on a large amount of data collected in advance, the makeup cosmetic that becomes an average makeup skin color is only predicted and proposed, and the color desired by the user is proposed. It did not correspond to the taste or ideal image, and was not enough to satisfy the user's taste.

  On the other hand, XYZ cameras that generate captured data as tristimulus values in the XYZ color system have been developed. This XYZ color system is a standard color system approved by the International Commission on Illumination (CIE) and is considered to be a color system close to human vision. Various attempts have been made to reproduce colors using accurate color information expressed in the XYZ color system.

  The present invention has been made in view of the above circumstances, and effectively utilizes color information that is closer to human vision and more accurately measured, and provides notification or proposal of color that matches the user's wishes and images. The purpose is to do.

In order to achieve the above object, a color measuring apparatus according to the present application obtains measurement color information of a subject from storage means in which predetermined color information is stored as reference color information and input means for measuring the color of the subject. Based on the acquired color detection means for detecting the first color of the subject from the measured color information and the color information and reference color information of the first color, the second color to be added to the subject is calculated. Color calculation means , wherein the reference color information is image information including a plurality of color information, and the color calculation means is one color information corresponding to the first color among the plurality of color information. Calculating the difference between the first color information and the color information of the first color, changing other color information included in the plurality of color information in accordance with the difference, and calculating the second color. Features.

  According to the present invention, it is possible to effectively use color information that is closer to human vision and more accurately measured, and perform notification or suggestion of a color that matches a user's desire or image.

It is a block diagram which shows the hardware structural example of the color measurement system provided with the color measurement apparatus which concerns on Example 1 of this application. FIG. 3 is a functional block diagram for explaining functions of the color measuring apparatus according to the first embodiment. FIG. 6 is an explanatory diagram for explaining a procedure for changing a characteristic color based on skin color information of ideal color information and skin color information of measurement color information in the color measurement device according to the first embodiment. It is explanatory drawing for demonstrating the procedure which calculates a difference and changes a characteristic color using a HSL color system, a L * a * b * color system, and a L * C * h color system. It is explanatory drawing for demonstrating the procedure which suppresses the value of H and L, and changes a characteristic color, when a difference is calculated using an HSL color system. 3 is a flowchart illustrating a flow of color measurement processing performed by the color measurement apparatus according to the first embodiment. FIG. 10 is a functional block diagram for explaining functions of a color measuring apparatus according to a fourth embodiment. 10 is a flowchart illustrating a flow of color measurement processing performed by the color measurement apparatus according to the fourth embodiment. FIG. 10 is a functional block diagram for explaining functions of a color measuring apparatus according to a fifth embodiment. 10 is a flowchart illustrating a flow of a color measurement process performed by a color measurement apparatus according to a fifth embodiment. It is a schematic block diagram which shows an example of the XYZ camera used by each Example, (a) shows a front view, (b) shows a side view. It is a schematic diagram for demonstrating the principle of the XYZ camera shown in FIG. It is a characteristic view which shows the spectral transmittance of each color filter in case the incident angle of a light ray is 0 degree | times in the XYZ camera shown in FIG. It is a schematic diagram for demonstrating the principle of the camera using a prism and three image sensors. It is a characteristic view which shows the spectral sensitivity of each optical system in case the incident angle of a light ray is 0 degree | times in the camera using a prism and four image sensors. It is explanatory drawing for demonstrating the camera which combined the optical image duplication element and the 4-part dividing filter mounted on the image pick-up element, Comprising: (a) shows the schematic diagram for demonstrating the principle of a camera, (b ) Shows a schematic diagram of a quadrant filter.

  Hereinafter, an example of a color measurement system including a color measurement device according to the present application will be described with reference to the drawings. In the color measurement system according to the first embodiment of the present application, based on accurate measurement color information measured by the XYZ camera, the second (point) (feature) such as the color of the lipstick added to the user (human body) as the subject. The color (hereinafter referred to as “characteristic color”) is calculated. At that time, the color of the characteristic color is adjusted (changed) according to the difference between the ideal skin color and the skin color of the user (human body) as the subject. Thus, when the entire face is viewed, a characteristic color that gives an impression close to an ideal makeup state is notified or proposed (hereinafter simply referred to as “notification”).

  The reason why such characteristic colors are calculated will be described. Characteristic colors such as lipstick, eye shadow, and blusher are applied to a part of the skin as a base, so that the surroundings are surrounded by the skin color. For this reason, color contrast occurs between the characteristic color and the skin color, and even if the same characteristic color is applied, the atmosphere may differ depending on the skin color. Therefore, as in the first embodiment, by adjusting the characteristic color corresponding to the user's skin color, makeup can be performed without changing the overall atmosphere even if the skin color is different. In the first embodiment, cosmetics as a color material are calculated based on the calculated feature color and notified to the user. The “coloring material” is a material that adds color to an object, that is, a coloring material, and generally includes dyes, pigments, paints, inks, paints, and the like. Cosmetics (cosmetics) such as eye shadow, lipstick and blusher are also included in the colorant.

  In view of the above, as shown in FIG. 1, a color measurement system 500 according to Example 1 of the present application includes a color measurement device (color measurement unit) 100, an XYZ camera 200 as an input unit, an output unit 300, It comprises and comprises. The output unit 300 includes a display (display unit) 301 such as a liquid crystal monitor and a printer (image forming unit) 302.

  The color measurement system 500 according to the first embodiment includes a database (HDD 105 in FIG. 1) in the color measurement apparatus 100, and stores a plurality of sample images with various makeups. These sample images are displayed on the display 301 so that the user can select an ideal makeup image. For this reason, the color measurement system 500 includes database access means, display information generation means, and the like. Furthermore, input means such as a mouse, keyboard, touch panel, etc. are provided. By operating these, the user can give various instructions to the color measurement process, specify an ideal makeup state image, specify an area, etc. Can be done.

  The color measurement system 500 according to the first embodiment includes, for example, a personal computer (hereinafter referred to as “PC”) including a XYZ camera 200 as an input unit, a display 301 as an output unit 300, a printer 302, a CPU, a memory, and the like. This can be realized by mounting the color measuring apparatus 100 on the above-mentioned items. The color measurement system 500 of the present application is not limited to being realized on a PC, and the color measurement system 500 is realized by the XYZ camera 200 itself by incorporating a color measurement device into an IC or the like built in the XYZ camera 200. You can also.

  In addition to realizing the color measurement system 500 with a PC, an XYZ camera 200, etc., a mobile phone having an imaging function and a display function, a personal digital assistant called PDA (personal data assistant), and the like It can also be realized by various information devices including mobile terminal devices such as so-called smartphones and tablet terminals including the above functions. In this case, the color measuring device 100 can be provided on the cloud side using a cloud computing service. Then, an image captured by an XYZ camera incorporated in an information device such as a portable terminal device or an XYZ camera connected to the information device is transmitted to the cloud, and color information is calculated by the color measuring device on the cloud side. The calculated color information may be transmitted from the cloud side to the information device side and output to an output unit such as a liquid crystal monitor of the information device or a printer connected to the information device.

  Hereinafter, each part of the color measurement system 500 will be described in detail. The color measuring apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an HDD interface (hereinafter referred to as “HDD I / I”). F ”) 104, HDD (Hard Disk Drive) 105, semiconductor memory 106, external interface (hereinafter referred to as“ external I / F ”) 107, and output interface (hereinafter referred to as“ output I / F ”). ) 108 and the like.

  The CPU 101 is connected to each part in the color measuring apparatus 100 via the data bus 109. The CPU 101 uses the RAM 103 as a work memory in accordance with a color measurement program stored in advance in the ROM 102 to control the overall operation of the color measurement apparatus 100 and realize various functions.

  The HDD I / F 104 is an interface for accessing the HDD 105 and reading stored information or writing information. The external I / F 107 is an interface that enables communication between the color measuring apparatus 100 and an external device such as the XYZ camera 200. The output I / F 108 is an interface for outputting a color measurement result (color information) in the color measuring apparatus 100 to the output unit 300.

  The HDD 105 stores various types of information such as reference color information used as a reference when calculating colors. As reference color information, “ideal color information” is stored in advance in an ideal color information storage unit 105 a (see FIG. 2) provided in the HDD 105. This “ideal color information” is information on the skin color and characteristic color (lipstick, eye shadow, blush, etc.) of the makeup state desired by the user (ideal), and the color information on the makeup state desired by the user. This includes face image information (reference face image information). In many cases, the makeup state is a state in which the appearance of the skin is adjusted with a foundation or the like, and a characteristic color is added to the skin with a point makeup product such as lipstick, eye shadow, or blusher. In the first embodiment, a plurality of pieces of face image information of the entire face subjected to the ideal makeup state are stored in the ideal color information storage unit 105a.

  The ideal color information may be expressed in any color system as long as the color is quantitatively expressed. For example, a color system based on three attributes consisting of hue, saturation, and brightness (or luminance), for example, L * a * b * color system, HSL (sometimes called HLS, HSI) color system. The L * C * h color system is desirable. The XYZ camera 200 may be an XYZ tristimulus value obtained by measuring an actual person such as a model or an ideal makeup state of a replica, magazine, catalog, or the like, and provide more accurate ideal color information. Can do. Further, the ideal color information is not limited to the information stored in the HDD 105 in advance. For example, the user can select from a plurality of images with various makeup with different skin colors and characteristic colors provided by the color measurement application software, or download from the Internet, etc. It can also be configured. In this case, by storing these pieces of image information in the ideal color information storage unit 105a of the HDD 105, the color measuring apparatus 100 can acquire the ideal color information from the ideal color information storage unit 105a. Alternatively, without being stored in the HDD 105, it can be temporarily stored (stored) in the RAM 103 or the like as one of the storage units so that each unit of the color measuring apparatus 100 can be referred to.

  In the case of image information acquired by downloading or the like, the ideal color information is an RGB value included in the digital image, and this is converted to L * a * b * using a conventionally known appropriate calculation tool. It can be converted into and used. In addition, in order to save the trouble of converting the RGB values into the L * a * b * color system, etc., a digital image is displayed or printed out on the display, and the image on the display or the printed out image is displayed again by the XYZ camera 200. You may measure. By using the measurement information, more accurate ideal color information close to the user's appearance can be obtained.

  Also, the HDD 105 stores information (color material information) that associates cosmetics with the colors of the cosmetics themselves. Further, the HDD 105 can store color information of the characteristic color calculated by the color measuring apparatus 100 and color information of the entire face including the user's skin color information, and can store the information as a history. Furthermore, various parameters for outputting reference color information, calculated characteristic color information, and the like to the output unit 300, an image template, a printing template, and other information can be stored.

  The semiconductor memory 106 stores measurement image information of the face of the user who is the subject measured by the XYZ camera 200. This measurement image information is written into the semiconductor memory 106 by the XYZ camera 200 via the external I / F 107. The semiconductor memory 106 is provided in the color measuring apparatus 100 in the first embodiment. The XYZ camera 200 stores measurement image information in the semiconductor memory 106 via the external I / F 107. The semiconductor memory 106 may be built in the XYZ camera 200, and the color measurement apparatus 100 reads the measurement image information stored in the semiconductor memory 106 of the XYZ camera 200 via the external I / F 107. It can also be.

  The XYZ camera 200 is used as input means for measuring the face of the user who is the subject and inputting the measured image information to the color measuring apparatus 100. The XYZ camera 200 is a camera that uses an optical bandpass filter corresponding to a color matching function defined in the XYZ color system by the International Commission on Illumination (CIE). An example of the XYZ camera 200 will be described with reference to FIGS. As shown in FIG. 10B, the XYZ camera 200 used in the first embodiment includes an imaging unit 201 that acquires spectral information from an object, and a plurality of types of spectral images based on the spectral information acquired by the imaging unit 201. FPGA (Field-Programmable Gate Array) 202 as a spectral image generating means for generating The FPGA 202 also functions as a correction unit that performs a correction operation for correcting the electrical signal for each light receiving position of the light receiving unit (the light receiving element array 211 in FIG. 10B) in order to measure the color.

  The imaging unit 201 includes a lens module 203 and a camera unit 204. The lens module 203 includes a lens barrel 205, a main lens 206 as an optical system provided in the lens barrel 205, a color filter 207, and a lens 208. The camera unit 204 includes a microlens array 210, a light receiving element array 211, and an FPGA 202 therein. Further, as shown in FIG. 10A, a plurality of LEDs 212 as light sources are provided at the distal end portion of the lens barrel 205 in an embedded state at equal intervals in the circumferential direction.

  The color filter 207 is disposed near the stop S (see FIG. 11) of the main lens 206. The color filter 207 is an optical bandpass filter corresponding to a tristimulus value of color having a spectral transmittance based on the color matching function of the XYZ color system. The color filter 207 includes a plurality (three in this case) of color filters 207a, 207b, and 207c having different spectral transmittances based on the color matching function of the XYZ color system. FIG. 12 shows spectral transmittances of the color filters 207a, 207b, and 207c when the incident angle of the light beam is 0 degree. Since it has the spectral transmittance as shown in FIG. 12, the X value of the tristimulus value can be obtained by detecting the light beam transmitted through the color filter 207a by the light receiving element array 211. Similarly, by detecting the light beams transmitted through the color filters 207b and 207c by the light receiving element array 211, the tristimulus values Y and Z can be acquired, respectively.

  The light receiving element array 211 functions as a light receiving unit that converts optical information collected by the main lens 206, the lens 208, and the microlens array 210 into an electrical signal. The microlens array 210 is disposed between the main lens 206 and the light receiving element array 211, and has a configuration in which a plurality of microlenses 210a are disposed substantially parallel to the two-dimensional plane direction of the main lens 206.

  The principle of the XYZ camera 200 configured as described above will be described with reference to FIG. Of the light emitted from the object O, the light beam that enters the aperture of the main lens 206 and passes through the stop S is the object of measurement. The light beam incident on the main lens 206 is a collection of innumerable light beams, and each light beam passes through different positions of the stop S of the main lens 206. In the XYZ camera 200 according to the first embodiment, since the three color filters 207a, 207b, and 207c are arranged at the stop S position of the main lens 206, each light beam passes through three filters having different spectral transmittances. Become.

  At this time, the angle of the light ray incident on the filter surface of the color filter 207 varies depending on the object height. This can also be seen from the fact that the chief rays of the light beams emitted from the points on the object O represented by P and Q in FIG. 11 pass through the stop surface of the main lens 206 at different angles. The light rays that have passed through the color filter 207 once form an image in the vicinity of the microlens array 210, but then reach the different positions of the light receiving element array 211 by the microlens array 210, respectively. That is, since the position (light receiving position) of the sensor surface of the light receiving element array 211 corresponds to the filter surface through which the light beam has passed, the light emitted from one point of the object O is converted into tristimulus values X, Y, and Z in terms of wavelength. The decomposed value can be measured.

  However, since the spectral transmittance of the color filter 207 has an incident angle dependency, if the output of the light receiving element array 211 is simply used, an accurate three-dimensional surface including the off-axis may be used even if it is on the optical axis. Stimulus values X, Y and Z cannot be measured.

  Therefore, in the XYZ camera 200 of the first embodiment, the FPGA 202 corrects each light receiving position using the reference value and the value calculated from the output value from the measuring device, and the accurate tristimulus value on the two-dimensional surface. Is configured to get. This correction is generally performed using a technique called multiple regression analysis. In the multiple regression analysis, explanatory variables and objective variables are prepared in advance, and correction calculation is performed using a regression matrix obtained from them.

  By using the XYZ camera 200 as described above, the color and spectrum of the two-dimensional surface of the object can be accurately measured, and more accurate color information can be acquired. In addition, the tristimulus values of X, Y, and Z can be acquired simultaneously by one shooting without the need for filter replacement or the like.

  The input means is not limited to the XYZ camera 200 having the above-described configuration, and an XYZ camera having another configuration can be used. Further, the present invention is not limited to an XYZ camera as long as accurate color information can be acquired. As another example of the XYZ camera, a camera using a prism and a plurality of image sensors, a camera combining an optical image duplicating element, and a division filter mounted on the image sensor can be used.

  An example of a camera using a prism and a plurality of image sensors is a camera using a prism and three image sensors. In this camera, the spectral sensitivities of the three optical systems correspond to x, y, z color matching functions. In addition, a camera using a prism and four image sensors can be cited. In this camera, the spectral sensitivities of the four optical systems correspond to x1, x2, y, z color matching functions in which x is divided into two.

  The principle of a camera using a prism and three image sensors will be described with reference to FIG. In this camera, the light beam that has passed through the lens 206 is divided into three optical paths by the three prisms 212a, 212b, and 212c, and the respective images are received by the three light receiving sensors 213a, 213b, and 213c. Is converted into an electrical signal. The camera is designed so that the spectral sensitivities of these three optical systems are the same as those shown in FIG. Also in this case, a more accurate tristimulus value can be obtained by correcting the tristimulus value using the arithmetic processing described in the XYZ camera 200 of the first embodiment.

  Next, a camera using a prism and four image sensors will be described with reference to FIG. The principle of this camera is basically the same as that of the camera using the prism and three image sensors described with reference to FIG. 13, but the prism and the light receiving sensor are added to further increase the optical path. Another branching is a four-plate system. The spectral sensitivities of the four optical systems of this camera are shown in FIG. As shown in FIG. 14, since the color matching function of x has two peaks, it is possible to create a camera easily and at low cost by dividing x into two of x1 and x2 to be unimodal. Is possible.

  In addition, as a camera that combines an optical image duplicating element and a division filter mounted on the image sensor, three optical images are duplicated by the optical image duplicating element, and x, y, and z color matching functions are supported on the image sensor. And a camera equipped with the three-divided filter. Further, there is a camera including a four-division filter corresponding to x1, x2, y, z, etc. color functions in which four optical images are duplicated by an optical image duplicating element and x is divided into two on the imaging element.

  The principle of a camera that combines an optical image duplicating element and a four-divided filter mounted on the imaging element will be described with reference to FIG. As shown in FIG. 15A, the camera includes a main lens 206, a field lens 214 and a microlens array 210, a filter 216, and an image sensor 215 as optical image duplicating elements. The optical image duplicating element composed of the field lens 214 and the microlens array 210 can duplicate almost the same optical image. Further, four optical images are generated by setting the small lenses constituting the microlens array 210 to 2 × 2. A spectral sensitivity filter 216 similar to that shown in FIG. 14 is provided immediately before the image sensor 215. As shown in FIG. 15B, the filter 216 is divided into four regions 216a, 216b, 216c, and 216d. Using such a filter 216, images corresponding to x1, x2, y, and z are taken. In this case as well, a more accurate tristimulus value can be obtained by correcting the tristimulus value using the arithmetic processing described in the XYZ camera 200 of the first embodiment. The principle of a camera equipped with a three-divided filter is basically the same as the principle of a camera equipped with a four-divided filter, but there are three optical images to be duplicated, and the filter is also divided into x , Y and z are taken.

  Next, functions of the color measuring apparatus 100 will be described with reference to the functional block diagram of FIG. As shown in FIG. 2, the color measurement apparatus 100 includes an ideal color information detection unit 10, a measurement color information acquisition unit 11 as a color detection unit, a color calculation unit 12, and a characteristic color output unit 16. Configure. The color calculation unit 12 includes a color difference calculation unit 13, a characteristic color change unit 14 as a color change unit, and a cosmetic calculation unit 15 as a color material calculation unit.

  The ideal color information detection means 10 acquires (detects) color information (ideal color information) of an ideal makeup state designated by the user. Therefore, the ideal color information detection unit 10 accesses the ideal color information storage unit 105a of the HDD 105, reads ideal color information designated by the user, and detects a face portion (face region) from the ideal color information. For example, feature parts such as eyebrows, eyes, and mouth can be detected based on information from the face recognition system or the like, image feature amounts, and the like, and face parts (face areas) can be detected. The user can also specify a face area by operating the mouse while confirming an image on the display 301. Moreover, the skin part can also be detected by the color. In this case, it is possible to make use of the fact that the color changes greatly between the face and its surroundings (for example, hair, clothes, background, etc.), or to predetermine a color range to be determined as the skin color.

  The ideal color information detection unit 10 further detects skin color information and feature color information from the face portion (face region) of the ideal color information. As the skin color information and the feature color information, the ideal color information detection unit 10 uses L *, a *, and L *, a *, when the image read from the ideal color information storage unit 105a is, for example, an L * a * b * color system. b * value is output, L *, C *, h values are detected in the case of L * C * h color system, and X, Y, Z tristimulus values are detected in the case of XYZ color system. To detect.

  The detection of the color information by the ideal color information detection means 10 is not limited to the above, and is based on, for example, feature color information registered in advance corresponding to the ideal color information designated by the user. The characteristic color information may be detected (determined).

  In order to detect the skin color, the ideal color information detection means 10 first determines a skin color area from which the skin color is detected from the face portion (face area). Since this skin color area is the most part of the face excluding the feature color area for detecting the feature color, the largest color value among all the color values of the detected face part (face area) is the skin color of the ideal color information. It can be employed as information (first reference color information). However, in many cases, the captured image has a shadow on the face due to the influence of illumination. In that case, it is detected that the brightness difference in the face part (face area) is more than a certain level, and the skin color near the feature color and the skin color under the cheek where the influence of lighting is relatively small are ideal colors. It can be employed as skin color information.

  Next, feature color detection will be described. The ideal color information detection unit 10 determines a characteristic color area from the face portion (face area). In the present specification, characteristic colors different from the skin color such as lipstick, eye shadow and blusher are called “characteristic colors”. It can be predicted where these feature colors exist in the face portion (face region). For example, eye shadow exists between the eyebrows and eyes, blusher exists beside the nose, and lipstick exists in the mouth itself. Therefore, a feature color region can be determined by detecting a face part (eyebrow, eye, nose, mouth, etc.) in the face part (face region). Characteristic color information (other reference color information) such as lipstick, eye shadow, blusher, etc. is detected from this characteristic color region. Note that the procedure for determining the characteristic color is not limited to this, and the determination may be made based on the hue as another different determination procedure. For example, if a part different from the skin color is detected more than a certain area, it is determined that the color of the part is a characteristic color and can be adopted as characteristic color information.

  The measurement color information acquisition unit 11 acquires color information of the user who is the subject. Specifically, the semiconductor memory 106 is accessed, measurement image information measured by the XYZ camera 200 is read, and color information (hereinafter referred to as “measurement color information”) is acquired for each area. Here, “each area” is, for example, the area of each pixel, but may be the area of each block when the measurement image is divided into a plurality of blocks. In the latter case, a value obtained by averaging the colors of all the pixels in each block is used as measurement color information of the block (area). The measurement color information for each area acquired by the measurement color information acquisition unit 11 is XYZ tristimulus values.

  Further, the measurement color information acquisition unit 11 acquires measurement color information of the skin color (first color) of the user (subject) based on the acquired measurement color information for each area. First, a face portion (face region) is detected from ideal color information. For example, as with the ideal color information detection unit 10, a face part (face area) of a user is detected using a face recognition system or the like, and the largest of all color values of the detected face part (face area). The color value is adopted as skin color information of the measurement color information. Alternatively, the skin color near the characteristic color or the skin color under the cheek may be adopted as the skin color information of the measurement color information.

  When the face color is measured with the XYZ camera 200, it is desirable for the user to perform at least base makeup. Here, the base makeup is to prepare the appearance of the skin using a foundation or the like, and is for making a foundation for adding a characteristic color. In order to prevent the user's skin color from changing depending on the presence or absence of the base makeup and the color to be used, and changing the atmosphere when the characteristic color is applied, it is preferable to perform the base makeup in advance.

  The color difference calculation means 13 of the color calculation means 12 calculates the difference between the skin color information of the ideal color information detected by the ideal color information detection means 10 and the skin color information of the measurement color information measured by the measurement color information acquisition means 11. For example, when a vector between two coordinates of the skin color information of the ideal color information and the skin color information of the measurement color information in the L * a * b * color space is calculated, the direction of the vector between the two coordinates is determined by the measurement color information. It shows what color the skin color information is compared with the skin color information of the ideal information. Further, it is indicated that the color difference between the skin color information of the measurement color information and the skin color information of the ideal color information is larger as the distance between the two coordinates is larger. Note that the difference calculation of the present application is not limited to the one based on the L * a * b * color space. For example, a difference between two coordinates may be taken in a color space based on hue, saturation, and brightness, such as an HSL color space.

  The characteristic color changing unit 14 of the color calculating unit 12 calculates a characteristic color (second color) to be notified to the user. Therefore, based on the difference between the skin color information of the ideal color information calculated by the color difference calculation unit 13 and the skin color information of the measurement color information, the color of the feature color information proposed to the user is changed, and the feature color to be notified to the user is changed. calculate. This feature color is called a changed feature color. At this time, each characteristic color is changed so as to correspond to the difference between the skin color information of the ideal color information and the skin color information of the measurement color information. For example, the feature color information is changed in accordance with the direction and magnitude of the change (vector) in the skin color information calculated by the color difference calculation means 13.

  However, if the skin color information of the ideal color information and the skin color information of the measurement color information are very large, if the characteristic color information is changed as much as the color difference of the skin color information, it is far from the color that the user wanted. There is a possibility. Therefore, when the color difference between the skin color information of the ideal color information and the skin color information of the measurement color information is large, that is, when the difference is equal to or greater than a predetermined threshold, the change degree of the characteristic color information is suppressed (weighted). (Refer to Modification 2). Thereby, the feature color information closer to the user's image can be notified.

  The color that appears when a color material (cosmetics) of a certain color is applied to the base (skin) is a mixture of the color of the color material and the color of the base. For this reason, it may be inappropriate to select a cosmetic having the calculated changed feature color as an ideal cosmetic. Therefore, the cosmetic calculation unit 15 of the color calculation unit 12 calculates the cosmetics most suitable for the user based on the skin color information of the measurement color information and the changed feature color information calculated by the feature color changing unit 14. That is, cosmetics that show the changed characteristic color when applied to the user's face are extracted from a database of predetermined cosmetics. As the calculation method, any known method may be used. For example, the method described in Patent Document 1 (Japanese Patent Laid-Open No. 2006-267115) can be used. Specifically, information in which the cosmetic and the color of the cosmetic itself are associated is stored in the HDD 105 or the like. Then, the database is searched based on the skin color information and the changed feature color information of the measurement color information, and a cosmetic product that becomes the changed feature color when applied to the user's face is calculated for the skin color of the measurement information. be able to. Therefore, it is possible to select a cosmetic product that gives an impression close to the ideal makeup state of the user when applied to the user.

  The characteristic color output unit 16 generates display information to be displayed on the display 301 of the output unit 300, print information to be printed on the printer 302, and the like based on the measurement result (calculation result) in the color measuring apparatus 100. The measurement result includes, for example, the changed feature color information calculated by the feature color changing unit 14, evaluation of the user's skin color, and the optimal cosmetic calculated by the cosmetic calculating unit 15.

  Specifically, as the changed feature color information, for example, the changed feature color itself may be displayed or printed and notified to the user, or the changed feature color information (coordinates) may be notified together with the feature color. Further, as the evaluation of the user's skin color, for example, based on the skin color information of the measurement color information, the user's skin color is a white skin color, a yellowish skin color, a reddish skin color, etc. Further, apart from the notified feature color, it is also possible to notify a color candidate that matches the user's skin color. In addition, as an optimal cosmetic, specifically, for example, an image, a product number, a product name, and the like of the cosmetic can be notified as appropriate. Further, the changed feature color may be synthesized on the measurement image of the user's face to notify the image image with makeup, and the user can more clearly confirm his / her state after makeup. Furthermore, the image image of the user who performed the makeup and the face in the ideal makeup state designated by the user may be notified side by side. As a result, the user can more clearly grasp whether the image is in accordance with the user's image by comparing the image of his / her makeup state with the ideal makeup state.

  Hereinafter, the operation of the color measurement system 500 according to the first embodiment will be described. The color measurement system 500 reads ideal color information from the ideal color information storage unit 105 a of the HDD 105 and displays it on the display 301. The user designates the ideal color information by selecting an ideal makeup state image from the ideal color information displayed on the display 301. Further, by measuring the user's face with the XYZ camera 200, the measurement information is stored in the semiconductor memory 106 by the XYZ camera 200. Based on these pieces of information, the color measuring apparatus 100 calculates a characteristic color to be notified to the user.

  The flow of color measurement processing (color measurement method) performed by the color measurement apparatus 100 will be described below based on the explanatory diagram of FIG. 3 and the flowchart of FIG. In addition, each step of the color measurement process demonstrated below is an example, and this application is not limited to this. In addition, the order of steps can be changed as appropriate. Details of the processing in each step are as described in the explanation of each means of the color measuring apparatus 100.

  First, in step S1, ideal color information is detected by the ideal color information detection means 10. In step S11, the ideal color information storage unit 105a of the HDD 105 is accessed, the ideal color information input by the user is read, and the face portion (face area) 1 (see FIG. 3) is detected from the ideal color information. Next, in step S12, the ideal color information detecting means 10 detects a skin color area 2 for detecting the skin color from the face part (face area) 1, as shown in FIG. 3, in order to detect the skin color information of the ideal color information. decide. From this skin color area 2, skin color information (a, b, c) is detected. Next, in step S13, a feature color region 3 is determined from the face portion (face region) 1 in order to detect a feature color (for example, blusher color) of the ideal color information. Feature color information (A, B, C) is detected from the feature color region 3. An image in which the color of the characteristic color information (A, B, C) is directly added to the user's face part (face area) 1 is shown in the center of FIG. If the user's skin color is different from the ideal skin color, color contrast occurs and the appearance of the characteristic color is different. Therefore, in the present application, the color of the characteristic color is changed in accordance with the user's skin color, so that the impression of the makeup state ideal for the user is made closer.

  In step S2, the measurement color information acquisition unit 11 acquires measurement color information of the user's face. First, in step S21, the measurement color information acquisition unit 11 accesses the semiconductor memory 106 to read out measurement image information, and detects the user's face portion (face area) 4. Next, in step S <b> 22, a skin color area 5 for detecting a skin color is determined from the detected face part (face area) 4, and skin color information (d, e, f) is detected from the skin color area 5.

  Thereafter, in step S3, the color difference calculation means 13 uses the skin color information (a, b, c) of the ideal color information detected in step S12 and the skin color information (d, e, f) of the measured color information detected in step S22. The difference is calculated. Next, in step S4, the feature color changing unit 14 changes the color of the feature color information (A, B, C) of the ideal color information based on the difference calculated in step S3S2, and proposes the change to the user. Characteristic color information (D, E, F) is calculated. At this time, the characteristic color is changed so as to correspond to the difference between the skin color information of the ideal color information and the skin color information of the measurement color information.

  Next, in step S5, the cosmetic calculation unit 15 uses the skin color information (d, e, f) and the changed feature color information (D, E, F) of the measurement color information to store the cosmetic database stored in the HDD 105. The optimal cosmetic product to be changed characteristic color information (D, E, F) is calculated. Thereafter, in step S6, the characteristic color output unit 16 generates display information to be displayed on the display 301 and print information to be printed on the printer 302 as output information based on the measurement result (calculation result). The measurement result is changed on the measured characteristic color information calculated in step S4, the skin color evaluation based on the user's skin color information measured in step S2, the optimal cosmetic calculated in step S5, and the measurement image of the user's face An image or the like in which makeup is performed by combining characteristic colors. For example, on the right side of FIG. 3, an image in which cosmetics of the changed characteristic color calculated in the characteristic color region 6 are added on the measurement image of the user's face is shown.

  The display information generated in this way by the color measuring apparatus 100 is displayed on the display 301, and the print information is printed on the printer 302. Thereby, the user can know the characteristic color information in an ideal makeup state and the optimal cosmetic for obtaining the characteristic color.

  As described above, in the color measurement system 500 according to the first embodiment, the feature color of the ideal color information is applied to the user as it is by changing the feature color of the ideal color information based on the difference between the ideal skin color and the measured skin color. It is possible to eliminate problems such as not following the image. In particular, color information measured by the XYZ camera 200 and accurately measured that is closer to human vision can be used effectively, and notification of colors that match the user's wishes and images can be performed. . Furthermore, in Example 1, the optimal cosmetic is calculated based on the calculated feature color information and the user's skin color information. Therefore, it is possible to notify the user of cosmetics that give an impression close to the ideal makeup state.

(Modification 1)
In the first embodiment, the difference is calculated and the characteristic color is changed using the L * a * b * color space or the HSL color space, but the present invention is not limited to this. For example, the above-described processing can be performed based on the L * C * h color system based on the L * a * b * color system. The L * C * h color system is a color system in which lightness, saturation, and hue are numerically expressed. L *, C *, and h represent lightness, saturation, and hue angle, respectively. Since the hue is not an orthogonal coordinate system but a polar coordinate system, the color difference calculated by the color difference calculation means 13 is directly used as the next characteristic color by using the L * C * h color system that represents the hue as an angle in this way. There is an advantage that it can be used in the changing means 14.

  The measurement color information acquired by the measurement color information acquisition unit 11 is measured by an XYZ camera 200 that measures XYZ tristimulus values. Since it is difficult for the XYZ camera 200 to directly measure L *, C *, and h, L *, C *, and h are calculated in terms of XYZ tristimulus values measured by the XYZ camera 200. Hereinafter, the calculation process will be described. First, the XYZ tristimulus values are converted into L * a * b * color system values. The conversion formula is defined by the International Commission on Illumination (CIE) as follows.

In the above formula, f (t) is represented by the following formula. X _n , Y _n , and Z _n represent tristimulus values for a subject with a reflectance of 100%.

  Next, L *, C *, and h are calculated from the values of L *, a *, and b * calculated by the above equations. L * can be used as it is. C * and h can be calculated by the following formulas using the values of a * and b *.

  Further, when the skin color information and the characteristic color information of the ideal color information are also stored in the ideal color information storage unit 105a in the L * a * b * color system, the L * C * h color specification is based on the above formula. When converted to a system and stored in the L * C * h color system, it is used as it is. Further, when the image is stored in the XYZ color system, it is converted into the L * C * h color system based on the above formula.

A procedure for calculating the color difference in the color difference calculating unit 13 and a procedure for changing the characteristic color in the characteristic color changing unit 14 in the L * C * h color space will be described. For example, the coordinates of the skin color information of the ideal color information and _{[L * 1, C * 1} , h 1], the coordinates of the characteristic color information of an ideal color information and _{[L * 2, C * 2} , h 2], measured the coordinates of the skin color information of the color information and _{[L * 3, C * 3} , h 3]. And skin color information of the ideal color information _{[L * 1, C * 1} , h 1], skin color information of the measured color information and _{[L * 3, C * 3} , h 3] the difference between the color difference information ΔL * C * h Then, this ΔL * C * h can be calculated by the following equation.

ΔL * C * h = [L * 3 -L 1 *, C * 3 -C * 1, h 3 -h 1]

The color difference calculating means 13 stores the color difference information ΔL * C * h calculated by the above formula together with the characteristic color information [L * ₂ , C * ₂ , h ₂ ] of the ideal color information, and storage means (memory such as the RAM 103 and the HDD 105). ) So that it can be referred to by the next characteristic color changing means 14.

In characteristic color change means 14, based on the characteristic color information of the calculated color difference information [Delta] L * C * h and the ideal color information stored in the storage means by the color difference calculating unit _{13 [L * 2, C *} 2, h 2], change the color of the characteristic color proposed to the user to capture changes characteristic color information _{[L * 4, C * 4} , h 4]. This change characteristic color information _{[L * 4, C * 4} , h 4] can be calculated by the following equation. Thus, the relative positional relationship between the skin color information of the ideal color information _{[L * 1, C * 1} , h 1] and the skin color information of the measuring color information _{[L * 3, C * 3} , h 3], by applying to the feature color information of an ideal color information _{[L * 2, C * 2} , h 2], it is possible to calculate the change characteristic color information _{[L * 4, C * 4} , h 4].

_{[L * 4, C * 4} , h 4] = [L * 2, C * 2, h 2] + ΔL * C * h

  In the first modification, the L * C * h color system is used, so that it is more consistent with the user's desire and image based on more accurate color information without depending on the type of device such as a display. Two color notifications can be made. In particular, by effectively using the color information accurately measured by the XYZ camera 200, more effective color notification can be performed.

As described above, in the case of the L * C * h color system, since polar coordinates are used, a change in hue can be expressed more accurately compared to other color systems. However, since polar coordinates are used, the magnitude of the hue change may depend on the saturation C *. Therefore, even if the hue angle difference Δh = h ₃ −h ₁ (the hue difference expressed by the angle difference of the hue angle h) used in the above calculation is the same, the color change increases as the saturation C * increases. Sometimes. In order to eliminate this, it is more desirable to change the amount of change in the hue h based on the magnitude of the saturation C *.

First, when the hue of the skin color information of the measured color information is h _u1 and the hue of the skin color information of the ideal color information is h _r1 , the hue change amount (hue angle difference) Δh _skin of the _skin color is calculated by the following formula: Can do.

Δh _skin = h _{U1 -hr1}

Also, saturation of the skin color information of an ideal skin information and C * _r1, when the saturation characteristic color information of an ideal information was C * _r2, with a Delta] h _skin calculated above, wherein color information (e.g., (Lipstick) change amount Δh _lip can be calculated by the following equation.

Δh _lip = Δh _skin × (C * _r1 / C * _r2 )

By changing the hue of the characteristic color using Δh _lip calculated by the above formula, the amount of color change can be made constant even when the saturation C * is different, and it can be changed according to the user's desire and image. It is possible to notify the matched feature color.

Note that the value used for calculating the hue change amount is not limited to the hue angle difference Δh, and for example, the hue difference ΔH * may be used. Here, ΔH * = (ΔE ² −ΔL * ² −ΔC * ² ) ^1/2 and ΔE = (ΔL * ² + Δa * ² + Δb * ² ) ^1/2 are shown. In this case, the characteristic color information closer to the ideal makeup state is obtained by changing the a * and b * of the changed characteristic color information so that the skin color ΔH * and the characteristic color ΔH * match. Notification can be made.

  Here, using FIG. 4A, in Example 1 and Modification 1, feature colors are calculated using the HSL color system, L * a * b * color system, and L * C * h color system. Each value at this time will be specifically described. FIG. 4A shows lipstick color information (ideal mouth) as skin color information and feature color information detected from ideal color information, skin color information detected from user measurement color information, skin color information difference, and changed feature color. Information RGB, HSL color system, L * a * b * color system, and L * C * h color values are displayed. Also, the change feature color information 1 is a value using the HSL color system, the change feature color information 2 is a value using the L * a * b * color system, and the change feature color information 3 is a value using the L * C * h color system. The changed characteristic color information 4 is a value in the case of the L * C * h color system and Δh applied to h. Each value is shown in FIG. 4A. For ease of understanding, values obtained by converting each value with another color system are also displayed.

  First, the skin color information difference (−3, −19, −15) was calculated based on the value of the HSL color system. According to this numerical value, the user's skin color is reddish, dull and darker than the ideal skin color. Therefore, if the feature color information detected from the ideal color information is added to the user's skin as it is, the hue may be lighter than the ideal makeup state and may appear to float.

  In order to solve this problem, the characteristic color information is changed based on the difference between the skin color information between the ideal color information and the measured color information. Specifically, the difference (−3, −19, −15) of the skin color information calculated by the HSL display system is applied to the characteristic color (ideal mouth) information (239, 163, 158) of the ideal color information, Change characteristic color information 1 (236, 144, 143) was calculated. In the case of the HSL display system, the characteristic color is changed according to the difference of hue H, saturation S, and lightness L where the color contrast occurs, so that when it is added to the user's skin, it is the same as the ideal makeup state Impression was obtained.

  Next, the difference (−6.39, 4.89, 2.17) of the skin color information was calculated based on the value of the L * a * b * color system. This is applied to the characteristic color information (57.62, 53.13, -1.57) of the ideal color information, and the changed characteristic color information 2 (51.23, 58.02, 0.6) is calculated. . When the calculated changed feature color information 2 was added to the user's skin, the saturation and brightness changed according to the skin color difference, but there was no change in hue.

  Next, based on the value of the L * C * h color system obtained by converting the L * a * b * color system based on the above formula, the difference between the skin color information (−6.39, 4.17663, − 0.14818) was calculated. This is applied to the characteristic color information (57.62, 53.15319, -0.02954) of the ideal color information, and the changed characteristic color information 3 (51.23, 57.32982, -0.17772) is calculated. did. In this example, due to the influence of the saturation C *, the hue has changed beyond the threshold value compared to the characteristic color information of the ideal color information.

For this reason, when the characteristic color is calculated using the above-described equation for changing the hue difference Δh according to the saturation C *, Δh _lip = −0.05740, which is the characteristic color information h of the ideal color information. By applying, h = −0.08694 of the feature color information of the changed feature color information 4 was calculated. As a result, the hue, saturation, and lightness change amount between the skin color and the characteristic color can be kept constant without being affected by the size of C *, and when it is added to the user's skin, the user feels uncomfortable. The impression that was closer to the ideal makeup was obtained.

(Modification 2)
Next, as a second modification, an example in which the feature color is calculated by suppressing (weighting) the change degree of the feature color information when the HSL display system is used will be described with reference to FIG. 4B. Here, the color of the lipstick is calculated as the characteristic color using the HSL color system. As shown in FIG. 4B, it is assumed that the HSL coordinates of the skin color information of the ideal color information are (19, 181, 210) and the HSL coordinates of the characteristic color information of the ideal color information are (239, 163, 158). On the other hand, it is assumed that the HSL coordinates of the skin color information of the measurement color information of the user are (12, 172, 178). In this case, the difference between the skin color information of the ideal color information and the skin color information of the measurement color information is (−7, −9, −32).

  When this difference is applied as it is, the changed characteristic color information becomes (232, 154, 126), and the hue H and the luminance S are greatly different from the characteristic color information of the original ideal color information. It was different from the impression of the state. Therefore, in the second modification, the degree of change is reduced. Specifically, H and L are suppressed (smaller) to (−4, −9, −16) with respect to the difference (−7, −9, −32). By applying the post-suppression difference, the changed feature color information becomes (235, 154, 142), and an appropriate change is obtained. Therefore, in the second modification, it is possible to propose a characteristic color that is close to the user's desire and does not conflict with the user's skin color.

  Note that the degree of weighting, that is, the degree of suppression with respect to the magnitude of the difference can be determined as appropriate. For example, it may be a constant within a predetermined value range, or a suppression rate can be set. In the second modification, the suppression rate is set to 0.5 times, and values obtained by multiplying the difference H and L by 0.5 times are used. However, the suppression rate may not be constant depending on the color system, but may be multiplied by a different suppression rate. Further, in addition to the HSL color system, the difference can be suppressed also in the L * a * b * color system or the like.

  As described above, according to Modification 1 and Modification 2, the L * a * b * color system, the HSL color system, and the L * C * h color system are preferably used in the calculation of the second color. it can. Among these, the HSL color system and the L * C * h color system can be used more suitably. When the difference is equal to or greater than the threshold value, it is preferable to suppress the change degree of the characteristic color information. For example, by suppressing the difference or changing the hue difference Δh or ΔH * in the L * C * h color system, the amount of change in the hue, saturation, and brightness between the skin color and the characteristic color is not excessively increased. Thus, it becomes possible to notify the characteristic color that gives an impression closer to the ideal makeup state.

(Modification 3)
In the first embodiment, the first modification, and the second modification, the color of the feature color information is changed based on the difference between the skin color information of the ideal color information and the skin color information of the measurement color information, and the changed feature color information is calculated. However, the present application is not limited to this. For example, as Modification 3, these processes may be performed based on the difference between the skin color information of the ideal color information and the characteristic color information of the ideal color information. In this case, the coordinates of the skin color information of the ideal color information and _{[L * 1, C * 1} , h 1], and the coordinates of the characteristic color information of an ideal color information and _{[L * 2, C * 2} , h 2] At this time, the color difference information ΔL * C * h ′ can be calculated by the following equation.

ΔL * C * h '= [ L * 2 -L * 1, C * 2 -C * 1, h 2 -h 1]

The color difference calculation means 13 stores the color difference information ΔL * C * h ′ calculated in this way together with the skin color information [L * ₃ , C * ₃ , h ₃ ] of the measurement color information in a storage means such as the RAM 103 or the HDD 105. And keep it.

The characteristic color changing unit 14 changes the characteristic color information of the ideal color information based on the color difference information calculated by the color difference calculating unit 13 and held in the storage unit and the skin color information of the measured color. Here, the characteristic color information after change (change characteristic color information) and _{[L * 4, C * 4} , h 4]. This change characteristic color information _{[L * 4, C * 4} , h 4] , the skin color information of the ideal color information _{[L * 1, C * 1} , h 1] , wherein color information of an ideal color information [L * _2, C * _2, h _2] the relative positional relationship between the, shall be applied to the skin color information of the measuring color information _{[L * 3, C * 3} , h 3]. In this case, change characteristic color information _{[L * 4, C * 4} , h 4] can be expressed as follows.

_{[L * 4, C * 4} , h 4] = [L * 3, C * 3, h 3] + ΔL * C * h '

  As described above, also according to the third modification, notification of a color that matches the user's desire or image can be performed based on accurate color information. In particular, by effectively using the color information accurately measured by the XYZ camera 200, more effective color notification can be performed. In the third modification, ΔL *, ΔC *, and Δh may be suppressed when the hue, saturation, and lightness change amount between the skin color and the characteristic color exceed the values, depending on the impression of the user. It is possible to notify near feature color information.

  In the first to third modifications, it is more preferable to change the feature color information in consideration of the saturation C *, and it is possible to notify the feature color information closer to the user's impression. Become.

  Next, a color measurement system according to Example 2 of the present application will be described. The color measurement system of the second embodiment has the same basic configuration as the color measurement system 500 of the first embodiment shown in FIGS. However, in the second embodiment, the processing in each means is different from the first embodiment and the first and second modifications thereof in that the hair color is referred to in addition to the skin color. That is, in the first embodiment and the first and second modifications, the feature color (second color) suitable for the user is notified based on the user's skin color (first color). On the other hand, in Example 2, based on the user's skin color (first color) and hair color (third color), a point makeup color (characteristic color: second color) suitable for the user is set. It is a notification. It is known that the impression of makeup changes greatly depending on the hair color so that the skin color affects the point makeup. Therefore, in the second embodiment, the user's face and the hair in the region surrounding the face are measured, and when the entire face is viewed, a characteristic color having a color that gives an impression close to ideal color information is notified. Therefore, in Example 2, the skin color information and the hair color information are acquired from the measurement color information, and the color of the characteristic color information of the ideal color information is changed based on the skin color information and the hair color information. The characteristic color information is determined. For example, it can be performed by the following procedure.

  In the second embodiment, the XYZ camera 200 measures not only the user's face but also an area surrounding the face, that is, an area including hair, and stores the measurement color information in the semiconductor memory 106. The measurement color information acquisition unit 11 reads out the measurement color information, and acquires the measurement color information of the user's skin color in the same manner as in the first embodiment. In addition to this, the measurement color information acquisition unit 11 acquires measurement color information of the hair color and stores it in a memory or the like. Next, the color difference calculation means 13 calculates the difference between the skin color information of the ideal color information and the skin color information of the measurement color information, or the difference between the skin color information of the ideal color information and the characteristic color information of the ideal color information.

  Based on the skin color information of the difference and the measured color information, the characteristic color changing unit 14 changes the color of the characteristic color information of the ideal color information and temporarily determines the changed characteristic color information in the same manner as in the first embodiment. . The characteristic color changing unit 14 according to the second embodiment further corrects the changed characteristic color information (second color) in consideration of the hair color information (third color) of the measurement color information. An example of this correction will be described. When the ideal color information detecting unit 10 acquires the skin color information and characteristic color information of the ideal color information, the hair color information of the ideal color information is also acquired as reference color information. To do. Using the hair color information as reference information, the changed characteristic color information may be corrected by calculating a difference between the measured color information and the hair color information.

  More specifically, for example, paying attention to lightness, if the hair color information of the measurement color information is brighter than the hair color information of the ideal color information as reference information, the changed characteristic color information is corrected to be brighter. . Also, paying attention to the hue, if the hair color information of the measurement color information is more reddish than the reference information, the changed characteristic color information is also corrected to a reddish color. That is, the characteristic color information is changed based on the skin color information, and the changed characteristic color information is further corrected based on the hair color information. For example, the correction can be made based on the difference between the hair color information of the measurement color information and the hair color information of the ideal color information × coefficient (coefficient is 1 or less). Alternatively, it can be corrected with a predetermined constant corresponding to the hair color information of the measurement color information.

  Alternatively, absolute evaluation can be performed using the hue and brightness values of the hair color information of the measurement color information without referring to the hair color information of the ideal color information. That is, the characteristic color information is corrected by changing the hue and lightness of the changed characteristic color information in accordance with the hue and lightness of the hair color information. More specifically, as absolute evaluation, threshold values such as bright, dark, strong reddish, strong yellowish are set. When the evaluation result exceeds the threshold, correction is performed.

  As described above, in the second embodiment, not only the user's skin color that affects the appearance of the characteristic color but also the hair color is taken into account, so it is possible to notify the user of the color and cosmetics that match the user's wishes and image. It becomes.

  Next, the color measurement system of Example 3 will be described. The color measurement system of the third embodiment also has the same basic configuration as the color measurement system 500 of the first embodiment shown in FIGS. 1 and 2, but the processing in each means is different from the first embodiment. Specifically, in the third embodiment, in addition to correcting the change feature color information proposed to the user based on the skin color and the hair color as in the second embodiment, the color of the user's clothes is further added. In consideration of this, the changed feature color information is corrected. Even if the skin color and feature color are the same, the appearance of the clothing may vary greatly depending on the color of the user's clothing, such as the skin color or feature color appearing differently or the contrast intensity may be different. The characteristic color is determined in consideration of the color.

  In the third embodiment, the XYZ camera 200 measures a wider area than the second embodiment, that is, a region including the user's face, hair, and clothes. Based on the measurement color information, the measurement color information acquisition unit 11 acquires the measurement color information of the clothing color in addition to the measurement color information of the user's skin color and the measurement color information of the hair color. Next, as in the first embodiment, the color difference calculating unit 13 calculates the difference, and the characteristic color changing unit 14 provisionally determines the changed characteristic color information. The characteristic color changing unit 14 further corrects the changed characteristic color information (second color) in consideration of the hair color information of the measurement color information as the third color and the color information of the clothes. In this case, in the same manner as in the second embodiment, after the changed feature color information is corrected based on the hair color information, the changed color feature information is further corrected based on the color information of the clothes in consideration of hue, brightness, and the like. be able to. Alternatively, the changed characteristic color information can be corrected by making a comprehensive determination based on the hair color information and the clothing color information. In this case, the correction based on the clothing color information can be performed according to the correction based on the hair color information described in the second embodiment.

  As described above, according to the third embodiment, not only the user's skin color that affects the appearance of the characteristic color but also the hair color and clothing color are taken into consideration. Notification can be performed. In the third embodiment, the characteristic color is corrected based on both the hair color and the color of the clothes. However, the present invention is not limited to this, and the characteristic color is calculated based only on the color of the clothes. It is also possible to correct.

  Next, a color measurement system according to Example 4 will be described with reference to FIGS. As illustrated in FIG. 6, the color measurement system 500A according to the fourth embodiment includes an XYZ camera 200 as an input unit, a color measurement device 100A, and an output unit 300. The color measurement apparatus 100A according to the fourth embodiment includes an ideal color information detection unit 10A, a measurement color information acquisition unit 11A, a color calculation unit 12A, and a characteristic color output unit 16A. The color calculation unit 12A includes a characteristic color calculation unit 17A, a characteristic color change unit 14A, and a cosmetic calculation unit 15A. The color measurement system according to the fourth embodiment has the same hardware configuration as that of the first embodiment shown in FIG.

  In each of the above embodiments, the face image in the ideal makeup state of the user is the ideal color information (reference color information), but the reference color information used in the fourth embodiment is the makeup that the user wants (desired). It is impression information (keywords) that expresses the impression of color, such as an image or a color request. Examples of the color impression information include “I want to make my skin look brighter”, “I want to make pink makeup”, and the like. Based on the reference color information and the user's skin color (first color), the color measuring apparatus 100A calculates a characteristic color (second color) to be notified to the user.

  Hereinafter, the operation of the color measurement system 500A according to the fourth embodiment will be described. In the color measurement system 500A, the face of the user after makeup is measured by the XYZ camera 200, and measurement image information is generated. The face area of the measurement image information includes colors such as skin and pupil. The measurement image information is stored in the semiconductor memory 106A. In addition, the color measurement system 500A acquires ideal color information input by the user from a keyboard, a mouse, or the like that is an input unit. These may be sent from the input means to the color measurement apparatus 100A as parameters via the RAM 103 or the like of the storage means, or the color measurement apparatus 100A reads out what is stored in the ideal color information storage unit 105a. It may be a thing.

  Next, the flow of color measurement processing (color measurement method) performed by the color measurement apparatus 100A will be described based on the flowchart of FIG. In step S1A, the ideal color information detection unit 10A accesses the ideal color information storage unit 105a of the HDD 105A, the RAM 103, etc., reads the ideal color information input by the user, and detects the ideal color information.

  Next, in step S2A, the measurement color information acquisition unit 11A acquires measurement color information of the user's face. First, in step S21A, the measurement color information acquisition unit 11A accesses the semiconductor memory 106A, reads the measurement image information, and detects the face portion (face area) of the user. Next, in step S22A, skin color information and pupil (iris) color information are detected from the detected face portion (face region). The color information other than the skin color to be detected is not limited to the color information of the pupil, and color information of eyebrows and eyelashes may be detected, and the measurement color information of the user's face is effectively used. , A feature color more suitable for the user can be notified. In addition, when the feature color is calculated in consideration of the hair color and the clothing color, the hair color information and the clothing color information are further detected.

  Next, in step S3A, the characteristic color calculation means 17A uses mascara, lipstick, etc. suitable for the user's skin color and pupil color based on the skin color information and pupil color information of the measurement color information detected in step S22A. A characteristic color such as blusher is calculated. Thereafter, in step S4A, the characteristic color changing unit 14A changes the color of the characteristic color calculated in step S3A based on the user's desired makeup image (ideal color information) detected in step S1A. This makes it possible to calculate a feature color that matches the user's skin color, pupil color, etc., and that matches the ideal color information.

  More specifically, as a first example, if the ideal color information is a desire to “make the skin look brighter”, a feature color suitable for the user's skin color or pupil color is calculated, and the calculated feature color Is changed based on the ideal color information. Then, a characteristic color of a color that is close to the hue of the skin color information of the measurement color information and has a lower brightness and higher saturation than the skin color information is calculated. As a second example, when the user's skin color is yellowish, an orange feature color close to the skin color hue is calculated instead of a pink feature color. Next, in accordance with the saturation and lightness of the skin color, the saturation and lightness of the characteristic color are changed as appropriate to obtain an orange characteristic color that better matches the user's skin color. As a third example, it is assumed that the ideal color information is “I want to make pinkish makeup” and the user's skin color is yellowish. In this case, after calculating a pink feature color, the hue, saturation, and lightness of the feature color are appropriately changed according to the skin color, and a feature color that matches the user's skin color (for example, a coral pink with a yellowish color) Characteristic color). As described above, proposals according to the user's wishes can be made. The feature color is not limited to one color, and color information of various feature colors that the user wants to apply to the face, such as lipstick, eye shadow, blusher, eyebrow, and mascara, can be calculated.

  In the fourth embodiment, the characteristic color is calculated according to the user's skin color, pupil color, and the like, and then the calculated characteristic color is appropriately changed based on the ideal color information. However, the present application is limited to this. It will never be done. For example, an optimal feature color can be calculated after comprehensively determining the user's skin color, pupil color, and ideal color information.

  Next, in step S5A, the cosmetic calculation means 15A selects an optimal cosmetic product for the user for each feature color from the cosmetic database stored in the HDD 105A based on the skin color information and the changed feature color information of the measurement color information. calculate.

  Thereafter, in step S6A, the characteristic color output unit 16A generates display information to be displayed on the display 301 and print information to be printed on the printer 302 as output information based on the evaluation result. The evaluation result is obtained by combining the changed feature color information calculated in step S3A and the user's skin color, the optimal cosmetic product calculated in step S5A, and the changed feature color calculated on the measurement image of the user's face, It is an image or the like with makeup. The output information is displayed on the display 301 or printed on the printer 302. Thereby, it is possible to notify the user of a makeup state that matches a wish such as “I want to make my skin look brighter”.

  As described above, also in the color measurement system 500A according to the fourth embodiment, it is possible to effectively use color information measured by the XYZ camera 200 that is closer to human vision and is accurately measured, and matches the user's desire and image. It is possible to notify the color and cosmetics. Furthermore, in Example 4, the user's makeup state is evaluated, it is determined whether or not it is appropriate, the result is notified to the user, and if it is not appropriate, a more appropriate characteristic color is notified. . Therefore, it is possible to notify the user of a characteristic color that is in an optimal makeup state.

  Next, a color measurement system according to Example 5 will be described with reference to FIGS. As shown in FIG. 8, the color measurement system 500B according to the fifth embodiment includes an XYZ camera 200 as an input unit, a color measurement device 100B, and an output unit 300. The color measurement apparatus 100B according to the fifth embodiment includes an ideal color information detection unit 10B, a measurement color information acquisition unit 11B, a color calculation unit 12B, and a characteristic color output unit 16B. The color calculation unit 12B includes a color evaluation unit 18B, a characteristic color change unit 14B, and a cosmetic calculation unit 15B. Further, the color measurement system of the fifth embodiment also has the same hardware configuration as that of the first embodiment shown in FIG.

  In the color measurement system 500B according to the fifth embodiment, the degree of relevance of the color of the user's makeup state is calculated based on measurement image information (measurement color information) including the user's face. Based on this degree of association, the makeup state is evaluated as to whether it is an appropriate makeup state suitable for the user's skin color or the like, or an inappropriate makeup state, and the evaluation result is notified. That is, the color measurement system 500B and the color measurement device 100B also function as a color evaluation system and a color evaluation device.

  Hereinafter, the operation of the color measurement system 500B according to the fifth embodiment will be described. In the color measurement system 500B, the face of the user after makeup is measured by the XYZ camera 200, and measurement image information is generated. The measurement image information is stored in the semiconductor memory 106B. Also, the ideal color information storage unit 105a of the HDD 105B stores reference color information when calculating the degree of association between the skin color and the characteristic color and evaluating the makeup state as the ideal color information. As ideal color information, for example, color information of various skin colors and color information (hue, saturation, brightness) appropriate for each skin color are associated with each other, For example, an acceptable range of hue angle.

  Next, the flow of color measurement processing (color measurement method) performed by the color measurement apparatus 100B will be described based on the flowchart of FIG. In step S1B, the ideal color information detection unit 10B accesses the ideal color information storage unit 105a of the HDD 105B, and as the ideal color information, color information of various skin colors and color information of characteristic colors suitable for each are associated with each other. The range information of the hue angles of the grouped characteristic colors is read out and stored in the RAM 103 or the like.

  In step S2B, the measurement color information acquisition unit 11B acquires measurement color information of the user's face. First, in step S21B, the measurement color information acquisition unit 11B accesses the semiconductor memory 106B, reads the measurement image information, and detects the face portion (face area) of the user. Next, in step S22B, skin color information and characteristic color information such as lipstick, eye shadow, blusher, etc. are detected from the detected face part (face region).

  Next, in step S3B, the color evaluation unit 18B calculates the degree of color relevance based on the ideal color information detected in step S1B and the skin color information and characteristic color information of the measured color information detected in step S22B. Then, it is evaluated whether or not the characteristic color is an appropriate makeup that matches the skin color of the user. For example, the evaluation is performed based on the skin color and the hue of each characteristic color. It is determined whether or not the hues of the skin color and each feature color have a group, and if there is a group, it can be determined that the feature color matches the user's skin color. Specifically, if the hue angle difference between the skin color hue and each feature color is equal to or less than a certain value, each feature color is determined to be suitable for the user's skin color and to be a united makeup. Further, even when there is a single feature color exceeding a certain value, it may be determined that the feature color is appropriate because it is considered as an accent color.

  On the other hand, when a plurality of feature colors having a hue angle difference with a skin color hue equal to or greater than a certain value are detected, vectors directed to each feature color are compared with the skin color as a reference. If the directions of the vectors are the same, the color tendencies are considered to be similar, so that it is determined to be appropriate. When the direction of each vector is the reverse direction, it can be said that there are a plurality of colors having greatly different hues, and therefore, it is determined to be inappropriate. Further, the evaluation can be performed based on the saturation and brightness. High saturation and low brightness colors appear dark. Since dark colors give a strong impression in makeup, it is generally considered desirable to use only one color (including similar hues). Therefore, if the hue angle difference between a plurality of feature colors is equal to or greater than a certain value, and both feature colors are larger than the skin color and have high saturation and low brightness, it is determined as inappropriate.

  Next, in step S4B, it is determined whether the characteristic color is appropriate based on the evaluation result. If it is appropriate (Yes), the process proceeds to step S7B. In step S7B, the characteristic color output unit 16B generates display information to be displayed on the display 301 and print information to be printed on the printer 302 based on the evaluation result. In this case, since it is an evaluation result that the user's makeup state is appropriate for the user, it may be output as text, or a picture, mark, stamp, etc. indicating that it is appropriate You may output, and a sound and light may be output.

  On the other hand, if it is determined in step S4B that the feature color is inappropriate (No), the fact may be notified and the process may be terminated. However, in the fifth embodiment, the feature color is suitable for the user. The feature color is changed to notify the user. Therefore, it progresses to step S5B and the characteristic color change means 14B changes the color of a characteristic color according to a user's skin color, and calculates the appropriate characteristic color which suits a user's skin color. This calculation can be performed by changing the hue, saturation, and lightness of the characteristic color detected from the user's face corresponding to the hue, saturation, and lightness of the user's skin color. Further, a more suitable feature color can be newly calculated based on the user's skin color without referring to the detected feature color.

  Next, in step S6B, based on the skin color information of the measurement color information and the feature color information calculated in step S5B by the cosmetic calculation means 15B, the cosmetics optimal for the user are selected from the cosmetic database stored in the HDD 105B. Are calculated for each feature color.

  Thereafter, in step S7B, the display information displayed on the display 301 and the printing information printed on the printer 302 are generated as output information by the characteristic color output means 16B. This output information is obtained by synthesizing the calculated feature color on the feature color information calculated in step S5B, the evaluation of the user's skin color, the optimal cosmetic calculated in step S6B, and the measurement image of the user's face. It is an image or the like with makeup. In addition, it can be notified that the user's original makeup state was inappropriate. With these output information, the user is informed that his makeup state does not match the skin color and is inappropriate, and has a characteristic color or a more appropriate makeup state that matches the user's skin color. Cosmetics can be notified.

  As described above, also in the fifth embodiment, based on accurate color information, the user's makeup state can be more accurately evaluated and notified to the user, and a more suitable color can be notified. In particular, by effectively using the color information accurately measured by the XYZ camera 200, more effective color notification can be performed.

  In Examples 1 to 4, an appropriate feature color is notified based on the user's skin color and the like. In Example 5, the suitability of the makeup state is evaluated based on the user's skin color and feature color. Yes. In the present application, a color measuring device or a color measuring system may be constructed by combining these embodiments. That is, when detecting the measurement color information from the measurement color information of the user, the skin color is detected, and if there is another color different from the skin color, the color is detected. Specifically, feature parts such as eyebrows, eyes, and mouth are detected from the face area, and the colors added to these are detected. When only the skin color is detected by the above detection process, the color measuring device according to the first to fourth embodiments is operated to calculate a suitable characteristic color corresponding to the skin color and notify the user. On the other hand, when the detected color includes a color (characteristic color) that can be changed by makeup such as lipstick, eye shadow, blusher, etc., the color measuring device according to the fifth embodiment is operated to make the user's makeup. The state is evaluated and the result of the evaluation is notified to the user. By using such a color measurement system, it is possible to provide a product having both color notification and color evaluation and excellent functionality.

  In each of the above embodiments, the user's skin color information is the first color, and the characteristic color such as lipstick and eye shadow is the second color, and the characteristic color (makeup color) is calculated and evaluated. However, the present application is not limited to the calculation of makeup color. For example, the second color is used as the hair color, and the hair color corresponding to the user's skin color, the hair coloring agent or the wig that becomes the hair color is notified based on the user's desired hair color and the user's skin color. You can also. In addition, the second color may be used as a clothing color to notify the user of clothing that matches the user's skin color and makeup state. The subject is not limited to a real human body, and may be a replica (three-dimensional object) such as a mannequin that simulates a human body, an artificial skin having a predetermined skin color, a design image (planar surface), or the like. Further, the subject is not limited to a human body or a model imitating a human body, and may be an animal or a plant such as a dog or a cat, and can be used for animal fashion or a design of a garden. The subject may be a product (inorganic material) such as an automobile or an electric product. The base color of these products is the first color, and based on the first color and appropriate reference color information, the second color added to the product is calculated and notified, or the second color It is also possible to notify a color material such as paint that becomes a color.

  Although the embodiments of the present invention have been described in detail with reference to the drawings, each of the above embodiments is only an example of the present invention, and the present invention is not limited only to the configuration of each of the above embodiments. . Needless to say, changes in design and the like within the scope of the present invention are included in the present invention.

10, 10A, 10B Ideal color information detection means 11, 11A, 11B Measurement color information acquisition means 12, 12A, 12B Color calculation means 13 Color difference calculation means 14, 14A, 14B Characteristic color change means 15, 15A, 15B Cosmetics calculation means ( Color material calculation means)
16, 16A, 16B Feature color output means 17A Feature color calculation means 18B Color evaluation means 100, 100A, 100B Color measurement device (color measurement means)
105, 105A, 105B HDD (storage means)
105a Ideal color information storage unit 200 XYZ camera (input means) 202 FPGA (correction means)
206 Main lens (optical system)
207 Color filter (optical bandpass filter)
207a, 207b, 207c Color filter 210 Micro lens array 210a Micro lens 211 Light receiving element array (light receiving means) 300 Output means 301 Display 302 Printer 500, 500A, 500B Color measurement system

JP 2006-267115 A

Claims (16)

Storage means for storing predetermined color information as reference color information;
Color detection means for acquiring measurement color information of the subject from an input means for measuring the color of the subject, and detecting a first color of the subject from the measurement color information;
Color calculating means for calculating a second color to be added to the subject based on the color information of the first color and the reference color information;
The reference color information is image information including a plurality of color information,
The color calculation unit calculates a difference between one color information corresponding to the first color and the color information of the first color among the plurality of color information, and is included in the plurality of color information. The color information is changed in accordance with the difference, and the second color is calculated. Storage means for storing predetermined color information as reference color information;
Color detection means for acquiring measurement color information of the subject from an input means for measuring the color of the subject, and detecting a first color of the subject from the measurement color information;
Color calculating means for calculating a second color to be added to the subject based on the color information of the first color and the reference color information;
The reference color information is image information including a plurality of color information,
The color calculation means calculates a difference between one color information corresponding to the first color and the other color information among the plurality of color information, and determines the difference and the color information of the first color. A color measuring device that calculates the second color based on the second color. Storage means for storing predetermined color information as reference color information;
Color detection means for acquiring measurement color information of the subject from an input means for measuring the color of the subject, and detecting a first color of the subject from the measurement color information;
Color calculating means for calculating a second color to be added to the subject based on the color information of the first color and the reference color information;
The color detection means detects a third color different from the first color in addition to the first color of the subject from the measurement color information;
The color calculation means calculates a second color to be added to the subject based on the color information of the first color, the color information of the third color, and the reference color information. Color measuring device. Storage means for storing predetermined color information as reference color information;
Color detection means for acquiring measurement color information of the subject from an input means for measuring the color of the subject, and detecting a first color of the subject from the measurement color information;
Color calculating means for calculating a second color to be added to the subject based on the color information of the first color and the reference color information;
The subject is a human body;
In the storage means, face image information including a plurality of color information is stored as reference face image information,
The measurement color information of the subject is face image information including at least the face of the subject,
The color detection means detects skin color information as the first color from the face image information of the subject;
The color calculation means includes a difference between the skin color color information detected from the skin color information and the reference face image information of the subject, or the skin color color information detected from the reference face image information and the reference Color difference calculating means for calculating a difference between other color information of the skin color detected from face image information ;
A color measuring apparatus comprising: color changing means for changing the other color information and calculating the second color based on the difference calculated by the color difference calculating means.   The color measurement device according to claim 1, wherein the color calculation unit weights the difference when the difference exceeds a threshold value. The reference color information is color impression information,
The color measurement device according to claim 1, wherein the color calculation unit calculates the second color based on the first color detected from the measurement color information of the subject and the impression information. . The subject is a human body;
In the storage means, face image information including a plurality of color information is stored as reference face image information,
The measurement color information of the subject is face image information including at least the face of the subject,
The color detection means detects skin color information as the first color from the face image information of the subject;
The color calculation means includes a difference between the skin color color information of the subject and the skin color information detected from the reference face image information, or the skin color color information detected from the reference face image information and the others. Color difference calculating means for calculating a difference in color information of
On the basis of the difference calculated by said color difference calculation means, said includes other color changing means for changing the color information to calculate the second color, the color according to claim 1 or 2 measuring device. The measurement color information of the subject is face image information including the face of the subject and a predetermined region surrounding the face,
In addition to the skin color information of the subject, the color detection means includes at least one of hair color information, clothing color information, and pupil color information as a third color. Detect
The color changing means calculates the second color by changing the other color information based on the difference calculated by the color difference calculating means, and based on the color information of the third color, The color measurement apparatus according to claim 7, wherein the color information of the second color is corrected. The reference color information is color relevance information,
The color detection means detects a first color of the subject from the measurement color information and detects a second color added to the first color;
The color calculation means includes color evaluation means for evaluating the second color based on the first color detected from the measurement color information of the subject and the color relevance information. The color measuring device according to any one of claims 1 to 4. The color relevance information is relevance information of at least one of hue, saturation, and brightness of the first color and the second color;
The color evaluation means evaluates that the second color is appropriate when the first color and the second color have the same tendency of at least one of hue, saturation, and brightness, The color measurement device according to claim 9, wherein when the trends are different, the second color is evaluated as inappropriate.   The said color evaluation means changes the said 2nd color based on the said 1st color and the said reference color information, when it evaluates that the said 2nd color is unsuitable. Color measuring device. The storage means stores color material information corresponding to the second color,
The color calculation means further comprises color material calculation means for calculating a color material corresponding to the second color based on the color material information in the storage means. The color measuring device according to item. An input means for measuring the color of the subject and inputting measurement color information;
Color measurement means for acquiring a first color from the measurement color information input from the input means, and calculating a second color to be added to the subject based on the first color;
Output means for outputting the second color calculated by the color measuring means,
A color measurement system using the color measurement device according to claim 1 as the color measurement unit. The input means is
Optical system,
A light receiving means for converting optical information collected by the optical system into an electrical signal;
A plurality of optical bandpass filters disposed near the stop of the optical system and having different spectral transmittances;
A lens array arranged between the optical system and the light receiving means, and a plurality of lenses arranged substantially parallel to a two-dimensional plane direction of the light receiving means;
Correction means for correcting the electrical signal for each light receiving position of the light receiving means;
With
The color measurement system according to claim 13, wherein the optical bandpass filter is a color filter corresponding to a tristimulus value of color. A color measurement method performed by the color measurement device according to any one of claims 1 to 12,
Obtaining measurement color information of the subject input from an input means for measuring the color of the subject, and obtaining first color information of the subject from the measurement color information;
Obtaining reference color information from storage means;
Calculating a second color to be added to the subject based on the first color information and the reference color information.   A program for causing a computer to execute the color measurement method according to claim 15.