JP2019028255A - Conversion lens unit and state measurement system - Google Patents

Abstract

To provide a conversion lens unit and state measurement system, capable of imaging skin in a substantially same way as the other persons even in a person unfamiliar with imaging and preventing image color imaged in a situation of a camera model used for imaging and external light from being affected.SOLUTION: A conversion lens unit 10 attached to a smart phone 1 comprises: a conversion lens 11; a contact part 20b contacting a skin 24 and having an opening 20c; and a lens housing 20 surrounding the conversion lens 10 and blocking external light from except the opening 20c to a master lens; a substantially white LED 12 lighting a portion facing on the inside of the lens housing 20 from the opening 20c in the state of contacting the skin 24 to the contact portion 20b in the lens housing 20; and a color scale including three or more kinds of reference colors. A distance between the contact surface of the contact part 20b and the scale surface of the color scale 41 is equal to or less than a distance of the range of the depth of field of the conversion lens 11.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a conversion lens unit and a state measurement system used for photographing a color image of a subject.

Conventionally, a staff member of a cosmetic company store or the like has used a dedicated machine for skin quality measurement to measure the skin quality, that is, the skin condition. Such skin quality measurement includes photographing skin and measuring skin texture state, pore state, stain state, and skin color state from the photographed image. There seem to be many people who want to know their skin condition, but there are not necessarily many people who measure skin quality at stores. However, for example, when considering the skin color, if a skin color database of many people who are not engaged in a doctor is created, it will be possible to grasp the physical condition of each individual and have extremely useful knowledge to aim for a healthier life. It is thought that it is obtained.
For example, in the example of the relationship between skin color and health condition, there is a possibility of jaundice or glycation due to liver function if the skin of the eyelid is yellow. The possibility of diabetes is considered, and if the skin of the eyelid is blue, the blood circulation may be poor. Therefore, if we can collect a lot of data from people before going to the doctor, it will serve as a guideline for examining the process leading to the deterioration of the health condition, and thus contribute to the promotion of the health of many people. It is also expected from the person.
Therefore, a service has been proposed in which skin is photographed using a smartphone's built-in camera, and the image is transmitted to a predetermined site using the smartphone's wireless communication function to determine the skin quality. Here, when the skin is photographed in color, the same subject depends on the characteristics of the camera lighting, image sensor (imaging device), integrated circuit for image processing, the ambient environment such as outside light, and the surface reflection state of light. However, it is necessary to correct the color and output the same color as substantially the same color. In particular, since there are various models of smartphones and each model is updated to a new model in a relatively short period of time, for example, shooting was performed in comparison with the case of using a dedicated camera for skin quality measurement. The image quality including the color of the image varies depending on the user.

For example, the reason why the color of the image varies due to shooting is based on the difference in the color filter of the image sensor of each camera, the difference in the spectrum of the illumination light, and the difference in the automatic white balance method of the camera image processing. And those based on the difference in the state of reflected light on the skin surface, and it is difficult to match colors at the photographing stage.
Therefore, even if a skin image taken with a smartphone is sent as it is to a site that diagnoses skin images from a smartphone, it is difficult to make a judgment based on the skin color.

Therefore, color samples with multiple types of reference colors for correcting the color of the captured image are displayed in the image on the enlargement imaging unit that is attached to a camera-equipped mobile phone and captures an enlarged photograph of the subject. It has been proposed to arrange so as to be embedded (see, for example, Patent Document 1).
Also, in order to measure the skin condition with the smartphone camera image, the same subject color will be almost the same color on any smartphone, and the subject color will be the same even if the lighting situation changes It has been proposed to shoot a color frame having various types of reference colors together with the skin and perform color correction based on the reference color on the captured image (see, for example, Non-Patent Document 1).

Japanese Patent No. 4175502

Koichi Tezuka, Shoji Suzuki, Masayoshi Shimizu “Skin condition measurement technology using smartphones” magazine FUJITSU, 2013-January issue (VOL.64, NO.1) 59-65

  By the way, when a general person who is not an expert photographs skin for skin quality measurement with a built-in camera of a smartphone, it is preferable that the photographing conditions are substantially the same regardless of who is photographing. Furthermore, it is preferable that the photographing conditions are always the same when photographing is performed almost regularly. In this case, a lens unit for skin shooting (conversion lens unit) is attached to the camera of the smartphone, the tip surface of the lens unit is in contact with the skin, and there is an opening through which the optical axis of the lens system for shooting passes. It is preferable to provide a contact portion, a shielding portion that blocks external light from the contact portion to the camera, and a light source that illuminates the skin exposed in the shielding portion from the opening. This makes it possible to shoot with the same illuminance at the same distance from the skin no matter who shoots during skin shooting. In this case, for example, by arranging a color scale having a plurality of reference colors so as to be reflected in the skin image in the shielding portion, the color can be corrected.

  However, smartphone cameras are basically autofocused, and the autofocus control system of mirrorless digital cameras such as smartphones works to focus on areas where the contrast is high due to strong reflected light. Become. When a color scale is provided in the photographing range in the shielding part, the reflected light from the “color scale” may have a higher contrast than the “facial skin” depending on the darkness of the skin color. Therefore, there is a possibility that the focus position is not on the skin but on the brightest illuminated portion of the color scale light source. In this case, the subject is slightly out of focus with respect to the skin, and the image quality of the skin image is degraded, which may hinder measurement of the skin condition.

  The present invention has been made in view of the above circumstances, and even a person who is not accustomed to shooting can shoot skin in the same manner as other people, and also shoots in the camera model used for shooting and the situation of external light. It is an object of the present invention to provide a conversion lens unit and a state measurement system in which the color of a displayed image is not easily affected.

In order to solve the above-described problems, a conversion lens unit according to the present invention guides reflected light from a subject to the master lens before a master lens of the camera of a mobile terminal that includes a camera and has a wireless communication function. A conversion lens unit for attaching a lens system of a conversion lens,
The lens system;
An abutting portion that abuts the subject and has an opening through which the optical axis of the lens system passes;
A shielding portion that surrounds the lens system and shields outside light from other than the opening with respect to the master lens;
A substantially white light that is disposed outside the field of view of the lens system in the shielding portion and illuminates a portion of the subject facing the inside of the shielding portion from the opening in a state where the subject is in contact with the contact portion. The light source of
A color scale provided in the field of view within the shielding portion and having three or more reference colors;
The distance along the optical axis direction of the lens system between the contact surface of the contact portion that contacts the subject and the scale surface on which the reference color of the color scale is displayed is the depth of field of the lens system. The distance is equal to or less than the distance within the range.

  According to such a configuration, since the portion facing the opening of the contact portion of the subject is photographed in a state where the subject is in contact with the contact portion, the subject and the camera no matter who the subject is photographed at any time. And the subject is illuminated with a substantially white light source with outside light blocked by the shield, so the lighting conditions such as the illuminance of the subject will be the same regardless of who shoots can do. In such a state, the reference color of the scale surface of the color scale in the field of view of the lens system is reflected in the image obtained by photographing the subject together with the subject. Therefore, the subject and the reference color are shown in the image, and the color correction of the subject can be performed based on the reflected reference color. As a result, in the image after color correction, the same color is substantially the same regardless of who the subject was photographed at any time. Thereby, the state of the subject can be determined by the color. For example, when the subject is skin, the skin state and health state can be determined from the skin color. In the determination, for example, different skin colors are registered depending on the skin condition, health condition, tanning condition, etc., and the skin condition is determined by which color the photographed skin color approximates. And health status.

  The lens system of the conversion lens is one or a plurality of lenses arranged in front of the master lens as a front type conversion lens. The field of view of the lens system is the range in the light space that passes through the lens system and reaches the master lens. Here, it is the range that appears in the captured image of the camera with the conversion lens attached, and the field of view of the lens system. If it is within, it will appear in the image taken by the camera with the conversion lens attached, and if it is outside the field of view, it will not appear. Therefore, the light source does not appear directly in the captured image. The color scale within the field of view of the lens system is reflected in the captured image. The color scale is preferably disposed outside the opening at a position where it appears in the photographed image so as not to interfere with the subject. The contact surface and the scale surface are preferably surfaces orthogonal to the direction of the optical axis of the lens system.

  The depth of field refers to a range of distance on the subject side where the photograph appears to be in focus. In a photographic lens, the exact focus in a single setting is only on one plane, but by allowing a certain amount of tolerance, the image is clearly and clearly formed before and after that. There is a range that can be said to be, and this range is called depth of field.

In the configuration of the present invention, it is preferable that a reflected light control unit that suppresses the influence of the unevenness of the subject surface from the reflected light of the subject surface is provided.
According to such a configuration, it is possible to observe under a so-called non-reflective condition without being affected by the unevenness of the surface shape of the subject to be photographed. In that case, color calibration can be performed with high accuracy without being affected by the reflected light that appears whitish from the surface of the subject and the color scale, and in particular, clear color information can be obtained. If so, a very accurate color calibration can be performed. The reflected light control unit uses, for example, polarized light, which will be described later. However, an imaging jelly for refractive index matching may be used, or shadowless illumination used in a dermoscope may be used.

  In the configuration of the invention, the reflected light control unit transmits a first polarizing plate that transmits light from the light source toward the subject from the light source, and reflected light from the subject that travels from the subject toward the master lens. It is preferable that the polarization direction of the first polarizing plate is orthogonal to the polarization direction of the second polarizing plate.

  According to such a configuration, the light polarized by the first polarizing plate is reflected by the surface of the subject and is reflected toward the second polarizing plate. However, the polarized light is maintained by the reflected light, and the first polarizing plate Reflected light due to surface irregularities can be suppressed without passing through the second polarizing plate whose polarization direction is orthogonal. Note that part of the light, for example, slightly enters the surface of the skin as the subject and is reflected to produce reflected light that reflects the skin color. Even if the reflected light is suppressed, the color of the skin Can be copied.

The state measurement system of the present invention includes the conversion lens unit having the above configuration, the mobile terminal having the wireless lens function and the camera having the master lens to which the conversion lens of the conversion lens unit is mounted in front.
Color correction that is provided in the portable terminal or an external device that can communicate with the portable terminal, and that performs color correction of the image of the subject photographed by the camera based on the reference color of the color scale captured in the image And a state determining means for determining the state of the subject from the color of the subject on the color-corrected image.

  According to such a configuration, the distance to the subject is the same regardless of who takes the image, the color condition is reflected under the same lighting conditions of the subject, and an image in which both the subject and the color scale are in focus is obtained. This image can be color calibrated on a color scale to determine the state of the subject. For the determination of the state of the subject, for example, color data or an image for each state of the subject is registered, and the determination can be made by comparing with the registered data.

  According to the present invention, the subject can be photographed under substantially the same conditions regardless of when and by which camera, and the color of the photographed image can be calibrated based on the same reference color, so that the state of the subject can be determined under substantially the same conditions.

It is a block diagram which shows the state measurement system (skin quality measurement system) of embodiment of this invention. It is a figure which shows schematic structure of a conversion lens unit. It is a front view showing a color scale. It is a figure which shows the relationship between the imaging | photography range of a conversion lens unit, and an image pick-up element similarly. It is a figure explaining how to obtain the depth of field. It is a figure which shows the color scale part of a conversion lens unit similarly. It is a front view which shows the color scale of a different example similarly.

A conversion lens unit and a state measurement system according to an embodiment of the present invention will be described.
As shown in FIG. 1, a skin quality measurement system that is a state measurement system according to the present embodiment includes a camera (master lens) 2, a wireless communication device (not shown), and a control means 3 as an arithmetic processing unit. As a smartphone 1 and a conversion lens unit 10 attached to the smartphone 1. In addition, it is good also as what performs the below-mentioned color correction (color calibration) of the image image | photographed with the camera 2 of the smart phone 1 using the conversion lens unit 10, the process of determination of skin quality, etc. by the control means 3 of the smart phone 1. As an external device capable of data communication with the smartphone 1, for example, data such as the above-mentioned image is transmitted from the smartphone 1 to a server (computer system) connected to the Internet, and color correction and skin quality determination are performed by the server. The determination result of the skin quality or the like may be transmitted from the server to the smartphone 1. In this case, data transmitted from each smartphone 1 or data generated on the server side based on the data is managed by the server, and data is transmitted to the smartphone 1 in response to a request from the smartphone 1 under a well-known authentication process. It may be transmitted, or data on skin quality may be provided to research institutions. In the following description, color correction and skin quality determination processing are performed by the smartphone 1, but part or all of the processing after image capturing may be performed by the server.

  As shown in FIGS. 1 and 2, the conversion lens unit 10 is attached to a portion in front of the master lens (not shown) of the camera 2 of the smartphone 1 and is used for close-up photography of the skin 24 as a subject. A conversion lens 11 that is a lens system having lenses 11a and 11b (shown in FIG. 2) and LEDs 12 and 13 for photographing illumination are provided.

  1 and 2, the conversion lens unit 10 is a front portion of the housing 10 a except for a later-described lens housing 20 that includes a barrel 20 a (shown in FIG. 2) that supports the conversion lens 11. Although not shown, the circuit board 16b on which the internal battery 17, the power switch 19 and the like are mounted is shown.

  The conversion lens unit 10 includes a housing 10a. The housing 10a is formed in a flat box shape except for the lens housing 20, and is opposite to the front side where the display (not shown) of the smartphone 1 is provided. The conversion lens 11 is arranged so as to overlap the lens portion of the camera 2 on the back side.

  The case 10a is fixed to the smartphone 1 by, for example, a clip type or a band type. In the clip type, for example, the conversion lens unit 10 is provided with a clip member (not shown) that sandwiches the smartphone 1 with the conversion lens unit 10. That is, the smartphone 1 is sandwiched by an elastic force between the conversion lens unit 10 and the clip member fixed to the conversion lens unit 10.

In this case, the conversion lens unit 10 can be moved vertically and horizontally within an allowable range with respect to the back surface of the smartphone 1, and can correspond to the arrangement of the cameras 2 of each model of the smartphone 1.
In the band type, for example, a band (not shown) such as a rubber band is attached to the conversion lens unit 10 like a wristwatch band, and the smartphone 1 is inserted into the band. In this case, the conversion lens 11 can be moved vertically and horizontally with respect to the back surface of the smartphone 1.

The conversion lens unit 10 includes the conversion lens 11 and the LEDs 12 and 13 described above, an LED drive circuit 14 (shown in FIG. 1) for driving the LED 12, and an LED drive circuit 15 (shown in FIG. 1) for driving the LED 13. .
The conversion lens unit 10 includes a lens housing 20 in which the conversion lens 11 is accommodated. The lens housing 20 includes a barrel 20 a that supports the conversion lens 11 on the base end side (the side that is attached to the camera 2), and the front end surface is a contact portion 20 b that contacts the skin 24 when the skin 24 is photographed. The outside light is blocked while the contact portion 20 b is in contact with the skin 24. In addition, a circular opening 20c is formed in the contact portion 20b, and a portion of the skin 24 facing the opening 20c is photographed. Therefore, the lens housing 20 is a shielding portion that surrounds the conversion lens 11 as a lens system and blocks external light from other than the opening 20 c against the master lens of the camera 2.

  As shown in FIGS. 3 to 5, the barrel 20 a supports the conversion lens 11, which is a lens system including the lens 11 a and the lens 11 b, and a base end portion (the side attached to the camera 2) of the lens housing 20. Is fixed in the end). Moreover, the below-mentioned 2nd polarizing plate 22 is provided in the base end part of the barrel 20a. In addition, an LED substrate 12a (shown in FIG. 1) including LEDs 12 and 13 that emit illumination light for photographing toward the opening 20c of the contact portion 20b is supported in the lens housing 20.

  The LEDs 12 and 13 are provided on the LED substrate 12a. Further, as will be described later, there is one LED 13 for photographing the texture of the skin 24 and two LEDs 12 for photographing the stain on the skin 24. The light from the two LEDs 12 on the LED substrate 12a passes through a first polarizing plate 21 described later and illuminates the skin 24 from the opening 20c. Further, the LED drive circuits 14 and 15 are also provided on the LED substrate 12a. The LED drive circuits 14 and 15 may be provided on the circuit board 16b.

  As shown in FIG. 1, the conversion lens unit 10 includes a battery 17 that supplies power to the LED drive circuits 14 and 15 and the CPU 16 on a circuit board 16b (shown in FIG. 2) fixed to the housing 10a. A power supply circuit 18 and a power switch 19 are provided.

  The two LEDs 12 and 13 described above are, for example, white LEDs, but need not be completely white. The two types of LEDs 12 and 13 are an LED 13 for photographing a skin texture and an LED 12 for photographing a skin stain and color, and may be slightly shifted from white depending on the purpose.

  Further, the first polarizing plate 21 is provided in the spot photographing LED 12, and the second polarizing plate 22 is provided between the conversion lens 11 and the camera 2. This is for photographing a stain slightly below the surface of the skin 24, and the polarizing plate 21, 22 causes the skin 24 to reflect the reflected light inside the skin 24 when photographing the stain on the skin 24 with the LED 12. The reflected light on the surface is greatly reduced to make it easier to photograph spots. In the present embodiment, the reference color of the color scale 41 is captured in the image, and the color of the image captured with the reference color is corrected as will be described later. Correction accuracy can be increased by performing correction with an image in which surface reflection of the color scale is suppressed. In the first polarizing plate 21 and the second polarizing plate 22, the polarization directions are orthogonal. The light of the LED 12 passes through the first polarizing plate 21 and is polarized, and this polarized light is maintained even if it is reflected by the skin 24, and is difficult to pass through the second polarizing plate 22. On the other hand, the light reflected inside the skin 24 is not polarized and passes through the second polarizing plate 22. Thereby, the 1st polarizing plate 21 and the 2nd polarizing plate 22 become a reflected light control part which suppresses the influence by the fine unevenness | corrugation of this skin 24 surface from the reflected light of the skin 24 surface. For example, the reflected light control unit can prevent the light from the LED 12 from appearing whitishly reflected on the surface of the skin 24. Note that the polarizing plates 21 and 22 may not be provided.

  As shown in FIGS. 1 and 2 and FIG. 6 which is an enlarged view of the front side of the lens housing 20 on the subject side, the opening 20c of the abutting portion 20b of the lens housing 20 has a peripheral portion thereof. An annular color scale 41 is provided. The substantial opening 20c of the contact portion 20b is an opening of the annular color scale 41, and is annular around the skin 24 facing from the opening 20c when the conversion lens unit 10 is used to photograph with the camera 2. The color scale 41 is reflected. The color scale 41 is disposed along a plane orthogonal to the optical axis of the conversion lens 11 and is orthogonal to the optical axis. The surface on the conversion lens 11 side of the color scale 41 is a scale surface on which at least three kinds of reference colors are arranged.

The distance L along the optical axis direction of the conversion lens 11 between the contact surface that contacts the skin of the contact portion 20 b and the scale surface of the color scale 41 is equal to or less than the distance that is within the range of the depth of field of the conversion lens 11. It has become. Note that the distance L that is the range of the depth of field is within a range in which the skin 24 and the color scale 41 are in focus when the camera 2 of the smartphone 1 to which the conversion lens unit 10 is attached is photographed. It is sufficient if it is a distance. The skin 24 is highly likely to be curved so as to be a convex surface, and the skin 24 may be closer to the color scale 41 than the contact surface of the contact portion 20b. The distance from the tangent surface to the scale surface may be corrected, and finally the distance between the skin surface and the scale surface along the optical axis of the conversion lens 11 may be within the range of the depth of field.
In the color scale 41, the scale surface is divided into three regions as shown in FIG. 3, and each region is colored with a different reference color. In this example, for example, red, Green and blue are used. The reference color is not limited to these, and for example, a color close to skin color may be used. In the present embodiment, the color scale 41 is provided with a red reference color region 41R, a green reference color region 41G, and a blue reference color region 41B.

  In such a conversion lens unit 10, under the control of the CPU 16 of the circuit board 16b, the LED 13 is caused to emit light (turn on) and the LED 12 remains off during photographing, so that when photographing spots and skin color. The LED 12 is caused to emit light and the LED 13 is kept off.

  As is well known, the smartphone 1 has a function as a mobile phone and can make a call using a mobile phone line. The smartphone 1 has a display (not shown) and can be connected to the Internet via a mobile phone line or WiFi, and data communication is possible via the Internet.

In addition, the smartphone 1 includes a control unit 3 having a CPU, a ROM, a RAM, and the like, and can execute an application as a program. The control unit 3 is connected to a flash memory 4 as a storage device, and can transmit data stored in the flash memory 4 and store received data.
In addition, the control unit 3 can control the camera 2 to shoot (capture) and store the captured image data in the flash memory 4. Image data can be analyzed and processed by an application for image processing and image analysis. Here, it is possible to determine the skin condition based on the color correction using the color scale 41 of the image and the corrected skin color.
The camera 2 is controlled by the control unit 3 and has an image sensor, and can capture still images and moving images. For example, the camera 2 captures images at a set frame rate such as 30 fps.

  Further, the control means 3 is a lighting / extinguishing detection means 3a for detecting the lighting and extinguishing of the LEDs 12 and 13, and when the lighting and extinguishing of the respective LEDs 12 and 13 are detected by the lighting / extinguishing detection means 3a, And a display control means 3c for displaying a photographed image photographed by the camera 2 under the control of the photography control means 3b in association with the LEDs 12 and 13 on the screen of the smartphone 1.

  Further, the control means 3 performs color correction of the image of the skin 24 photographed by the camera 2 based on the reference color of the color scale 41 captured in the image and the skin color on the corrected image. It functions as a state determining means for determining the skin state. In addition, the color correction unit and the state determination unit may be arranged not on the smartphone 1 but on, for example, a server as an external device capable of data communication with the smartphone 1. Communication is possible. The server transmits the image photographed by the camera 2 from the smartphone 1 to the server in a state where the smartphone 1 accessed via the Internet is authenticated, performs color correction and skin condition determination at the server, and displays the determination result on the smartphone 1. Send to.

Next, how to obtain the depth of field of the conversion lens 11 in the present embodiment will be described. FIG. 4 illustrates a state in which an imaging range 43 that is an area having a diameter of about 10 mm to be photographed facing the opening 20c of the contact portion 20b is imaged on the effective pixel area 42 of the imaging device of the camera 2. For example, when a skin texture (size of about 100 μm or more) is to be observed, a pixel size corresponding to about 20 μm is desirable on the object to be photographed, but with this pixel size, a diameter of 10 mm corresponds to about 500 pixels. Therefore, the VGA (640 × 480) level is sufficient for the resolution required for the smartphone image pickup device, and a commercially available camera 2 of a normal smartphone 1 (having a high-resolution image pickup device of 1 million pixels or more) is used. Can do. In addition, with the electronic zoom function or the like, it is possible to set one pixel size on the digital photograph to about 20 μm regardless of the pixel size of the actual image sensor of the smartphone 1.
When one pixel size of the imaging range 43 is set to about 20 μm, the range that can be said to be in focus is a fine pattern with a period of 20 μm [a line with 10 μm width (black) and a space with 10 μm width (white) is one period]. It can be defined that the degree of modulation is 50% or more when captured.
When the pixel size of the imaging range 43 is A μm, the period of the fine pattern may be A μm. The distance L between the scale surface and the contact surface is set based on such a range of depth of field.

  As shown in FIG. 5, the specific depth of field is measured by, for example, a smartphone 1 having a camera 2 that can be fixed focus, a conversion lens unit 10 having a conversion lens 11, and the above-described fine pattern. The fine movement stage capable of moving the panel 44 in the direction of the optical axis of the conversion lens 11 and measuring the position of the panel 44 relative to the conversion lens 11 with respect to the smartphone 44 to which the conversion lens unit 10 is attached. 45 is used. The panel 44 can be moved into the opening 20 c of the lens housing 20. The camera 2 may be a digital camera other than the smartphone 1. By repeatedly moving the position of the panel 44 with the fine movement stage 45 and photographing, the range of the position of the panel 44 having a modulation degree of 50% or more is obtained.

Next, a color calibration method using the color scale 41 of the image will be described.
If the spectrum of illumination is substantially constant, practical skin color detection is possible with simple calibration. In this embodiment, since the skin 24 is illuminated with the light of the LED 12 as a light source while blocking external light, the LED 12 Even if there is some variation in the characteristics, the calibration is possible as described above. Some white LEDs have different white light emission methods, but the mainstream is to emit yellow phosphor with a blue LED and output white light in combination with blue light and yellow light. The emission spectrum of the yellow phosphor has a broad peak that spans the green wavelength band and the red wavelength band. In the present embodiment, the above-described white LED is used for each conversion lens unit 10, and in a captured image using each conversion lens unit 10, the illumination spectrum is substantially constant.

  First, as a standard environment, the conversion lens unit 10 having the color scale 41 is attached to a color calibrated camera, and the skin 24 (photographing object) is photographed while being shielded from light as described above. Average intensity calibration values Pr0, Pg0, Pb0 in each of the red reference color region 41R, the green reference color region 41G, and the blue reference color region 41B are calculated from the photographed photograph, and the calibration values are stored in advance in the server (cloud). ) Or stored in the flash memory 4 by the application of the smartphone 1. At this time, an image obtained by decomposing the captured image of the skin 24 and the color scale 41 into RGB components of red, green, and blue is created, and each reference color region 41R, 41G, 41B is extracted. Next, average intensities Pr, Pg, and Pb in the reference color regions 41R, 41G, and 41B are calculated and set as average intensity calibration values Pr0, Pg0, and Pb0.

Next, in each smartphone 1 to be color calibrated, an image obtained by decomposing the photographed skin 24 and color scale 41 images into red, green, and blue RGB components is created, and the image is decomposed into RGB components. The reference color regions 41R, 41G, 41B are extracted from the above. Next, average intensities Pr, Pg, and Pb in the reference color regions 41R, 41G, and 41B are calculated.
Next, in order to normalize the captured image for each component, the red component intensity of each pixel of the red component image is multiplied by (Pr0 / Pr). Similarly, the green component intensity of each pixel of the green component image is multiplied by (Pg0 / Pg), and the blue component intensity of each pixel of the blue component image is multiplied by (Pr0 / Pr).
Next, each color component normalized by the above-described method can be synthesized to obtain an accurate color-corrected image. Note that the color calibration method is not limited to this, and a known calibration method using a reference color reflected in an image can be used.

  As described above, in the present embodiment, the light from the LED 12 is irradiated to the skin 24 through the first polarizing plate 21, and the reflected light based on the light of the LED 12 from the skin 24 passes through the conversion lens 11 and is second. It reaches the camera 2 through the polarizing plate 22. Further, the first polarizing plate 21 and the second polarizing plate 22 are orthogonal in polarization direction, and can suppress variations in reflected light as described above. Therefore, when determining the skin state from the color of the skin 24, the color calibration is performed from the reference color of the color scale 41 reflected in the image using the image of the skin 24 that is captured by turning on the LED 12 instead of the LED 13. Do. Thereby, for example, an image in which the light of the LED 12 is reflected by an object having a glossy surface and the reflected light of the LED 12 appears as a bright spot on the object, or a part of the object is close to the image. It is possible to prevent the image from appearing whitish, correct the skin color with high accuracy, and acquire the skin color with high accuracy.

  For example, digital data indicating a color such as an RGB value indicating the color is obtained from the skin color-corrected image acquired as described above. In this case, as the representative value of the digital data such as the RGB value of the image, for example, the average value, the mode value, or the median value of the RGB values of each pixel of the entire image or the main part is used, or by other statistical processing. The obtained representative value may be used. Further, for example, as a skin color database, for example, for each RGB value as the skin color, a level or comment indicating the degree of whiteness or blackness of the skin or the degree of yellow, red, etc. other than white is provided. It is also possible to register and display the level and comment of each color corresponding to the representative value of the RGB value of the image as the skin state determination process. Note that the database is stored in the flash memory 4 of the smartphone 1 or the storage of the server. Moreover, the determination process of the skin state may be performed by the smartphone 1 or by a server.

According to the conversion lens unit 10 and the skin condition measurement system of this embodiment, the skin condition is determined from the skin color of the image obtained by photographing the skin with the camera 2 of the smartphone 1 using the conversion lens unit 10. When the color scale 41 having three or more reference colors is reflected in the photographed image, the color of the image can be calibrated using the reference colors. In this case, the auto-focus function of the camera 2 of the smartphone 1 may focus on the color scale, not the skin, and the subject may be out of focus from the skin 24 as a subject. Since the distance along the optical axis direction of the conversion lens 11 between the contact surface of the contact portion 20b and the scale surface of the color scale 41 is within the depth of field, the color scale 41 is obtained by autofocus. Even if the scale surface is focused, the skin 24 as the subject can also be focused. Thereby, when the color scale 41 is reflected in the image together with the subject, it is possible to prevent the focus from being shifted from the subject by autofocus.
From the above, by using the conversion lens unit 10 of the present embodiment at the time of shooting, the distance between the subject and the camera 2 is constant and the lighting state is constant regardless of who is shooting with which camera 2 of the smartphone 1. In addition, it is possible to shoot in a state where both the color scale 41 and the skin 24 are in focus, and the captured color can be corrected with a simple method with high accuracy. Further, when a subject of the same color is photographed, it is possible to obtain an image of the subject of substantially the same color no matter who or who is photographing with which camera 2 of the smartphone 1. Thereby, when the color of the skin 24 is calculated | required from the above-mentioned image and the state of skin is determined from the color of this skin 24, determination with higher precision can be performed.

In the present embodiment, two polarizing plates 21 and 22 having different polarization directions function as a reflected light control unit that suppresses the influence of unevenness on the subject surface from the reflected light on the subject surface.
Further, as the reflected light control unit for suppressing direct reflection on the subject, the above-described photographing jelly for refractive index matching may be used in addition to the polarizing plate, or the shadowless illumination used in the dermoscope may be used. It may be used. Further, the shape of the color scale 41 is not limited to that shown in FIG. 3, and three or more kinds of reference colors are displayed. The color scale 41 does not block shooting of a necessary part of the subject and is close to the subject. It suffices if the reference color is within the range of the above-mentioned depth of field. For example, when the lens housing 20 has a square cross section, the color scale 51 having the shape shown in FIG. 7 may be used. The color scale 51 is provided with a red reference color region 51R, a green reference color region 51G, and a blue reference color region 51B, which are arranged at the same position as the color scale 41 and used in the same manner. The Further, the subject is not limited to the skin 24, and may be various living bodies such as plant leaves and fruits, and the degree of growth and the degree of ripeness can be determined from the color.

1 Smartphone (mobile terminal with wireless communication function)
2 Camera 3 Control means (color correction unit, state determination means)
10 Conversion lens unit 11 Conversion lens 11a Lens (lens system)
11b Lens (lens system)
12 LED (light source)
20 Lens housing (shielding part)
20b Contact part 20c Opening part 21 1st polarizing plate (reflected light control part)
22 Second polarizing plate (reflected light control unit)
41 color scale

Claims (4)

A conversion lens unit for attaching a lens system of a conversion lens that guides reflected light from a subject to the master lens before the master lens of the camera of a mobile terminal that includes a camera and has a wireless communication function,
The lens system;
An abutting portion that abuts the subject and has an opening through which the optical axis of the lens system passes;
A shielding portion that surrounds the lens system and shields outside light from other than the opening with respect to the master lens;
A substantially white light that is disposed outside the field of view of the lens system in the shielding portion and illuminates a portion of the subject facing the inside of the shielding portion from the opening in a state where the subject is in contact with the contact portion. The light source of
A color scale provided in the field of view within the shielding portion and having three or more reference colors;
The distance along the optical axis direction of the lens system between the contact surface of the contact portion that contacts the subject and the scale surface on which the reference color of the color scale is displayed is the depth of field of the lens system. A conversion lens unit characterized in that the distance is equal to or less than the distance within the range.   The conversion lens unit according to claim 1, further comprising a reflected light control unit that suppresses the influence of the unevenness of the subject surface from the reflected light of the subject surface.   The reflected light control unit includes: a first polarizing plate that transmits light from the light source toward the subject from the light source; and a second polarizing plate that transmits reflected light from the subject toward the master lens from the subject. The conversion lens unit according to claim 2, wherein the polarization direction of the first polarizing plate and the polarization direction of the second polarizing plate are orthogonal to each other. The mobile phone comprising the conversion lens unit according to any one of claims 1 to 3 and the camera having the master lens to which the conversion lens of the conversion lens unit is attached in front and having a wireless communication function. A terminal,
Color correction that is provided in the portable terminal or an external device that can communicate with the portable terminal, and that performs color correction of the image of the subject photographed by the camera based on the reference color of the color scale captured in the image And a state determination unit that determines the state of the subject from the color of the subject on the image subjected to color correction.