JP5561520B2 - Image processing apparatus, image processing method, program, and electronic apparatus - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, a program, and an electronic device, and in particular, an image processing device that can detect, for example, a portion where skin such as a human hand is exposed, based on a photographed image. The present invention relates to an image processing method, a program, and an electronic apparatus.

  There is a skin detection technique for detecting a region where skin is exposed (hereinafter referred to as a skin region) such as a face or a hand from a captured image obtained by capturing a person (see, for example, Patent Document 1). ).

  In this skin detection technology, a first image obtained by imaging a subject (person) irradiated with an LED (light emitting diode) that outputs light of wavelength λ1 and light of wavelength λ2 different from wavelength λ1 are output. A second image obtained by capturing an image of a subject illuminated by the LED is acquired. Then, an area where the difference in luminance between the first image and the second image is larger than a predetermined threshold is detected as a skin area.

  The wavelengths λ1 and λ2 are determined using the reflection characteristics of human skin.

  FIG. 7 shows reflection characteristics with respect to human skin. This reflection characteristic has generality regardless of the color difference (racial difference) or the state (sunburn etc.) of the human skin. In the figure, the horizontal axis indicates the wavelength of the irradiation light irradiated on the human skin, and the vertical axis indicates the reflectance of the irradiation light irradiated on the human skin.

  It is known that the reflectance of irradiated light radiated to human skin decreases rapidly from around 900 nm with a peak at around 800 nm, and rises again around 1000 nm as a local minimum. Specifically, for example, the reflectance of reflected light obtained by irradiating human skin with 870 nm light is 63%, and the reflectance of reflected light obtained by irradiating 950 nm light is 50%. %.

  This is peculiar to human skin, and for objects other than human skin (for example, hair, clothes, etc.), the change in reflectance is moderate in the vicinity of 800 to 1000 [nm]. Many.

  Using such reflection characteristics, for example, the wavelength λ1 is a value of 800 to 900 nm, and the wavelength λ2 is a value of 900 to 1000 nm.

JP 2006-47067 A

  By the way, it is known that the LED which is an irradiation light source changes in the light emission wavelength and the light output as the ambient temperature changes. Therefore, when the emission wavelength or light output of the LED, which is the irradiation light source, changes as the ambient temperature changes, the sensitivity of the imaging means and the irradiance on the subject (hereinafter also simply referred to as `` illuminance '') change. Even in a region, the difference in luminance between the first image and the second image may become small and may not be detected as a skin region. On the other hand, even if it is not a skin region, a difference in luminance between the first image and the second image may increase and may be erroneously detected as a skin region.

  The present invention has been made in view of such circumstances, and can detect a skin region with high accuracy even when the light emission wavelength or light output of an LED, which is an irradiation light source, changes as the ambient temperature changes. It is to make.

An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that detects a skin region representing human skin from an image, and irradiates a subject with light having a first wavelength and a light having a first wavelength. A first irradiating means, a second irradiating means for irradiating the subject with light having a second wavelength longer than the first wavelength, and the subject with the light having the first wavelength. A first image is generated based on the reflected light from the subject when the second light is emitted, and a second image is generated based on the reflected light from the subject when the subject is irradiated with light of the second wavelength. A generating unit configured to generate an image; a detecting unit configured to detect the skin region based on a comparison result obtained by comparing a difference between the first image and the second image with a threshold; and corresponding to the measured temperature The first or the second irradiation hand so as to have an illuminance ratio If to change the output of at least one of the illumination light, or, and a temperature corresponding means for changing the threshold value in response to said measured temperature.

When the measured temperature is higher than a reference temperature, the temperature corresponding unit is configured to increase the illuminance ratio obtained by dividing the illuminance of the second irradiation unit by the illuminance of the first irradiation unit. or change the output of at least one of the irradiation Shako the second irradiation means, when said measured temperature is lower than the reference temperature, of the so said illuminance ratio decreases first or second irradiation means change the output of at least one of the irradiation Shako, the first and second radiation means in accordance with the output of the irradiation beam irradiated is changed to irradiate the first wavelength or the light of the second wavelength to the subject Can be.

  The temperature corresponding means changes the direction to increase the threshold when the measured temperature is higher than the reference temperature, and decreases the threshold when the measured temperature is lower than the reference temperature. The detection means can detect the skin region based on a comparison result with the calculated difference-changed threshold value.

  The first wavelength may be less than 930 nm, and the second wavelength may be 930 nm or more.

The image processing method according to the first aspect of the present invention includes a first irradiating unit that irradiates a subject with light having a first wavelength, and light having a second wavelength longer than the first wavelength. Generating a first image based on reflected light from the subject when the subject is irradiated with light of the first wavelength and the second wavelength. Generating means for generating a second image based on reflected light from the subject when the subject's light is irradiated to the subject, and comparing a difference between the first image and the second image with a threshold value In the image processing method of the image processing apparatus comprising the detection means for detecting the skin region based on the comparison result obtained, a measurement step for measuring the temperature by the image processing apparatus, and an illuminance ratio corresponding to the measured temperature The first or the first Change the output of at least one of the irradiation light of the irradiation means, or includes a temperature corresponding step of changing the threshold value in response to said measured temperature.

A program according to a first aspect of the present invention is configured to irradiate a subject with a first irradiating unit that irradiates a subject with light having a first wavelength, and a light having a second wavelength longer than the first wavelength. A second irradiating means for generating a first image based on reflected light from the subject when the subject is irradiated with light having the first wavelength, and light having the second wavelength. An illuminance ratio corresponding to a detected temperature in a program for controlling an image processing apparatus comprising: generating means for generating a second image based on reflected light from the subject when the subject is irradiated with A temperature corresponding step of changing the output of the irradiation light of at least one of the first or the second irradiation means, or changing the threshold according to the measured temperature, Difference between the image and the second image To execute processing including a detection step of detecting a skin region on the basis of a result of comparison with the threshold value to the computer of the image processing apparatus.

In the first aspect of the present invention, the output of at least one of the first and second irradiation means is changed or measured so that the illuminance ratio corresponds to the detected temperature. The threshold value to be compared with the difference is changed according to the measured temperature .

An electronic device according to a second aspect of the present invention includes a measuring unit that measures temperature, a first irradiating unit that irradiates a subject with light having a first wavelength, and a second that has a longer wavelength than the first wavelength. A second irradiating means for irradiating the subject with light of a wavelength of the first and a first image based on the reflected light from the subject when the subject is irradiated with light of the first wavelength. And generating means for generating a second image based on reflected light from the subject when the subject is irradiated with light of the second wavelength, and the first image and the second image. And detecting means for detecting the skin region based on a comparison result obtained by comparing a difference between the threshold value and a threshold value, and at least the first or the second irradiating means so as to obtain an illuminance ratio corresponding to the measured temperature. change the output of one of the illumination light, or was determined A temperature corresponding means for changing the threshold depending on the serial temperature, in response to changes in the detected skin region including an operation control means for performing a predetermined process.

In the second aspect of the present invention, the output of the irradiation light of at least one of the first or second irradiation means is changed or measured so that the illuminance ratio corresponds to the measured temperature. The threshold value to be compared with the difference is changed according to the measured temperature . In addition, a predetermined process is executed according to the detected change in the skin area.

  According to the present invention, a skin region can be detected with high accuracy even when the light emission wavelength or light output of an LED, which is an irradiation light source, changes as the ambient temperature changes.

It is a block diagram which shows the structural example of the detection apparatus to which this invention is applied. It is a figure which shows the spectral sensitivity characteristic of an image pick-up element. It is a figure which shows the change of the illumination intensity ratio of LED with respect to the change of environmental temperature. It is a flowchart explaining a 1st skin area | region detection process. It is a figure which shows the change of the reflectance difference detection signal resulting from the change of ambient temperature. It is a flowchart explaining a 2nd skin area | region detection process. It is a figure which shows the reflective characteristic of a human skin.

  Hereinafter, the best mode for carrying out the invention (hereinafter referred to as an embodiment) will be described in detail with reference to the drawings.

<1. Embodiment>
[Configuration example of detection device]
FIG. 1 shows a configuration example of a detection apparatus according to an embodiment of the present invention. Even when the emission wavelength or light output of the LED, which is the irradiation light source, changes with changes in the ambient temperature, the detection device 10 detects the human skin region (for example, face, Hand, etc.) can be detected.

  The detection device 10 includes a control unit 11, an LED control unit 12, a light source group (LED) 13-1, a light source group (LED) 13-2, an imaging unit 14, an imaging control unit 15, an image processing unit 16, and a temperature detection unit. 17.

  The control unit 11 controls the overall operation of each unit of the detection apparatus 10. The control unit 11 controls the LED control unit 12 based on the temperature information input from the temperature detection unit 17 to set the illuminance ratio between the light source group 13-1 and the light source group 13-2, or image processing. The unit 16 is controlled to set a threshold (described later).

  In accordance with control from the control unit 11, the LED control unit 12 turns on and off the light sources 13-1 and 13-2, and the light output level (that is, the illuminance ratio between the light sources 13-1 and 13-2). ) To control.

  The light source group 13-1 includes a plurality of LEDs, and emits light having an emission spectrum half-width of about 50 nm and a peak wavelength of the emission spectrum of λ1 (hereinafter referred to as light of wavelength λ1) according to the control of the LED control unit 12. Emits light. The light source group 13-2 includes a plurality of LEDs, and emits light having an emission spectrum half-width of about 50 nm and a peak wavelength of the emission spectrum of λ2 (hereinafter referred to as light of wavelength λ2) according to the control of the LED control unit 12. Emits light. The value of wavelength λ1 at room temperature 25 ° C. is 680 nm to 930 nm, the value of wavelength λ2 longer than wavelength λ1 is 930 nm to 1100 nm, more preferably the value of wavelength λ1 at room temperature 25 ° C. is 780 nm to 930 nm, The value of the wavelength λ2 may be 930 nm to 1020 nm.

  Note that the light source groups 13-1 and 13-2 have characteristics that the emission wavelength and the light output change with changes in ambient temperature (details will be described later with reference to FIG. 3).

  The imaging unit 14 incorporates a condensing lens and an imaging device such as a CCD or CMOS, and generates an image by receiving reflected light from the subject in accordance with control from the imaging control unit 15. FIG. 3 shows spectral sensitivity characteristics estimated for the image sensor incorporated in the imaging unit 14. An image generated when the light source group 13-1 emits light is referred to as a first image, and an image generated when the light source group 13-2 emits light is referred to as a second image.

  The imaging control unit 15 controls the imaging timing of the imaging unit 14, the luminance amplification gain, and the like according to the control from the control unit 11. In addition, the imaging control unit 15 outputs the first and second images generated by the imaging unit 14 to the image processing unit 16.

  The image processing unit 16 calculates a reflectance difference detection signal S = Y1−Y2 that is a difference between the luminances Y1 and Y2 of corresponding pixels of the first image and the second image, and determines the reflectance difference detection signal S as the difference. Binarization is performed by comparing with a predetermined threshold value, and one of the binary regions is detected as a skin region.

  The temperature detection unit 17 detects the ambient temperature of the light source groups 13-1 and 13-2 and outputs temperature information indicating the detection result to the control unit 11.

[Change in illuminance ratio of light source groups 13-1 and 13-2 accompanying change in ambient temperature]
FIG. 3A shows a case where the ambient temperature is changed from 0 ° C. to 75 ° C. with respect to a standard sample (human hair, clothes, etc.) having a constant reflectance in the emission wavelength range of the light source groups 13-1 and 13-2. The illuminance ratio (λ2) is maintained so that the imaging luminance when irradiated by an arbitrary LED belonging to the light source group 13-1 and the imaging luminance when irradiated by an arbitrary LED belonging to the irradiation light source 13-2 are equal. The change in the irradiance of the light of 1 / (irradiance of the light of λ1) is shown. FIG. 5B shows the average value of FIG. Note that the illuminance ratio in the figure is standardized with the illuminance ratio at room temperature of 25 ° C. being 1.

  As shown in the figure, even if there are variations in the emission wavelengths of the light source groups 13-1 and 13-2, the illuminance ratio increases monotonously with the increase in ambient temperature, and is approximately 0.96 at 0 ° C. , Approximately 1.07 at 75 ° C.

  Therefore, the ambient temperature is such that the illuminance ratio is approximately 0.96 when the ambient temperature is 0 ° C. even for skin regions having different reflectivities in the light emission wavelength ranges of the light source groups 13-1 and 13-2. When the temperature is 75 ° C., the light output levels of the light source group 13-1 and the light source group 13-2 may be adjusted so that the illuminance ratio is approximately 1.07. Specifically, the light output level of the light source group 13-2 may be decreased as the ambient temperature becomes lower than the room temperature of 25 ° C., and conversely, the light source group 13 becomes higher as the ambient temperature becomes higher than the room temperature of 25 ° C. -2 light output level may be increased.

[First Skin Area Detection Processing by Detection Device]
Next, the 1st skin area | region detection process by the detection apparatus 10 is demonstrated. FIG. 4 is a flowchart for explaining the first skin region detection process.

  In step S <b> 1, the temperature detection unit 17 detects the ambient temperature of the light source groups 13-1 and 13-2 and outputs temperature information indicating the detection result to the control unit 11. Based on this temperature information, the control unit 11 determines the radiance of each of the light source groups 13-1 and 13-2 so as to obtain an illuminance ratio corresponding to the ambient temperature, and notifies the LED control unit 12 of the radiance.

  In step S2, the LED control unit 12 controls the light source group 13-1, and irradiates the subject with light of wavelength λ1 with the irradiance determined in step S1. In step S <b> 3, the imaging unit 14 generates a first image by photoelectrically converting reflected light from the incident subject in accordance with control from the imaging control unit 15. Thereafter, the light source group 13-1 is turned off.

  In step S4, the LED control unit 12 controls the light source group 13-2 to irradiate the subject with light having the wavelength λ2 with the irradiance determined in step S1. In step S <b> 5, the imaging unit 14 generates a second image by photoelectrically converting reflected light from the incident subject in accordance with control from the imaging control unit 15. Thereafter, the light source group 13-2 is turned off.

  In step S6, the image processing unit 16 calculates a reflectance difference detection signal S = Y1-Y2 that is the difference between the luminances Y1 and Y2 of the corresponding pixels of the first image and the second image, and in step S7. The reflectance difference detection signal S is binarized by comparing it with a predetermined threshold value. In step S8, the image processing unit 16 generates a binarized image having the binarized values as pixel values, and detects one of the binarized areas as a skin area. This completes the first skin region detection process.

  According to the first skin region detection process described above, the luminance ratio between the light source group 13-1 and the light source group 13-2 is adjusted according to the ambient temperature, so that the skin region on the image is detected with high accuracy. Can do.

  FIG. 5 illustrates the light source groups 13-1 and 13-2 of the first skin area detection process (described as the present invention in the figure) and the conventional skin area detection process (described as the conventional technique in the figure). A comparison of reflectance difference detection signals S corresponding to standard samples (human hair, clothes, etc.) having a constant reflectance in the emission wavelength range is shown.

  That is, since the reflectance difference detection signal S corresponds to a standard sample having a constant reflectance, the reflectance difference detection signal S is preferably 0 regardless of the ambient temperature. However, in the conventional skin region detection processing, the reflectance difference detection signal S = 0 when the ambient temperature is 25 ° C. (room temperature), but the reflectance difference detection signal S = −1.9 when the ambient temperature is 0 ° C. When the ambient temperature is 75 ° C., the reflectance difference detection signal S = 3.2, which is a value away from zero.

  In contrast, in the first skin region detection process, the reflectance difference detection signal S = 0 when the ambient temperature is 25 ° C. (room temperature), and the reflectance difference detection signal S = −0 when the ambient temperature is 0 ° C. When the ambient temperature is 75 ° C., the reflectance difference detection signal S = 0.6, which is close to 0. Therefore, it can be seen that the first skin region detection process can detect the skin region with higher accuracy than the conventional skin region detection process.

[Second Skin Area Detection Processing by Detection Device]
Next, a second skin region detection process that can be executed instead of the above-described first skin region detection process will be described. In the second skin region detection processing, the reflectance difference detection signal S and the light source group 13-1 and the light source group 13-2 are fixed in a state corresponding to a predetermined temperature (for example, room temperature 25 ° C.). The threshold value to be compared is changed according to the ambient temperature.

From here, FIG. 6 is a flowchart for explaining the second skin region detection processing.

  In step S11, the LED control unit 12 controls the light source group 13-1, and the luminance ratio with the light source group 13-2 at a predetermined irradiance (a predetermined temperature (for example, room temperature 25 ° C.)) is determined. The object is irradiated with light of wavelength λ1 at a value of 1). In step S <b> 12, the imaging unit 14 generates a first image by photoelectrically converting reflected light from the incident subject in accordance with control from the imaging control unit 15. Thereafter, the light source group 13-1 is turned off.

  In step S13, the LED control unit 12 controls the light source group 13-2, and the luminance ratio with the light source group 13-1 is 1 at a predetermined irradiance (predetermined temperature (for example, room temperature 25 ° C.)). The object is irradiated with light of wavelength λ2. In step S <b> 14, the imaging unit 14 generates a second image by photoelectrically converting reflected light from the incident subject in accordance with control from the imaging control unit 15. Thereafter, the light source group 13-2 is turned off.

  In step S15, the image processing unit 16 calculates a reflectance difference detection signal S = Y1-Y2 that is the difference between the luminances Y1 and Y2 of the corresponding pixels of the first image and the second image.

  In step S <b> 16, the temperature detection unit 17 detects the ambient temperature of the light source groups 13-1 and 13-2 and outputs temperature information indicating the detection result to the control unit 11. The controller 11 determines a temperature-dependent threshold for comparison with the reflectance difference detection signal S based on this temperature information.

  Specifically, according to the result of the conventional skin region detection process shown in FIG. 5, a conventional skin region corresponding to an ambient temperature of 0 ° C. with a temperature-dependent threshold at an ambient temperature of 25 ° C. (room temperature) as a reference value TH. What is necessary is just to add the value of the reflectance difference detection signal S by a detection process. For example, when the ambient temperature is 0 ° C., the temperature dependent threshold is set to the reference value TH-1.9, and when the ambient temperature is 75 ° C., the temperature dependent threshold is set to the reference value TH + 3.2. The determined temperature dependence threshold value is notified to the image processing unit 16.

  In step S17, the image processing unit 16 binarizes the reflectance difference detection signal S calculated in step S15 by comparing it with the temperature-dependent threshold value determined in step S16. In step S18, the image processing unit 16 generates a binarized image having the binarized value as a pixel value, and detects one of the binarized areas as a skin area. The second skin region detection process is thus completed.

  According to the second skin region detection process described above, the temperature-dependent threshold for comparison with the reflectance difference detection signal S is determined according to the ambient temperature, so that the skin region on the image is detected with high accuracy. Can do.

  Note that the detection device to which the present invention is applied can be incorporated in an arbitrary electronic device such as a television receiver, for example. In the electronic device, predetermined processing can be executed in accordance with the detected movement of the hand of the subject.

  By the way, the above-described series of processing can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The program may be processed by a single computer, or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention. For example, the binarization threshold value change for the difference signal is equivalent to adding a constant having a different size to each of the original images on which the difference calculation is performed. Similarly, the change in the illuminance ratio can obtain the same effect by multiplying each of the original images by a constant having a different size. The present invention can also be implemented by performing the above addition or multiplication processing with a signal amplifier, digital signal processing means, or the like according to the ambient temperature.

  DESCRIPTION OF SYMBOLS 10 Detection apparatus, 11 Control part, 12 LED control part, 13 LED, 14 Imaging part, 15 Imaging control part, 16 Image processing part, 17 Temperature detection part

Claims (7)

In an image processing apparatus for detecting a skin region representing human skin from an image,
A measuring means for measuring the temperature;
First irradiation means for irradiating the subject with light of a first wavelength;
A second irradiating means for irradiating the subject with light having a second wavelength longer than the first wavelength;
A first image is generated based on reflected light from the subject when the subject is irradiated with light of the first wavelength, and the subject is irradiated with light of the second wavelength. Generating means for generating a second image based on the reflected light from the subject at the time;
Detecting means for detecting the skin region based on a comparison result obtained by comparing a difference between the first image and the second image with a threshold;
The output of at least one irradiation light of the first or second irradiation means is changed so that the illuminance ratio corresponds to the measured temperature , or the threshold is set according to the measured temperature. An image processing apparatus including: The temperature corresponding means is
When the measured temperature is higher than the reference temperature, the first or second irradiating means so that the illuminance ratio obtained by dividing the illuminance of the second irradiating means by the illuminance of the first irradiating means is increased. change the output of at least one of the irradiation Shako of
When said measured temperature is lower than the reference temperature, change the output of at least one of the irradiation Shako of the so said illuminance ratio decreases first or second irradiation means,
Said first and second radiation means in accordance with the output of the irradiation beam irradiated is changed, the image processing apparatus according to claim 1 for irradiating said first wavelength or the light of the second wavelength to the subject. The temperature corresponding means is
If the measured temperature is higher than the reference temperature, change the direction to increase the threshold,
If the measured temperature is lower than the reference temperature, change the direction to lower the threshold,
The image processing apparatus according to claim 1, wherein the detection unit detects the skin region based on a comparison result with the calculated difference-changed threshold value. The image processing apparatus according to claim 2, wherein the first wavelength is less than 930 nm, and the second wavelength is 930 nm or more. First irradiation means for irradiating the subject with light of a first wavelength;
A second irradiating means for irradiating the subject with light having a second wavelength longer than the first wavelength;
A first image is generated based on reflected light from the subject when the subject is irradiated with light of the first wavelength, and the subject is irradiated with light of the second wavelength. Generating means for generating a second image based on the reflected light from the subject at the time;
In an image processing method of an image processing apparatus, comprising: a detecting unit that detects the skin region based on a comparison result obtained by comparing a difference between the first image and the second image with a threshold value.
According to the image processing device,
A measuring step for measuring the temperature;
The output of at least one irradiation light of the first or second irradiation means is changed so that the illuminance ratio corresponds to the measured temperature , or the threshold is set according to the measured temperature. An image processing method including: First irradiation means for irradiating the subject with light of a first wavelength;
A second irradiating means for irradiating the subject with light having a second wavelength longer than the first wavelength;
A first image is generated based on reflected light from the subject when the subject is irradiated with light of the first wavelength, and the subject is irradiated with light of the second wavelength. A program for controlling an image processing apparatus comprising: generating means for generating a second image based on reflected light from the subject when
Change the output of at least one irradiation light of the first or second irradiation means so that the illuminance ratio corresponds to the detected temperature, or change the threshold according to the measured temperature. A temperature step,
A program for causing a computer of an image processing apparatus to execute a process including a detection step of detecting a skin region based on a comparison result obtained by comparing a difference between the first image and the second image with the threshold value. A measuring means for measuring the temperature;
First irradiation means for irradiating the subject with light of a first wavelength;
A second irradiating means for irradiating the subject with light having a second wavelength longer than the first wavelength;
A first image is generated based on reflected light from the subject when the subject is irradiated with light of the first wavelength, and the subject is irradiated with light of the second wavelength. Generating means for generating a second image based on the reflected light from the subject at the time;
Detecting means for detecting the skin region based on a comparison result obtained by comparing a difference between the first image and the second image with a threshold;
The output of at least one irradiation light of the first or second irradiation means is changed so that the illuminance ratio corresponds to the measured temperature , or the threshold is set according to the measured temperature. Temperature response means to change,
And an operation control means for executing a predetermined process in response to the detected change in the skin area.

Images