JP2013003859A - Projection device, projection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a projection device accurately recognize posture or gesture or the like of a user even in an environment of using the projection device.SOLUTION: A first LED irradiates a subject with light of a first wavelength, a second LED irradiates the subject with light of a second wavelength longer than the first wavelength, an imaging element images the subject, an image processing part detects a skin area of the subject, on the basis of a first picked-up image obtained by the imaging performed when the light of the first wavelength is radiated, and a second picked-up image obtained by the imaging performed when the subject is irradiated with the light of the second wavelength, and a projection part projects a projection image changed according to the detection result of the image processing part. The present disclosure can be applied to, for example, a projector and the like.

Description

  The present disclosure relates to a projection apparatus, a projection method, and a program, and in particular, a projection apparatus, a projection method, and the like that can recognize a user's posture or gesture in an environment where the projection apparatus such as a projector is used. And the program.

  There is a recognition technique for recognizing a human posture (posture) or gesture (motion) from a captured image obtained by imaging a subject.

  In this recognition technique, a human hand or the like on a captured image is detected by pattern matching or the like, and a human posture or gesture is recognized based on the detection result (see, for example, Patent Document 1).

  Note that pattern matching refers to, for example, a pattern image representing the shape of a human hand is compared with each area on the captured image, and a human hand exists in an area where the comparison result is most similar to the pattern image. A technology to detect what to do.

JP 2004-302992 A

  In the above-described recognition technique, for example, when the illumination for illuminating the subject is not sufficient (dark), the subject may be covered with darkness in the surroundings and the subject may not be captured on the captured image.

  In this case, since a human hand or the like is not shown in the captured image, the human hand or the like cannot be detected by the above-described pattern matching or the like.

  Therefore, for example, when a projection device such as a projector is controlled by a user's posture or gesture, the space other than the projection range projected from the projection device is often dark in the environment used. It is difficult to recognize human postures or gestures using the recognition techniques described above.

  The present disclosure has been made in view of such a situation, and allows the projection apparatus to accurately recognize a user's posture or gesture even in an environment where the projection apparatus is used.

  A projection device according to one aspect of the present disclosure includes a first irradiation unit that irradiates a subject with light having a first wavelength, and a second wavelength that is longer than the first wavelength with respect to the subject. A second irradiating unit that irradiates light, an imaging unit that images the subject, a first captured image obtained by imaging when irradiating light of the first wavelength, and the subject And a detection unit that detects a skin area of the subject based on a second captured image obtained by imaging when the light of the second wavelength is irradiated, and is changed according to a detection result of the detection unit. A projection unit that projects a projection image.

  The imaging unit can image the subject in an imaging range including a projection range on which the projection image is projected.

  A calculation unit that calculates the projection range on a captured image obtained by imaging by the imaging unit can be further provided.

  The projection unit also projects a contour projection image representing the contour of the projection range, and the calculation unit calculates the projection range on the captured image obtained by imaging when the contour projection image is projected. Can do.

  The first irradiation unit emits invisible light of the first wavelength, the second irradiation unit emits invisible light of the second wavelength, and the projection unit displays the contour projection image. , Projection light including at least visible light, or invisible light can be projected.

  The projection unit projects the contour projection image as the projection light, and the imaging unit includes a first light receiving element provided with a visible light blocking unit that blocks visible light, and the visible light blocking unit. A second light receiving element, the first light receiving element generates the first and second captured images, and the second light receiving element generates the pattern. An image can be generated.

  The projection unit projects the contour projection image as invisible light, and the imaging unit includes an imaging element provided with a visible light blocking unit that blocks visible light. In addition to the two captured images, the pattern image can also be generated.

  A visible light blocking unit configured to block visible light; and when the projection light is projected as the contour projection image, the visible light blocking unit is moved so that visible light is incident on the imaging element, and A movement control unit that moves the visible light blocking unit so that visible light is not incident on the imaging element when irradiated with light having the first wavelength or light having the second wavelength can be further provided. .

  The projection unit includes a reflection unit that reflects light incident from a predetermined direction to the projection range, and the first irradiation unit transmits light having the first wavelength to the reflection unit. Reflecting and irradiating the projection range, the second irradiation unit can reflect the light of the second wavelength to the reflection unit and irradiate the projection range.

  In the first irradiation unit, the light of the first wavelength is reflected to the reflection unit that reflects each pixel value as a projection image having a uniform pixel value to irradiate the projection range, and in the second irradiation unit, The light of the second wavelength can be reflected on the reflection unit that reflects the pixel value as a projection image having a uniform pixel value, and can be applied to the projection range.

  The first wavelength that is reflected by the reflection unit among the first polarized light or the second polarized light different from the first polarized light is emitted from the first irradiation unit. The first polarization conversion unit that converts the light into the polarized light and the light having the second wavelength that is irradiated from the second irradiation unit is the reflection unit of the first polarization or the second polarization. A second polarization conversion unit that converts the first polarized light to be reflected may be further provided.

  A motion detection unit that detects the motion of the projection device may be further provided, and the calculation unit may include the projection range on the captured image even when the motion detection unit detects the motion of the projection device. Can be calculated.

  The projection unit can project the projection image in a range excluding the detected part of the skin region in the projection range.

  In the imaging unit, the imaging direction can be changed.

  A position conversion unit that converts a first position on the captured image from which the skin region is detected to a second position according to an imaging direction of the imaging unit can be further provided. The projection image changed according to the second position can be projected.

  The first irradiation unit irradiates the projection range with light of the first wavelength, the second irradiation unit irradiates the projection range with light of the second wavelength, and the detection unit The skin area within the projection range can be detected based on the first captured image and the second captured image.

The first wavelength λ1 and the second wavelength λ2 are:
640nm ≤ λ1 ≤ 1000nm
900nm ≤ λ2 ≤ 1100nm
Can be met.

The first irradiation unit can irradiate invisible light having the first wavelength λ1, and the second irradiation unit can irradiate invisible light having the second wavelength λ2.
An output unit that outputs the detection result in the projection range as a signal corresponding to a position in the projection range may be further provided.

  A projection method according to one aspect of the present disclosure is a projection method of a projection device, wherein the projection device irradiates a subject with light having a first wavelength, and the first wavelength of the projection device. A first imaging step of imaging the subject when irradiated with light, and a second irradiation step of irradiating the subject with light having a second wavelength longer than the first wavelength. A second imaging step for imaging the subject when the second wavelength light is irradiated, and a first captured image obtained by imaging when the first wavelength light is irradiated And a detection step of detecting a skin region of the subject based on a second captured image obtained by imaging when the subject is irradiated with light of the second wavelength, and detection of the detection step Projecting a projection image that changes depending on the result It is a projection method that includes a shadow step.

  A program according to one aspect of the present disclosure includes a first irradiation step of causing a computer to irradiate light of a first wavelength on a subject, and imaging of the subject when the light of the first wavelength is irradiated. A first imaging step to be performed; a second irradiation step of irradiating the subject with light having a second wavelength longer than the first wavelength; and irradiating the light with the second wavelength. A second imaging step for imaging the subject when the second imaging step is performed, a first captured image obtained by imaging when the light of the first wavelength is irradiated, and the second for the subject. A detection step for detecting a skin region of the subject based on a second captured image obtained by imaging when light having a wavelength of λ is irradiated, and a projection image that is changed according to the detection result of the detection step A projection step to project Is a program for executing the untreated.

  According to one aspect of the present disclosure, the subject is irradiated with light having a first wavelength, and the subject is imaged when the subject is irradiated with light having the first wavelength. The light of the second wavelength that is longer than the first wavelength is irradiated, and the subject is imaged when the light of the second wavelength is irradiated, and the light of the first wavelength is irradiated Skin of the subject based on a first captured image obtained by imaging when the image is captured and a second captured image obtained by imaging when the subject is irradiated with light of the second wavelength A region is detected, and a projection image that is changed according to the detection result is projected.

  According to the present disclosure, even in an environment where a projection apparatus is used, the projection apparatus can recognize a user's posture or gesture with high accuracy.

It is a block diagram which shows the structural example of the projector in this indication. It is a figure for demonstrating a calibration process. It is a figure which shows the detailed 1st structural example of the skin detection part and projection part which are contained in a projector. It is a figure which shows the spectral reflection characteristic with respect to human skin. It is a flowchart for demonstrating the gesture recognition process which the skin detection part of FIG. 3 performs. It is a flowchart for demonstrating the projection process which the projection part of FIG. 3 performs. It is a figure which shows the detailed 2nd structural example of the skin detection part and projection part which are contained in a projector. It is a figure which shows the detailed 3rd structural example of the skin detection part and projection part which are contained in a projector. It is a flowchart for demonstrating the gesture recognition projection process which the skin detection part and projection part of FIG. 8 perform. It is a block diagram which shows the structural example of a computer.

Hereinafter, embodiments of the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. 1st Embodiment (an example when the projection range projected by a projector is included in the gesture detectable range which can detect a human gesture etc.)
2. Second embodiment (an example when the gesture detectable range coincides with the projection range)
3. Third Embodiment (Another example when the gesture detectable range and the projection range match)
4). Modified example

<1. First Embodiment>
[Configuration Example of Projector 21]
FIG. 1 shows a configuration example of a projector 21 according to the first embodiment.

  The projector 21 can be connected to, for example, a personal computer 22, and projects a projection image supplied from the connected personal computer 22 and can be operated by an input operation corresponding to a user's gesture or posture. It is possible.

  The projector 21 includes a skin detection unit 41, a projection unit 42, a control unit 43, and an operation unit 44. In FIG. 1, the flow of the control signal is represented by a dotted line, and the flow of the data signal is represented by a solid line. This also applies to FIGS. 3, 7, and 8 described later.

  The skin detection unit 41 detects, for example, a user's gesture or posture (hereinafter referred to as a gesture) performed in a gesture detectable range as shown in FIG. 1 and supplies the detection result to the personal computer 22. . As will be described later, the skin detection unit 41 captures the gesture detectable range as an imaging range, and detects a user's gesture or the like based on a captured image obtained by the imaging.

  In addition, before detecting the user's gesture or the like, the skin detection unit 41 determines which region within the gesture detectable range includes a projection range representing a range where the projection image is projected by the projection unit 42. Perform calibration processing to detect. This calibration process will be described in detail with reference to FIG.

  The skin detection unit 41 determines whether the detected gesture or the like has been detected within the projection range, and supplies the determination result to the personal computer 22 together with the detection result.

  For this reason, for example, in the personal computer 22, even when the same gesture or the like is supplied as a detection result, different processing is performed depending on whether or not the detection is within the projection range. it can.

  That is, for example, when a detection result indicating that the user's hand has been detected is supplied within the projection range, the personal computer 22 controls the projection unit 42 to project a projection image based on the detection result. Let

Specifically, for example, the personal computer 22 projects a projection image on which a cursor or the like is displayed at a position on the projection image corresponding to the position where the user's hand is detected.
For example, the personal computer 22 can highlight an icon on the projection image corresponding to the position where the user's hand is detected.
Furthermore, for example, when the user's hand stays on the icon for a predetermined time, that is, when the user's hand stays at the detected position for a predetermined time, it corresponds to the position where the user's hand is detected. It is determined that the icon on the projection image is selected, and the corresponding projection image is projected.
In this case, the number of hands and fingers detected and used for control is not necessarily one. For example, when both hands are detected on the projection screen and the distance between both hands is enlarged, a projection image enlarged according to the distance between both hands may be projected.

  Further, for example, the personal computer 22 controls the projection unit 42 when a detection result indicating that the user's hand has been detected is supplied within a range different from the projection range in the gesture detectable range. Then, a projection image based on the detection result is projected.

Specifically, for example, the personal computer 22 projects a projection image that does not display a cursor or the like.
The skin detection unit 41 outputs detection results and the like to the personal computer 22. And although the personal computer 22 has changed the projection image projected according to the detection result from the skin detection part 41, etc., it is not limited to this.
That is, for example, the skin detection unit 41 may calculate the coordinates (position) of a cursor or the like to be displayed on the projection image according to the detection result or the like, and output the calculated coordinates to the personal computer 22.
In this case, the personal computer 22 projects a projection image in which a cursor or the like is displayed on the coordinates from the skin detection unit 41.
The skin detection unit 41 calculates a change value between the previous cursor coordinate and the current cursor coordinate instead of the cursor coordinate, and outputs the calculated change value to the personal computer 22. Also good.
In this case, the personal computer 22 calculates coordinates such as a cursor based on the change value from the skin detection unit 41, and projects a projection image on which the cursor is displayed on the calculated coordinates.
For example, when projecting an image stored in a memory (not shown) built in the projector 21 or a memory stick connected to the projector 21 from the outside as a projection image, the personal computer 22 is not used. Can be projected.
That is, for example, the skin detection unit 41 controls the projection unit 42 to change the projection image based on the detection result or the like, and changes the projection image from the projection unit 42 in the same manner as the personal computer 22 described above. Project.

  For example, the projection unit 42 projects a projection image from the connected personal computer 22 onto a projection range as shown in FIG. In FIG. 1, the projection range is within the gesture detectable range, but the projection range is not limited to this.

  That is, for example, the projection range is a range that is entirely included in the gesture detectable range, a range that is partially included in the gesture detectable range, or a range that is not included in the gesture possible range at all. You may do it. The case where the projection range and the gesture detectable range are the same will be described later with reference to FIGS. 7 and 8.

  For example, the control unit 43 controls the skin detection unit 41 and the projection unit 42 based on an operation signal from the operation unit 44.

  The operation unit 44 includes, for example, operation buttons and the like, and supplies an operation signal corresponding to the user's operation to the control unit 43 in response to the operation by the user.

[Details of calibration process]
Next, an example of calibration processing performed by the skin detection unit 41 will be described with reference to FIG.

  FIG. 2 shows an example of a captured image I_pattern obtained when an image of the gesture detectable range is captured as an imaging range in a state where a contour projection image having high brightness values is projected only at the four corners. Note that the contour projection image is, for example, an image representing the contour of the projection range by the four corner portions having high luminance values, but is not limited thereto. That is, for example, the outline of the projected image may be represented by four sides having high luminance values, or the outline of the projected image may be simply represented by a rectangle having the entire screen with high luminance values.

  Here, the projector 21 is generally used in a state where the illumination in the room is darkened. Therefore, the captured image I_pattern shown in FIG. 2 is a camera built in the skin detection unit 41 (the imaging element 63, the lens 64, etc. in FIG. 3) in the dark, with the projected contour image projected by the projection unit 42. It is acquired by imaging.

Therefore, in the captured image I_pattern, the luminance value of the portion indicated by the white circle (corresponding to the four corner portions of the contour projection image) is high, and the luminance value of the other portion is low.
Here, a visible light cut filter 65 is movably disposed on the front surface of the lens 64. Since the image sensor 63 is generally sensitive to both visible light and near-infrared light that is invisible light emitted from the LEDs 62a and 62b for skin detection, the visible light cut filter uses visible light as an image sensor. This is to prevent the incidence and improve the accuracy of skin detection. However, since the projection image is visible light and the contour projection image is also visible light in this configuration, when acquiring the contour projection image, the visible light cut filter 65 is moved, and the contour projection in which the imaging element is made of visible light is performed. Make the image available.

In the calibration process, the position and size of the projected image in the captured image are calculated. Since the projected image in the captured image changes depending on the relative position and angle between the projector 21 and the screen on which the projected image is projected, calibration processing may be performed when these change.
The calibration process is started, for example, when the user performs a power-on operation to turn on the projector 21 by operating the operation unit 44. At this time, the operation unit 44 supplies an operation signal corresponding to the user's power-on operation to the control unit 43.

In response to the operation signal from the operation unit 44, the control unit 43 controls the skin detection unit 41 and the projection unit 42 to perform calibration processing. Further, the projector 21 may be provided with a sensor for detecting the movement of the projector 21 so that the calibration process is automatically performed when the movement or tilt change of the projector is detected.

  That is, for example, the projection unit 42 projects the contour projection image on the projection range in accordance with the control from the control unit 43.

  The skin detection unit 41 captures the gesture detectable range as an imaging range in a state where the contour projection image is projected onto the projection range. The skin detection unit 41 includes a camera that performs imaging. Details of the skin detection unit 41 will be described in detail with reference to FIG.

  The skin detection unit 41 acquires, for example, a captured image I_pattern as illustrated in FIG.

  The skin detection unit 41 detects a portion to be a projection range in the gesture detectable range based on the captured image I_pattern as illustrated in FIG.

  That is, for example, the skin detection unit 41 determines whether or not the luminance value of each pixel constituting the captured image I_pattern as illustrated in FIG. 2 is equal to or higher than a predetermined threshold value. (The part shown in white in FIG. 2) is detected.

In addition, the skin detection unit 41 detects, as a projection range, a rectangle whose corner is a pixel with a detected high luminance value, and calculates the position and size of the projection range on the captured image based on the detection result. .
Here, when the projection unit 42 and the screen on which the projection image is projected are arranged obliquely, that is, for example, when the projection unit 42 is installed on the upper front surface of the screen installed vertically, the projection image is It becomes a trapezoid instead of a rectangle.
This correction of the projection image distorted into a trapezoid into a rectangle by image processing is also called keystone correction. When the keystone correction is performed, a calibration process may be performed after the correction, and a portion to be a projection range in the gesture detectable range may be detected based on the captured image I_pattern.
In the keystone correction, for example, the shape of the projected image is corrected from a trapezoid to a rectangle by using affine transformation or the like.

  The skin detection unit 41 holds projection range information indicating the calculated position and size of the projection range on the captured image in a memory (not shown) built in the skin detection unit 41. This projection range information is used to determine whether a user gesture or the like has been detected within the projection range.

  By the way, the projection unit 42 is configured to project the contour projection image in which only the four corner portions are set to the high luminance value, but, for example, among the four corners, only the two corners on the diagonal line of the projection range have the high luminance value. The contour projected image may be projected.

That is, the outline projection image may be any projection image as long as it is a projection image representing the outline of the projection range.
An operator's gesture operation can also be used to acquire projection range information on the captured image. For example, with a projected contour image with high brightness only at the four corners, the operator gives instructions from the projector to the projection screen to place his hands and fingers in order on the brightened parts at the four corners for a predetermined time. You may make it take out. For example, the fingertip of the operator may be detected by skin detection, and a quadrangle with the four corners pointed by the operator may be recognized as a contour projection image.
In this method, since it is not necessary to move the visible light cut filter in front of the image sensor 63, it is possible to reduce the size and cost of the skin detection unit.

[Details of Skin Detection Unit 41 and Projection Unit 42]
Next, FIG. 3 shows a detailed first configuration example of the skin detection unit 41 and the projection unit 42 included in the projector 21.

  The skin detection unit 41 includes a DSP (digital signal processor) 61, an LED (light-emitting diode) 62a, an LED 62b, an image sensor 63, a lens 64, and a visible light cut filter 65. Note that the number of the LEDs 62a and the number of the LEDs 62b are not limited to one each, and may be plural if necessary.

  The DSP 61 functions as the LED control unit 81 and the image processing unit 82 by, for example, executing a control program held in a memory (not shown) or the like.

  The LED control unit 81 controls the LEDs 62a and 62b based on the frame synchronization signal from the image sensor 63, and controls turning off and lighting of the LEDs 62a and 62b. Here, the frame synchronization signal represents the timing at which the imaging element 63 is imaged.

  That is, for example, based on the frame synchronization signal from the image sensor 63, the LED control unit 81 lights only the LED 62a, lights only the LED 62b, and turns on the LED 62a and the image 62a each time an image is captured by the image sensor 63. All of the LEDs 62b are repeatedly turned off.

  The image processing unit 82 acquires a captured image I_pattern from the image sensor 63. The image processing unit 82 detects the position and size of the projection range on the captured image from the image sensor 63 based on the captured image I_pattern from the image sensor 63 as described with reference to FIG. Perform calibration processing.

  The image processing unit 82 holds projection range information indicating the position and size of the projection range on the captured image detected by the calibration process in a built-in memory (not shown).

  Further, the image processing unit 82 acquires captured images I_λ1, I_λ2, and I_off from the image sensor 63. Here, the captured image I_λ1 represents an image obtained by imaging with the imaging element 63 in a state where only the LED 62a that emits light of wavelength λ1 is turned on.

  The captured image I_λ2 represents an image obtained by imaging with the imaging element 63 in a state where only the LED 62b that emits light with the wavelength λ2 is turned on. Furthermore, the captured image I_off represents an image obtained by imaging with the imaging element 63 in a state where both the LED 62a and the LED 62b are turned off.

  The image processing unit 82 smoothes the acquired captured images I_λ1, I_λ2, and I_off using LPF (low pass filter). Further, the image processing unit 82 subtracts the luminance value Y (λ2) of the captured image I_λ2 from the luminance value Y (λ1) of the captured image I_λ1 for each pixel (Y (λ1) −Y ( λ2)) is calculated.

  Furthermore, the image processing unit 82 uses, for example, a luminance value (Y (λ1) Y (off)) as a value based on at least one of the captured image I_λ1 or the captured image I_λ2, and a difference (Y (λ1) −Y (λ2)) Normalize (divide). Note that the luminance value Y (off) represents the luminance value of the captured image I_off.

  The image processing unit 82 multiplies the normalized difference {(Y (λ1) −Y (λ2)) / (Y (λ1) Y (off))} by a predetermined value (for example, 100). The obtained difference image I_diff (= {(I_λ1−I_λ2) / (I_λ1 I_off)} × 100) is binarized with a predetermined threshold for binarization to calculate a binarized skin image I_skin.

Note that the difference image I_diff (= {(I_λ1-I_λ2) / (I_λ1 I_off)} × 100) indicates how much the luminance value of the captured image I_λ1 is higher than the luminance value of the captured image I_λ2. To express. As the binarization threshold value, for example, 10% can be used.
Here, the luminance value Y (off) of the captured image I_off is used to remove the influence of external light other than the irradiation light of the LEDs 62a and 62b, thereby improving the accuracy of skin detection. However, since the projector 21 is often used in a relatively dark environment, acquisition of the captured image I_off may be omitted.

  The image processing unit 82 detects the skin portion of the user based on the calculated binarized skin image I_skin.

  That is, for example, the image processing unit 82 detects, as a skin region representing a user's skin portion, a region that is equal to or greater than the threshold for binarization among all the regions constituting the calculated binarized skin image I_skin. To do.

  In addition, the principle which can detect a user's skin area | region in this way is later mentioned with reference to FIG.

  In the first embodiment, for example, the user moves his / her hand in front of the camera including the image sensor 63 and the lens 64 in order to cause the projector 21 to execute predetermined processing, Is supposed to change.

  Therefore, for example, the image processing unit 82 detects, for example, a user's hand portion as a skin region based on the calculated binarized skin image I_skin. Note that the skin portion detected as the skin region is not limited to the hand portion, and for example, a face portion or the like may be detected.

Further, when a plurality of skin regions are detected based on the binarized skin image I_skin, the image processing unit 82 detects, for example, a skin region having the highest luminance among the plurality of skin regions as a hand region. This is because the distance to the LEDs 62a and 62a is assumed to be the closest to the hand portion of each portion of the user.
However, since the positional relationship between the operator and the skin detection unit 41 varies depending on the operation method, the method for detecting the hand region is not limited to this, and is appropriately determined. For example, a hand region may be detected by pattern matching using a binarized skin image I_skin or the like. According to this method, since the operator's skin area has already been specified, in order to apply pattern matching only to the skin area portion, when the hand area is detected by pattern matching from a visible light image acquired by a normal camera Compared with, CPU processing is lighter.
In other words, since the area where pattern matching is performed is specified as the skin area, compared with the case where the hand area is detected by pattern matching from a visible light image acquired by a normal camera, the projected image and the finger or hand are displayed on the projection screen. It is difficult for the problem that pattern matching is difficult due to the mixture.
Furthermore, in the dark space outside the projection screen, the hand and fingers are not clearly visible in the normal visible light image, but in this case, the reflected light reflected by the subject is imaged because the irradiated invisible light is reflected, The problem that the hands and fingers are not clearly visible is less likely to occur. Moreover, since the area | region which performs pattern matching can be specified to a skin area | region, compared with the case where the area | region of a hand is detected by pattern matching from the visible light image acquired with the normal camera, the process of CPU may be light.

  For example, the image processing unit 82 detects the center of gravity of the detected hand portion as the position of the user's hand.

  Further, the image processing unit 82 includes the detected hand position based on the projection range information stored in a built-in memory (not shown) within the projection range displayed in the captured image obtained by the imaging element 63. The detected hand position is supplied to the personal computer 22 together with the result of the determination.

  When the determination result supplied to the personal computer 22 is a determination result indicating that the detected hand position is included in the projection range on the captured image, the personal computer 22 corresponds to the position of the hand on the image capturing range. A projection image in which a cursor or the like is displayed at a position on the projection image is generated.

  Then, the personal computer 22 supplies the generated projection image to the projection unit 42 for projection. Thereby, a cursor or the like is displayed at a position on the projection image corresponding to the position of the user's hand.

  For example, when a determination result indicating that the detected hand position is not included in the projection range on the captured image is supplied to the personal computer 22, the personal computer 22 generates a projection image on which a cursor or the like is not displayed. Then, it is supplied to the projection unit 42 for projection.

  Further, for example, when a determination result indicating that the detected hand position is not included in the projection range on the captured image is supplied to the personal computer 22, the personal computer 22 generates a projection image on which a cursor or the like is not displayed. Then, it is supplied to the projection unit 42 for projection.

  In addition, for example, when the determination result that the detected hand position is not included in the projection range on the captured image is supplied to the personal computer 22, the personal computer 22 determines the position of the user's hand (or the hand). Application corresponding to the shape of the hand) and the shape of the hand may be executed.

  Note that the image processing unit 82 may output the detection result or the like to the control unit 43 (FIG. 1). In this case, the control unit 43, for example, an actuator for changing the imaging direction of the camera (the imaging element 63 and the lens 64) or the projection direction of the projection unit 42 according to the detection result from the image processing unit 82, and the like. Drive.

  For example, the image processing unit 82 may output a detection result or the like to the control unit 111 of the projection unit 42. In this case, for example, the control unit 111 may control the laser light source units 101 to 103, the LCOS unit 108, and the like according to the detection result from the image processing unit 82 and adjust the brightness of the projected image. Good.

  The LED 62a is turned on or off in accordance with control from the LED control unit 81. That is, the LED 62a irradiates light of wavelength λ1 (for example, infrared light of wavelength λ1) or stops irradiating light of wavelength λ1 according to control from the LED control unit 81.

  The LED 62b is turned on or off in accordance with control from the LED control unit 81. That is, the LED 62b irradiates light having a wavelength λ2 that is longer than the wavelength λ1 (for example, infrared light having a wavelength λ2) or stops emitting light having a wavelength λ2 in accordance with control from the LED control unit 81.

  The LED 62a and the LED 62b irradiate at least the gesture detectable range (the imaging range of the imaging device 63).

  Further, the combination (λ1, λ2) of the wavelength λ1 and the wavelength λ2 is determined in advance based on, for example, spectral reflection characteristics with respect to human skin.

  Next, FIG. 4 shows spectral reflection characteristics with respect to human skin.

  This spectral reflection characteristic is general regardless of the color difference (race difference) and state (sunburn, etc.) of human skin.

  In FIG. 4, the horizontal axis indicates the wavelength of the irradiation light irradiated on the human skin, and the vertical axis indicates the reflectance of the irradiation light irradiated on the human skin.

  It is known that the reflectance of irradiated light radiated on human skin decreases rapidly from around 900 [nm], peaking around 800 [nm], and rises again around 1000 [nm] as a minimum. ing.

  Specifically, for example, as shown in FIG. 4, the reflectance of reflected light obtained by irradiating human skin with light of 870 [nm] as infrared rays is about 63%. Moreover, the reflectance of the reflected light obtained by irradiating light of 950 [nm] as infrared rays is about 50 percent.

  This is peculiar to human skin, and in an object other than human skin (for example, clothing), the change in reflectance is moderate in the vicinity of 800 to 1000 [nm]. In addition, as the frequency becomes higher, it often increases gradually.

  In the first embodiment, for example, the combination (λ1, λ2) = (870,950). This combination is a combination in which the reflectance when the human skin is irradiated with light of wavelength λ1 is larger than the reflectance when the light of wavelength λ2 is irradiated.

  Therefore, the luminance value constituting the skin area on the captured image I_λ1 is a relatively large value, and the luminance value constituting the skin area on the captured image I_λ2 is a relatively small value.

  Therefore, the luminance value of the skin area on the difference image I_diff (= {(I_λ1−I_λ2) / (I_λ1 I_off)} × 100) is a relatively large positive value α.

  Further, in the combination (λ1, λ2) = (870,950), the reflectance when the light of the wavelength λ1 is irradiated to the thing other than the human skin is more than the reflectance when the light of the wavelength λ2 is irradiated. A combination that does not grow.

  Therefore, the luminance value of the skin region on the difference image I_diff (= {(I_λ1−I_λ2) / (I_λ1 I_off)} × 100) is a relatively small positive or negative value β.

  Therefore, the difference image I_diff is binarized with a predetermined threshold value for binarization (for example, a threshold value smaller than α and larger than β) so as to detect the skin region. I have to.

  Here, the combination (λ1, λ2) is not limited to (λ1, λ2) = (870,950), and may be any combination as long as the difference in reflectance is sufficiently large (λ1, λ2). .

  For accurate skin detection, the wavelength λ1 is generally in the range of 640 [nm] to 1000 [nm], and the wavelength λ2 is in the range of 900 [nm] to 1100 [nm]. Thus, it is known from experiments previously conducted by the present inventor that it is desirable to set each of them. However, when the wavelength λ1 is in the visible light region, the operator feels dazzling, or the combination with the projector 21 affects the color tone of the projected image, so the value of the wavelength λ1 is 800 [nm] in the invisible light region. It is desirable to set it above.

  That is, for example, within the above-described range, the value of the wavelength λ1 can be set to 800 [nm] or more and less than 900 [nm], and the value of the wavelength λ2 can be set to 900 [nm] or more.

  Returning to FIG. 3, as described above, the imaging element 63 is provided with the lens 64 on the front surface thereof, and the front surface of the lens 64 is covered with the visible light cut filter 65 that blocks visible light.

  The imaging element 63 is configured by, for example, a CMOS (complementary metal oxide semiconductor) sensor and the like, and images a subject in accordance with a frame synchronization signal. Further, the image sensor 63 supplies a frame synchronization signal to the LED control unit 81.

  Specifically, for example, the image sensor 63 receives reflected light of light having a wavelength λ 1 that is irradiated on the subject by the LED 62 a via the lens 64. Then, the imaging element 63 supplies a captured image I_λ1 obtained by photoelectrically converting the received reflected light to the image processing unit 82.

  Further, for example, the image sensor 63 receives reflected light of the light having the wavelength λ 2 irradiated on the subject by the LED 62 b via the lens 64. Then, the imaging element 63 supplies a captured image I_λ2 obtained by photoelectrically converting the received reflected light to the image processing unit 82.

  Further, for example, the imaging element 63 receives reflected light from the subject via the lens 64 in a state where neither the LED 62a nor the LED 62b is irradiating light. Then, the imaging element 63 supplies a captured image I_off obtained by photoelectrically converting the received reflected light to the image processing unit 82.

  The visible light cut filter 65 blocks visible light out of light incident thereon, and transmits invisible light. The visible light cut filter 65 is provided so as to cover the front surface of the lens 64, and can be driven in the left-right direction in the figure. The visible light cut filter 65 is driven in the left-right direction in the figure in accordance with, for example, control from the LED control unit 81.

  The projection unit 42 includes laser light source units 101 to 103, prisms 104 and 105, a lens 106, a polarization beam splitter 107, an LCOS (liquid crystal on silicon) unit 108, lenses 109 and 110, and a control unit 111.

  The laser light source unit 101 irradiates the prism 104 with a B component laser beam in accordance with control from the control unit 111. The laser light source unit 102 irradiates the prism 104 with a G component laser beam in accordance with control from the control unit 111. The laser light source unit 103 irradiates the prism 105 with R component laser light in accordance with control from the control unit 111.

  The prism 104 is configured to transmit the B component laser light emitted from the laser light source unit 101 and reflect the G component laser light emitted from the laser light source unit 102 toward the prism 105.

  Therefore, the B component laser light from the laser light source unit 101 passes through the prism 104 and is irradiated onto the prism 105. The G component laser light from the laser light source unit 102 is reflected by the prism 104 and applied to the prism 105.

  The prism 105 is configured to transmit the B component and G component laser beams emitted from the prism 104 and reflect the R component laser beam emitted from the laser light source unit 103 in the direction of the lens 106. .

  Therefore, the B component and G component laser beams incident from the prism 104 are transmitted through the prism 105 and irradiated onto the polarization beam splitter 107 via the lens 106. Further, the R component laser light from the laser light source unit 103 is reflected by the prism 105 and applied to the polarization beam splitter 107 via the lens 106.

  The lens 106 is provided between the prism 105 and the polarization beam splitter 107 and refracts light from the prism 105.

  The polarization beam splitter 107 reflects S-polarized light emitted from the prism 105 through the lens 106 in the direction of the LCOS unit 108.

  In addition, each laser beam shall be output from each laser light source part 101 thru | or 103 as S polarization | polarized-light. The polarizing beam splitter 107 reflects each laser beam irradiated from the prism 105 via the lens 106 and irradiates the LCOS unit 108 with it.

  The polarizing beam splitter 107 transmits the P-polarized light emitted from the LCOS unit 108 and irradiates the projection range via the lenses 109 and 110. Thereby, a projection image is projected on the projection range.

  The LCOS unit 108 includes a reflective liquid crystal panel, and generates an R component projection image, a G component projection image, and a B component projection image based on each laser beam from the polarization beam splitter 107, respectively. To do.

  Further, the LCOS unit 108 irradiates the polarization beam splitter 107 with light corresponding to the projected image of the R component, the projected image of the G component, and the projected image of the B component as P-polarized light.

  The lens 109 is provided between the polarization beam splitter 107 and the lens 110. In addition, the lens 110 is provided between the lens 109 and a region to be a projection range.

  The lenses 109 and 110 refract the P-polarized light from the polarization beam splitter 107 and irradiate it within the projection range. Thereby, a projection image is projected on the projection range.

For example, the control unit 111 controls the laser light source units 101 to 103 and the LCOS unit 108 according to control from the personal computer 22 to project the image displayed on the display screen of the personal computer 22 as a projection image. Project to the range.
In addition, although the example which uses a laser light source as a light source of the projection part 42 was shown here, it is not limited to this, An LED light source may be used and a lamp | ramp may be used. Further, for example, a laser and an LED may be combined.

[Description of Operation of Skin Detection Unit 41 in FIG. 3]
Next, the gesture recognition process performed by the skin detection unit 41 in FIG. 3 will be described with reference to the flowchart in FIG.

  This gesture recognition process is started, for example, when the user performs a power-on operation to turn on the projector 21 using the operation unit 44. At this time, the operation unit 44 supplies an operation signal corresponding to the user's power-on operation to the control unit 43. The control unit 43 controls the skin detection unit 41 and the projection unit 42 according to an operation signal from the operation unit 44.

  That is, in step S21, the LED control unit 81 controls an actuator (not shown) for driving the visible light cut filter 65 according to the control from the control unit 43 so as to cover the front surface of the lens 64. The visible light cut filter 65 arranged as described above is moved in a predetermined direction. Thereby, the front surface of the lens 64 is not covered with the visible light cut filter 65.

  In step S <b> 22, the control unit 111 controls the laser light source units 101 to 103 and the LCOS unit 108 in accordance with the control from the control unit 43, and converts the contour projection image from the LCOS unit 108 into projection light including at least visible light. As shown in FIG. A method of projecting a projection image by the projection unit 42 will be described in detail with reference to FIG.

  In step S <b> 23, the image sensor 63 captures the gesture detectable range, and supplies the captured image I_pattern (FIG. 2) obtained as a result to the image processing unit 82.

  In step S <b> 24, the image processing unit 82 detects a position indicating the projected four corner portions based on each luminance value of the captured image I_pattern from the image sensor 63. Then, the image processing unit 82 detects a rectangular area surrounded by the four corner positions in the captured image I_pattern as a projection range on the captured image. Based on the detection result, the image processing unit 82 calculates the position and size of the projection range on the captured image, and holds the calculated projection range information in a built-in memory (not shown).

  In step S <b> 25, the LED control unit 81 drives an actuator (not shown) or the like according to control from the control unit 43, and moves the visible light cut filter 65 so that the lens 64 is covered with the visible light cut filter 65. .

  Note that the processes in steps S21 to S25 are calibration processes, and are executed only once, for example, at the start of the gesture recognition process.

  Thereafter, as processing for recognizing the user's gesture or the like, processing in steps S26 to S35 described later is repeatedly executed.

  In step S <b> 26, the LED control unit 81 turns on the LED 62 a at the timing when the image pickup of the image pickup device 63 (step S <b> 27) is performed in accordance with the frame synchronization signal from the image pickup device 63. Thereby, the LED 62a irradiates the subject with light having the wavelength λ1 while the image pickup device 63 is picked up. The LED 62b is turned off.

  In step S <b> 27, the imaging element 63 starts imaging of the subject irradiated with the light of the LED 62 a and supplies the captured image I_λ <b> 1 obtained as a result to the image processing unit 82.

  In step S28, the LED control unit 81 turns off the LED 62a at the timing when the imaging of the imaging device 63 in step S27 ends according to the frame synchronization signal from the imaging device 63.

  In addition, the LED control unit 81 turns on the LED 62b at the timing when the next imaging (processing in step S29) of the imaging device 63 starts in accordance with the frame synchronization signal from the imaging device 63. Accordingly, the LED 62b irradiates the subject with light having the wavelength λ2 while the next imaging of the imaging element 63 is performed. The LED 62a is turned off.

  In step S29, the image sensor 63 starts imaging the subject irradiated with the light from the LED 62b, and supplies the captured image I_λ2 obtained as a result to the image processing unit 82.

  In step S30, the LED control unit 81 turns off the LED 62b at the timing when the imaging of the imaging device 63 in step S29 ends according to the frame synchronization signal from the imaging device 63. As a result, both the LED 62a and the LED 62b are turned off.

  In step S <b> 31, the imaging element 63 starts imaging in a state where both the LED 62 a and the LED 62 b are turned off, and supplies the captured image I_off obtained as a result to the image processing unit 82.

  In step S32, the image processing unit 82 calculates the difference image I_diff (= {(I_λ1−I_λ2) / (I_λ1-I_off)} × 100) based on the captured images I_λ1, I_λ2, and I_off from the image sensor 63. To do.

  In step S33, the image processing unit 82 binarizes the calculated difference image I_diff using a predetermined threshold for binarization to calculate a binarized skin image I_skin.

  In step S34, the image processing unit 82, based on the calculated binarized skin image I_skin, represents a skin region representing the user's skin portion on the captured image obtained from the image sensor 63, that is, for example, the user's hand. Is detected (calculated).

  Further, the image processing unit 82 determines whether or not the calculated hand position is a position within the projection range based on the projection range information already held in the process of step S24.

  In step S35, when the image processing unit 82 determines that the calculated hand position is within the imaging range, the image processing unit 82 sets the position on the projection image projected by the projection unit 42 to a position corresponding to the position of the user's hand. , Display the cursor, etc.

That is, for example, if the image processing unit 82 determines that the calculated hand position is within the imaging range, the image processing unit 82 supplies position information indicating the position of the user's hand to the personal computer 22 together with the determination result. .
For example, when the user's hand is detected within the projection range, the image processing unit 82 may calculate the position of the hand within the imaging range including the projection range as position information,
The position of the hand within the projection range may be calculated as position information.
That is, for example, the position defined with the lower left end portion of the imaging range as the origin, the horizontal direction as the X axis, and the vertical direction as the Y axis may be calculated as position information, or the lower left end of the projection range A position defined with the portion as the origin, the horizontal direction as the X axis, and the vertical direction as the Y axis may be calculated as position information.
In addition, for example, the image processing unit 82 may calculate a change value representing a change between the previous position information and the current position information, and supply the change value to the personal computer 22.

  The personal computer 22 generates a projection image in which a cursor or the like is displayed at a position corresponding to the position of the user's hand on the projection image based on the determination result and the position information from the image processing unit 82. Control information is generated and supplied to the control unit 111 of the projection unit 42.

  The control unit 111 controls the laser light source units 101 to 103 and the LCOS unit 108 based on the control information from the personal computer 22 so that a cursor or the like is displayed at a position corresponding to the position of the user's hand. Project a projected image.

  In step S35, when the image processing unit 82 calculates the position of the user's face as the position of the skin region, and determines that the calculated position of the face is within the projection range, the control unit 111 is By controlling, the projection light from the projection unit 42 is not projected onto the face existing in the projection range. This is to reduce glare caused by the projection light from the projection unit 42.

  After the process of step S35 is completed, the process returns to step S26, and thereafter the same process is repeated.

  Note that this gesture recognition processing is ended when, for example, the user performs a power-off operation for turning off the power of the projector 22 using the operation unit 44.

  As described above, according to the gesture recognition process, the human hand or the like on the image is detected using the human spectral reflection characteristics as shown in FIG.

  Therefore, according to the gesture recognition process, it is possible to accurately detect a human hand or the like on the image regardless of the brightness of the illumination that illuminates the subject.

  Therefore, it is possible to accurately detect a human hand or the like on the captured image even when the brightness of the illumination illuminating the subject is insufficient (dark) as in the case of using the projector 21.

  Therefore, the projector 21 is relatively large, and even if the light projected as the projection image is weak, such as a pico projector that is about the same size as a mobile phone, the human hand on the captured image, etc. Can be detected with high accuracy.

  For this reason, the projector 21 can be reduced in size while maintaining the detection accuracy for detecting a human hand or the like.

  In the gesture recognition process, the binarized skin image I_skin representing the skin area and the non-skin area is calculated based on the captured images I_λ1, I_λ2, and I_off obtained by imaging. Then, the human posture or gesture can be recognized by a simple process of detecting the shape of the human hand from the calculated binarized skin image I_skin.

  For this reason, for example, complicated processing such as pattern matching for detecting a human hand or the like is performed by comparing each region on the image showing a human hand and a pattern image representing the human hand. Compared to the case, it is possible to reduce the load on the DSP, CPU, or the like that performs the gesture recognition processing.

  Therefore, for example, in the projector 21, it is not necessary to mount an expensive DSP in order to improve the processing capability. For this reason, since an inexpensive DSP can be used, the manufacturing cost of the projector 21 can be kept low.

  Further, for example, the projector 21 that performs the gesture recognition process is configured by parts that can be procured relatively easily, such as the LED 62a, the LED 62b, and the imaging element 63.

  Therefore, for example, the projector 21 can be manufactured at a lower cost than when a human gesture is recognized using a special and very expensive sensor.

  As a special and very expensive sensor, for example, the distance of each target is measured, and according to the measured distance, a distance sensor that detects a target such as a human hand or the temperature of each target is measured. A thermal sensor that detects an object such as a human hand in accordance with the measured temperature can be considered.

  By the way, when examining the recognition technique described in the background art, the present inventor causes a user to perform a gesture or the like within a projection range projected from a projection device such as a projector, and uses the projection range as an imaging range. I also considered taking images.

However, in this case, the projection image from the projection device is projected onto the user's hand or the like. For this reason, the projection image itself projected onto the hand or the like that appears on the captured image becomes noise, which makes it difficult to detect the user's hand or the like due to pattern matching or the like. .
Further, as a new usage of the projection device such as the projector 21, control is performed so that the projection image is operated as a touch panel by the movement of the user's hand or finger on the screen on which the projection image from the projection device is projected. Can be considered.
The projected part of the projected image is often brighter than the space outside the projected range, but if the projected image itself is dark or there is a dark part, even if you move your hand or finger there, it will still be a lack of illuminance. And fingers are not clearly visible, and pattern matching cannot be used.
On the other hand, when the projected image is bright, the problem of insufficient illumination as described above does not occur. However, when the screen is imaged, both the projected image and the operator's hand and fingers are captured. In extreme cases, there may be hands and fingers other than the operator on the projected image, and it is not possible to distinguish them from the hands and fingers of the operator by pattern matching or the like. Even if this is not the case, it is difficult to detect by pattern matching because the projected image is reflected on the hand or finger.

  Therefore, the present inventor has considered applying the present technology that uses spectral reflection characteristics as shown in FIG. 4 instead of pattern matching to the projector 21.

  Also in the present disclosure, when the user's skin area is detected within the projection range, the reflected light of the projected image projected on the user's skin portion may be incident on the image sensor 63.

  However, in consideration of such a situation, the projector 21 is provided with a visible light cut filter 65 so as to cover the front surface of the image sensor 63 (and the lens 64) as shown in FIG. For this reason, in the projector 21 of the present disclosure, a situation in which the detection accuracy of the skin region is lowered due to the reflected light of the projection image projected onto the skin portion of the user is suppressed.

  In addition, the present inventor previously stores each luminance value of a captured image obtained by imaging the gesture detectable range in a state where the gesture detectable range is irradiated with light of one wavelength (for example, light of wavelength λ1). A method of detecting the user's hand or the like by binarizing with a predetermined threshold was also considered.

  However, in this method, when the reflectance of the human skin with respect to light of one wavelength is close to the reflectance of the human background portion, the background portion of the human skin portion as well as the human skin portion can be converted into the skin by binarization. It will be detected as a region.

  Therefore, in the present disclosure, the difference between the reflectances of two different wavelengths is used to prevent a situation in which the background portion as well as the human skin portion is detected as the skin region.

  Furthermore, the present inventor also considers that a dark portion (a portion that becomes a shadow due to the presence of the user in the projection range) is detected as a portion where the user exists in the captured image obtained by imaging the projection range. It was.

  However, in an environment where the projector 21 or the like is used, since the surrounding brightness is not sufficient, the entire captured image becomes a dark image, and it is difficult to accurately detect a portion where the user exists. For this reason, this inventor came to invent this technique using two different wavelengths.

  By the way, in the gesture recognition process, in step S21, the LED control unit 81 controls an actuator (not shown) to move the visible light cut filter 65 so that the lens 64 covers the visible light cut filter 65. I tried not to break.

  In step S23, the imaging element 63 receives reflected light of the contour projection image projected onto the projection range as projection light including at least visible light, and generates a captured image I_pattern as shown in FIG. It is done to make it possible.

  Accordingly, in step S22, if the contour projection image is projected from the LCOS unit 108 to the projection range as invisible light, the invisible light as the reflected light of the projection image projected onto the projection range is cut by visible light. The light passes through the filter 65 and enters the image sensor 63. In this case, the process of step S21 and the process of step S25 can be omitted, and the image sensor 63 generates a captured image I_pattern.

  It should be noted that the visible light cut filter 65 is not provided, and the imaging device 63 is provided with a first light receiving element provided with a visible light cut filter that blocks visible light, and a second light received without a visible light cut filter. It can also be constituted by an element.

  In this case, the imaging element 63 can generate the captured image I_pattern and the captured image I_λ1 at the same time.

  That is, for example, when the light of wavelength λ1 is irradiated by the LED 62a and the projection light is projected by the LCOS unit 108, the reflected light of the light of wavelength λ1 and the projection projected on the projection range are projected on the image sensor 63. Reflected light or the like is incident.

  The imaging element 63 receives the reflected light of the irradiation light having the wavelength λ1 by the first light receiving element to generate a captured image I_λ1, and reflects the projection light projected on the projection range by the second light receiving element. The captured image I_pattern can be generated by receiving light or the like.

[Description of Operation of Projecting Unit 42 in FIG. 3]
Next, the projection processing performed by the projection unit 42 in FIG. 3 will be described with reference to the flowchart in FIG.

  This projection process is performed when the contour projection image is projected in step S22 of FIG. 5 in accordance with control from the control unit 43.

  In addition, this projection processing is performed in accordance with control from the personal computer 22, for example, during the execution of the processing from step S26 to step S35 in FIG.

  In step S <b> 51, the laser light source unit 103 irradiates the prism 105 with R component laser light in accordance with control from the control unit 111. As a result, the R component laser light is reflected by the prism 105 and applied to the polarization beam splitter 107 via the lens 106.

  The irradiated R component laser light is reflected by the polarization beam splitter 107 and irradiated to the LCOS unit 108. It is assumed that the R component laser light emitted from the prism 105 via the lens 106 to the polarization beam splitter 107 is S-polarized light.

  In step S <b> 52, the LCOS unit 108 generates an R component projection image based on the R component laser light emitted from the polarization beam splitter 107 in accordance with the control from the control unit 111. Then, the LCOS unit 108 irradiates the polarization beam splitter 107 with the generated R component projection image as P-polarized light.

  The polarization beam splitter 107 transmits the P-polarized light from the LCOS unit 108 and irradiates the projection range via the lenses 109 and 110. Thereby, an R component projection image is projected onto the projection range.

  In step S <b> 53, the laser light source unit 102 irradiates the prism 104 with a G component laser beam in accordance with control from the control unit 111. Thus, the G component laser light is reflected by the prism 104, passes through the prism 105, and is irradiated onto the polarization beam splitter 107 via the lens 106.

  The irradiated G component laser light is reflected by the polarization beam splitter 107 and applied to the LCOS unit 108. It is assumed that the G component laser light emitted from the prism 105 through the lens 106 to the polarization beam splitter 107 is S-polarized light.

  In step S <b> 54, the LCOS unit 108 generates a G component projection image based on the G component laser light emitted from the polarization beam splitter 107 in accordance with the control from the control unit 111. Then, the LCOS unit 108 irradiates the polarization beam splitter 107 with the generated projected image of the G component as P-polarized light.

  The polarization beam splitter 107 transmits the P-polarized light from the LCOS unit 108 and irradiates the projection range via the lenses 109 and 110. As a result, a G component projection image is projected onto the projection range.

  In step S <b> 55, the laser light source unit 101 irradiates the prism 104 with the B component laser light in accordance with the control from the control unit 111. As a result, the B component laser light passes through the prism 104 and the prism 105, and is applied to the polarization beam splitter 107 via the lens 106.

  The irradiated B component laser light is reflected by the polarization beam splitter 107 and applied to the LCOS unit 108. It is assumed that the B component laser light emitted from the prism 105 through the lens 106 to the polarization beam splitter 107 is S-polarized light.

  In step S <b> 56, the LCOS unit 108 generates a B component projection image based on the B component laser light emitted from the polarization beam splitter 107 in accordance with the control from the control unit 111. Then, the LCOS unit 108 irradiates the polarization beam splitter 107 with the generated B component projection image as P-polarized light.

  The polarization beam splitter 107 transmits the P-polarized light from the LCOS unit 108 and irradiates the projection range via the lenses 109 and 110. Thereby, a projected image of the B component is projected onto the projection range.

  After the end of step S56, the process returns to step S51, and thereafter the same process is repeated.

<2. Second Embodiment>
Next, FIG. 7 shows a second detailed configuration example of the skin detection unit 41 and the projection unit 42 included in the projector 21.

  In FIG. 7, parts that are configured in the same manner as in FIG. 3 are given the same reference numerals, and therefore description of those parts will be appropriately omitted below.

  That is, in the skin detection unit 41 of FIG. 7, a polarization conversion element 131a and a polarization conversion element 131b are provided on the front surface of the LED 62a and the front surface of the LED 62b, respectively, and between the polarization conversion element 131a and the polarization beam splitter 107. 3 except that a prism 132 is newly provided.

  7 differs from the case of FIG. 3 in that the skin detection unit 41 of FIG. 7 also includes the polarization beam splitter 107 of the projection unit 42 and the lenses 109 and 110.

  The polarization conversion element 131a is a conversion element for converting the irradiation light from the LED 62a into S-polarized light. The irradiation light from the LED 62a is composed of S-polarized light and P-polarized light (for example, approximately half of S-polarized light and P-polarized light), and almost all of the irradiated light passes through the polarization conversion element 131a ( For example, about 70% to 80%) is S-polarized light.

  Therefore, the irradiation light having the wavelength λ1 that is mostly S-polarized light is incident on the prism 132 via the polarization conversion element 131a from the LED 62a.

  The polarization conversion element 131b is a conversion element for converting the irradiation light from the LED 62b into S-polarized light. Note that the irradiation light from the LED 62b is composed of S-polarized light and P-polarized light (for example, approximately half of S-polarized light and P-polarized light), and almost all of the irradiated light passes through the polarization conversion element 131b ( For example, about 70% to 80%) is S-polarized light.

  Therefore, the irradiation light having the wavelength λ2 that is mostly S-polarized light is incident on the prism 132 via the polarization conversion element 131b from the LED 62b.

  The prism 132 transmits the irradiation light having the wavelength λ1 incident from the LED 62a via the polarization conversion element 131a, and the irradiation light having the wavelength λ2 incident from the LED 62b via the polarization conversion element 131b to the direction of the polarization beam splitter 107. To reflect.

  As a result, the irradiation light with the wavelength λ1 and the irradiation light with the wavelength λ2 are incident on the polarization beam splitter 107 via the prism 132. Note that most of the irradiation light having the wavelength λ1 is converted to S-polarized light by the polarization conversion element 131a. Moreover, most of the irradiation light with wavelength λ2 is converted to S-polarized light by the polarization conversion element 131b.

  Therefore, the polarization beam splitter 107 reflects the irradiation light with the wavelength λ1 and the irradiation light with the wavelength λ2 incident through the prism 132 in the direction in which the lens 109 and the lens 110 exist.

  In this case, most of the irradiation light with the wavelength λ 1 and the irradiation light with the wavelength λ 2 is S-polarized light, and most of the irradiation light is reflected by the polarization beam splitter 107. Therefore, the amount of irradiation light passing through the polarization beam splitter 107 as P-polarized light is very small.

  For this reason, it is possible to irradiate the subject with the irradiation light while maintaining almost the respective light amounts of the irradiation light with the wavelength λ1 and the irradiation light with the wavelength λ2.

  In the case of the configuration shown in FIG. 7, the gesture recognition process by the skin detection unit 41 and the projection process by the projection unit 42 are performed independently as in the first embodiment. In the case shown in FIG. 7, the imaging range and the projection range are the same. For this reason, the LED 62a and the LED 62b irradiate at least the imaging range (projection range).

<3. Third Embodiment>
Next, FIG. 8 shows a third detailed configuration example of the skin detection unit 41 and the projection unit 42 included in the projector 21.

  In FIG. 8, parts that are configured in the same manner as in FIG. 3 are given the same reference numerals, and therefore description of those parts will be omitted as appropriate.

  That is, in FIG. 8, the projection unit 42 is configured in the same manner as the projection unit 42 of FIG. 3 except that a prism 151a and a prism 151b are newly provided between the prism 105 and the lens 106.

  In the skin detection unit 41 of FIG. 8, a polarization conversion element 131a is provided on the front surface of the LED 62a, and a polarization conversion element 131b is provided on the front surface of the LED 62b. The polarization conversion elements 131a and 131b are the same as the polarization conversion elements 131a and 131b shown in FIG.

  Further, the lens 106, the polarization beam splitter 107, the LCOS unit 108, the lenses 109 and 110, and the prisms 151a and 151b of the projection unit 42 are the same as the skin detection unit 41 in FIG. Configured.

  Irradiation light having a wavelength λ1 is incident on the prism 151a via the polarization conversion element 131a from the LED 62a. The prism 151a is irradiated with irradiation light of wavelength λ2, laser light of R component, G component, and B component through the prism 151b.

  The prism 151a reflects the irradiation light having the wavelength λ1 incident from the LED 62a via the polarization conversion element 131a in the direction of the lens 106 and transmits the light incident through the prism 151b.

  Therefore, the polarization beam splitter 107 receives the irradiation light of wavelength λ1, the irradiation light of wavelength λ2, the R component laser light, the G component laser light, and the B component laser light from the prism 151a through the lens 106. .

  Irradiation light having a wavelength of λ2 is incident on the prism 151b from the LED 62b via the polarization conversion element 131b. Further, R, G, and B component laser beams are incident on the prism 151b through the prism 105.

  The prism 151 b reflects the irradiation light having the wavelength λ 2 incident from the LED 62 b through the polarization conversion element 131 b in the direction of the prism 151 a and transmits the laser light incident through the prism 105.

  Accordingly, irradiation light having a wavelength λ2, R component laser light, G component laser light, and B component laser light are incident on the prism 151a through the prism 151b.

[Description of operation when configured as shown in FIG. 8]
Next, a gesture recognition projection process performed by the projector 21 when configured as shown in FIG. 8 will be described with reference to the flowchart of FIG.

  This gesture recognition projection processing is started in response to control from the personal computer 22, for example. In this case, the gesture detection range and the projection range are the same range. Therefore, in the gesture recognition projection process, it is not necessary to perform the calibration process described in steps S21 to S25 in FIG.

  In steps S71 to S76, processing similar to that in steps S51 to S56 in FIG. 6 is performed, and a projection image is projected by the projection unit 42 in FIG.

  In step S77, the control unit 111 controls the LCOS unit 108 so as to reflect all pixels uniformly.

  In step S78, the LED control unit 81 controls the LED 62a at the timing at which the image pickup device 63 is picked up (the process of step S79) in accordance with the frame synchronization signal from the image pickup device 63, as in step S26 of FIG. Light up.

  Thereby, the irradiation light of wavelength λ1 passes through the polarization conversion element 131a from the LED 62a, and is reflected in the direction of the polarization beam splitter 107 by the prism 151a. When the irradiation light having the wavelength λ1 passes through the polarization conversion element 131a, most of the irradiation light is converted to S-polarized light by the polarization conversion element 131a.

  For this reason, the polarization beam splitter 107 reflects the S-polarized light of the irradiation light having the wavelength λ <b> 1 in the direction of the LCOS unit 108. The LCOS unit 108 irradiates the polarization beam splitter 107 with the irradiation light having the wavelength λ1 from the polarization beam splitter 107 as P-polarized light.

  The polarization beam splitter 107 transmits the irradiation light having the wavelength λ 1 from the LCOS unit 108 and irradiates the projection range via the lens 109 and the lens 110.

  Thereby, the LED 62a irradiates the subject within the projection range with the light having the wavelength λ1 while the image pickup device 63 is picked up. The LED 62b is turned off.

  In step S79, the same process as in step S27 of FIG. 5 is performed, and the captured image I_λ1 is supplied from the image sensor 63 to the image processing unit 82. In this case, the projection range is a gesture detectable range. Therefore, the imaging device 63 performs imaging using the projection range as the imaging range. The same applies to step S81 and step S83.

  In step S80, the LED control unit 81 turns off the LED 62a at the timing when the imaging of the image sensor 63 in step S79 ends according to the frame synchronization signal from the image sensor 63, as in step S28 of FIG. .

  Further, the LED control unit 81 turns on the LED 62b at the timing when the next imaging (processing in step S81) of the imaging device 63 starts in accordance with the frame synchronization signal from the imaging device 63.

  Thereby, the irradiation light of wavelength λ2 passes from the LED 62b through the polarization conversion element 131b, and is reflected by the prism 151b in the direction of the prism 151a (polarization beam splitter 107). Then, the irradiation light having the wavelength λ 2 passes through the prism 151 a and the lens 106 and is incident on the polarization beam splitter 107.

  When the irradiation light having the wavelength λ2 passes through the polarization conversion element 131b, most of the irradiation light is converted to S-polarized light by the polarization conversion element 131b.

  For this reason, the polarization beam splitter 107 reflects the S-polarized light of the irradiation light having the wavelength λ <b> 2 in the direction of the LCOS unit 108. The LCOS unit 108 irradiates the polarization beam splitter 107 with the irradiation light having the wavelength λ2 from the polarization beam splitter 107 as P-polarized light.

  The polarization beam splitter 107 transmits the irradiation light having the wavelength λ 2 from the LCOS unit 108 and irradiates the projection range via the lens 109 and the lens 110.

  Thereby, the LED 62b irradiates the subject within the projection range with the light of the wavelength λ2 while the next imaging of the imaging element 63 is performed. The LED 62a is turned off.

  In step S81, processing similar to that in step S29 in FIG. 5 is performed, and the captured image I_λ2 is supplied from the image sensor 63 to the image processing unit 82.

  In steps S82 to S87, processing similar to that in steps S30 to S35 of FIG. 5 is performed.

  In this case, since the gesture detectable range is a projection range, a gesture or the like performed within the projection range is detected as the position or shape of the user's hand, and a projection image is generated according to the detection result. A cursor or the like is displayed on the top.

  After the process of step S87 is completed, the control unit 111 controls the LCOS unit 108 to return the setting for reflecting all the pixels uniformly to the setting for projecting the projection image. The process returns to step S71, and thereafter the same process is repeated.

  Note that this gesture recognition projection processing is ended, for example, when the user operates the operation unit 44 to turn off the power.

  As described above, according to the gesture recognition projection processing, it is possible to accurately detect a human hand or the like on an image regardless of the brightness of the illumination that illuminates the subject.

  Further, in the gesture recognition process, it is possible to reduce the load on the DSP, CPU, or the like that executes the gesture recognition process, as compared with a case where complicated processes such as pattern matching are performed.

  Further, for example, the projector 21 that performs the gesture recognition processing is configured by easily procurable parts such as the LED 62a, the LED 62b, and the imaging element 63. For this reason, for example, the projector 21 can be manufactured at a lower cost than when a human gesture or the like is recognized using a special and very expensive sensor or the like.

  In addition, when irradiating light of wavelength λ1 and light of wavelength λ2, in step S77, the control unit 111 controls the LCOS unit 108 so as to reflect all pixels uniformly.

  For this reason, the light with the wavelength λ1 and the light with the wavelength λ2 are irradiated as uniform light onto the projection range, which is the gesture detectable range, so the light with the wavelength λ1 and the light with the wavelength λ2 are non-uniform Therefore, the situation where the detection accuracy of skin detection falls can be prevented.

<4. Modification>
In the projector 21, the imaging direction by the imaging element 63 can be fixed or variable. That is, for example, the skin detection unit 41 including the image sensor 63 can be driven, and the imaging direction by the image sensor 63 can be changed.

  Further, for example, the image processing unit 82 may convert the detected first position such as the user's hand into a second position corresponding to the imaging direction by the imaging element 63.

  That is, for example, when the imaging direction is the same as the projection direction of the projection image as shown in FIG. 1, the user existing in the imaging direction looks at the projection image projected in the projection direction in front of It seems to perform gestures.

  Specifically, for example, when it is desired to move the cursor or the like on the projection image in the right direction when viewed from the user, the user moves the hand or the like in the right direction while viewing the projection image in front.

  In this case, since the direction in which the cursor or the like on the projected image is desired to be moved coincides with the direction in which the user moves his / her hand or the like, the image processing unit 82 first detects the detected user's hand or the like. This position can be used as it is as a gesture for recognizing the locus of the moving hand as a gesture.

  However, for example, when the imaging direction is exactly opposite to the projection direction of the projection image, the user existing in the imaging direction faces the projection range in front and projects the projection image (projected onto the projection range) displayed on the personal computer 22. It is conceivable to perform a gesture or the like within the imaging range while viewing the same projected image).

  That is, for example, when it is desired to move the cursor or the like on the projection image in the right direction in FIG. 1, the user looks in the right direction (right direction in FIG. 1) as viewed from the user while looking at the projector 21 in front. Move hands etc.

In this case, when viewed from the image sensor 63, the user's hand moves to the left. Therefore, when the cursor or the like on the projection image is moved to the continuous first position such as the user's hand detected by the image processing unit 82 using the captured image obtained by the imaging element 63, the cursor is It will move to the left.
Note that the image processing unit 82 detects the movement of the user's hand or the like based on the continuous first position of the user's hand or the like, for example.

Therefore, the image processing unit 82 converts the detected first position into, for example, a second position that is symmetrical with respect to the center line extending in the vertical direction on the captured image, and the converted second position. Is used in the subsequent processing.
In this case, the image processing unit 82 detects the movement of the user's hand or the like based on the continuous second position of the user's hand or the like, for example.

  As a result, the image processing unit 82 can recognize that the user's hand has moved in the right direction in FIG. 1 instead of in the left direction in FIG. 1, and the cursor on the projected image is to be moved. 1 can be moved to the right.

  In the second embodiment, as shown in FIG. 7, using the polarization beam splitter 107, the irradiation light of wavelength λ1 incident from the LED 62a via the prism 132 and the incident light from the LED 62b via the prism 132 are incident. The irradiation light having the wavelength λ2 is reflected in the direction of the projection range.

  However, for example, by using a mirror or the like, the irradiation light with the wavelength λ1 and the irradiation light with the wavelength λ2 may be reflected in the direction of the projection range.

  In the first to third embodiments, the projector 21 may be provided with an acceleration sensor, for example, so that the movement of the projector 21 can be detected. When the movement of the projector 21 is detected, calibration processing for calculating the projection range on the captured image may be performed again.

  In this case, even when the projection range is changed due to the movement of the projector 21, the changed projection range can be calculated by a second calibration process.

In addition, this technique can also take the following structures.
(1) A first irradiating unit that irradiates a subject with light having a first wavelength, and a second irradiating the subject with light having a second wavelength that is longer than the first wavelength. An irradiating unit, an imaging unit for imaging the subject, a first captured image obtained by irradiating with light of the first wavelength, and the second wavelength with respect to the subject A projection for projecting a detection unit that detects a skin region of the subject and a projection image that is changed according to a detection result of the detection unit based on a second captured image obtained by imaging when light is irradiated A projection apparatus.
(2) The projection apparatus according to (1), wherein the imaging unit performs imaging of the subject in an imaging range including a projection range on which the projection image is projected.
(3) The projection apparatus according to (2), further including a calculation unit that calculates the projection range on a captured image obtained by imaging of the imaging unit.
(4) The projection unit also projects a contour projection image representing the contour of the projection range, and the calculation unit calculates the projection range on the captured image obtained by imaging when the contour projection image is projected. The projection apparatus according to (3), wherein the calculation is performed.
(5) The first irradiation unit irradiates invisible light of the first wavelength, the second irradiation unit irradiates invisible light of the second wavelength, and the projection unit includes the contour. The projection device according to (4), wherein the projection image is projected as one of projection light including at least visible light or invisible light.
(6) The projection unit projects the contour projection image as the projection light, and the imaging unit includes a first light receiving element provided with a visible light blocking unit that blocks visible light, and the visible light blocking. A second light receiving element that is not provided with a portion, the first light receiving element generates the first and second captured images, and the second light receiving element The projection apparatus according to (5), wherein the pattern image is generated.
(7) The projection unit projects the contour projection image as invisible light, and the imaging unit includes an imaging element provided with a visible light blocking unit that blocks visible light. The projection apparatus according to (5), wherein the pattern image is generated in addition to the first and second captured images.
(8) A visible light blocking unit that blocks visible light, and the visible light blocking unit is moved so that visible light is incident on the imaging device when the projection light is projected as the contour projection image. And a movement control unit that moves the visible light blocking unit so that visible light is not incident on the imaging device when the first wavelength light or the second wavelength light is irradiated. The projection device according to (5).
(9) The projection unit includes a reflection unit that reflects light incident from a predetermined direction to the projection range, and the first irradiation unit emits the light having the first wavelength. (2) to (8) wherein the second irradiation unit reflects the light of the second wavelength to the reflection unit and irradiates the projection range with reflection on the reflection unit. ).
(10) The first irradiation unit irradiates the projection range by reflecting the light of the first wavelength to the reflection unit that reflects each pixel value as a projection image having a uniform pixel value, and the second irradiation. The projection apparatus according to (9), wherein the unit reflects the light of the second wavelength to the reflection unit that reflects the projection image with uniform pixel values and irradiates the projection range.
(11) The light of the first wavelength irradiated from the first irradiation unit is reflected by the reflection unit among the first polarized light or the second polarized light different from the first polarized light. The first polarized light converting unit that converts the first polarized light, and the second wavelength light emitted from the second irradiating unit is the first polarized light or the second polarized light. The projection apparatus according to (9), further including a second polarization conversion unit that converts the first polarized light reflected by the reflection unit.
(12) It further includes a motion detection unit that detects the motion of the projection device, and the calculation unit also includes the projection range on the captured image even when the motion detection unit detects the motion of the projection device. The projection apparatus according to (3) or (4), wherein:
(13) The projection device according to (1), wherein the projection unit projects the projection image in a range excluding the detected portion of the skin region in the projection range.
(14) The projection device according to (1), wherein the imaging unit is capable of changing an imaging direction.
(15) a position conversion unit that converts a first position on the captured image in which the skin region is detected to a second position corresponding to an imaging direction of the imaging unit; and the projection unit includes: The projection apparatus according to (14), wherein the projection image changed according to the second position is projected.
(16)
The first irradiation unit irradiates the projection range with light of the first wavelength, the second irradiation unit irradiates the projection range with light of the second wavelength, and the detection unit The projection apparatus according to (2), wherein the skin region within the projection range is detected based on the first captured image and the second captured image.
(17) The first wavelength λ1 and the second wavelength λ2 are:
640nm ≤ λ1 ≤ 1000nm
900nm ≤ λ2 ≤ 1100nm
The projection device according to (1), wherein
(18) The first irradiation unit irradiates invisible light with the first wavelength λ1, and the second irradiation unit irradiates invisible light with the second wavelength λ2. Projection device.
(19) The projection apparatus according to (3), further including an output unit that outputs the detection result in the projection range as a signal corresponding to a position in the projection range.
(20) In the projection method of the projection device, a first irradiation step of irradiating the subject with light of the first wavelength by the projection device, and the subject when the light of the first wavelength is irradiated A first imaging step that performs imaging, a second irradiation step that irradiates the subject with light having a second wavelength that is longer than the first wavelength, and light having the second wavelength. A second imaging step for imaging the subject when irradiated with light, a first captured image obtained by imaging when irradiated with light of the first wavelength, and the subject with respect to the first A detection step for detecting a skin region of the subject based on a second captured image obtained by imaging when light of two wavelengths is irradiated, and a projection image that is changed according to the detection result of the detection step A projection step of projecting Shadow method.
(21) A first imaging step for causing a computer to irradiate a subject with light having a first wavelength, and a first imaging for causing the subject to take an image when the subject has been irradiated with light having the first wavelength. A step of irradiating the subject with light of a second wavelength that is longer than the first wavelength, and the subject of irradiating the subject with light of the second wavelength A second imaging step for performing imaging of the first, a first captured image obtained by imaging when the light of the first wavelength is irradiated, and the light of the second wavelength irradiated to the subject A detection step for detecting a skin region of the subject based on a second captured image obtained by imaging at the time of imaging, and a projection step for projecting a projection image that is changed according to the detection result of the detection step To execute a process that includes Of the program.

[Computer configuration example]
FIG. 10 is a block diagram illustrating an example of a hardware configuration of a computer that executes the above-described series of processes using a program.

  A CPU (Central Processing Unit) 201 executes various processes according to a program stored in a ROM (Read Only Memory) 202 or a storage unit 208. A RAM (Random Access Memory) 203 appropriately stores programs executed by the CPU 201, data, and the like. These CPU 201, ROM 202, and RAM 203 are connected to each other by a bus 204.

  An input / output interface 205 is also connected to the CPU 201 via the bus 204. Connected to the input / output interface 205 are an input unit 206 composed of a keyboard, a mouse, a microphone, and the like, and an output unit 207 composed of a display, a speaker, and the like. The CPU 201 executes various processes in response to commands input from the input unit 206. Then, the CPU 201 outputs the processing result to the output unit 207.

  A storage unit 208 connected to the input / output interface 205 includes, for example, a hard disk, and stores programs executed by the CPU 201 and various data. The communication unit 209 communicates with an external device via a network such as the Internet or a local area network.

  Further, a program may be acquired via the communication unit 209 and stored in the storage unit 208.

  The drive 210 connected to the input / output interface 205 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives programs and data recorded there. Etc. The acquired program and data are transferred to and stored in the storage unit 208 as necessary.

  As shown in FIG. 10, a recording medium for recording (storing) a program installed in a computer and ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc- Removable media 211, which is a package media made up of read only memory), DVD (digital versatile disc), magneto-optical disc (including MD (mini-disc)), or semiconductor memory, or the program is temporarily or It is composed of a ROM 202 that is permanently stored, a hard disk that constitutes the storage unit 208, and the like. Recording of a program on a recording medium is performed using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 209 that is an interface such as a router or a modem as necessary. Is called.

  In the present specification, the steps describing the series of processes described above are not limited to the processes performed in time series according to the described order, but are not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  The embodiments in the present disclosure are not limited to the first to third embodiments described above, and various modifications can be made without departing from the scope of the present disclosure.

  21 projector, 22 personal computer, 41 skin detection unit, 42 projection unit, 43 control unit, 44 operation unit, 61 DSP, 62a, 62b LED, 63 image sensor, 64 lens, 65 visible light cut filter, 81 LED control unit, 82 Image processing unit, 101, 102, 103 Laser light source unit, 104, 105 prism, 106 lens, 107 polarization beam splitter, 108 LCOS unit, 109, 110 lens, 111 control unit, 131a, 131b polarization conversion element, 132, 151a , 151b prism

Claims (21)

A first irradiating unit that irradiates a subject with light having a first wavelength;
A second irradiation unit configured to irradiate the subject with light having a second wavelength that is longer than the first wavelength;
An imaging unit for imaging the subject;
A first captured image obtained by imaging when the light of the first wavelength is irradiated and a second imaging obtained by imaging when the subject is irradiated with the light of the second wavelength A detection unit for detecting a skin area of the subject based on an image;
A projection unit that projects a projection image that is changed according to a detection result of the detection unit. The projection apparatus according to claim 1, wherein the imaging unit images the subject in an imaging range including a projection range on which the projection image is projected.   The projection device according to claim 2, further comprising a calculation unit that calculates the projection range on a captured image obtained by imaging of the imaging unit. The projection unit also projects a contour projection image representing a contour of the projection range,
The projection device according to claim 3, wherein the calculation unit calculates the projection range on the captured image obtained by imaging when the contour projection image is projected. The first irradiation unit irradiates invisible light of the first wavelength,
The second irradiation unit irradiates invisible light of the second wavelength,
The projection device according to claim 4, wherein the projection unit projects the contour projection image as one of projection light including at least visible light or invisible light. The projection unit projects the contour projection image as the projection light,
The imaging unit includes an imaging device including a first light receiving element provided with a visible light blocking unit that blocks visible light, and a second light receiving element not provided with the visible light blocking unit. The projection apparatus according to claim 5, wherein the first and second captured images are generated by the first light receiving element, and the pattern image is generated by the second light receiving element. The projection unit projects the contour projection image as invisible light,
6. The imaging unit includes an imaging element provided with a visible light blocking unit that blocks visible light, and the imaging element generates the pattern image in addition to the first and second captured images. The projection apparatus described in 1. A visible light blocking unit that blocks visible light; and
As the contour projection image, when the projection light is projected, the visible light blocking unit is moved so that visible light is incident on the imaging element,
And a movement control unit that moves the visible light blocking unit so that visible light is not incident on the imaging device when the first wavelength light or the second wavelength light is irradiated. Item 6. The projection device according to Item 5. The projection unit includes a reflection unit that reflects light incident from a predetermined direction to the projection range,
The first irradiation unit reflects the light of the first wavelength to the reflection unit to irradiate the projection range,
The projection apparatus according to claim 2, wherein the second irradiation unit irradiates the projection range with the light having the second wavelength reflected by the reflection unit. The first irradiation unit irradiates the projection range by reflecting the light of the first wavelength to the reflection unit that reflects each pixel value as a projection image having a uniform pixel value,
The projection device according to claim 9, wherein the second irradiation unit irradiates the projection range with the light having the second wavelength reflected by the reflection unit that reflects each pixel value as a projection image having a uniform pixel value. The first wavelength that is reflected by the reflection unit among the first polarized light or the second polarized light different from the first polarized light is emitted from the first irradiation unit. A first polarization converter that converts the polarized light into
Second light that converts light of the second wavelength irradiated from the second irradiation unit into the first polarized light reflected by the reflection unit out of the first polarized light or the second polarized light. The projection apparatus according to claim 9, further comprising: a polarization conversion unit. A motion detector for detecting the motion of the projection device;
The projection device according to claim 3, wherein the calculation unit calculates the projection range on the captured image when the motion detection unit detects a motion of the projection device. The projection device according to claim 1, wherein the projection unit projects the projection image in a range excluding the detected portion of the skin region in the projection range. The projection device according to claim 1, wherein the imaging unit is capable of changing an imaging direction. A conversion unit that converts a first position on the captured image in which the skin region is detected into a second position corresponding to an imaging direction of the imaging unit;
The projection device according to claim 14, wherein the projection unit projects the projection image changed according to the second position. The first irradiation unit irradiates the projection range with light of the first wavelength,
The second irradiation unit irradiates the projection range with light of the second wavelength,
The projection device according to claim 2, wherein the detection unit detects the skin region within the projection range based on the first captured image and the second captured image. The first wavelength λ1 and the second wavelength λ2 are:
640nm ≤ λ1 ≤ 1000nm
900nm ≤ λ2 ≤ 1100nm
The projection apparatus according to claim 1, wherein: The first irradiation unit irradiates invisible light having the first wavelength λ1,
The projection apparatus according to claim 17, wherein the second irradiation unit irradiates invisible light having the second wavelength λ2. The projection apparatus according to claim 3, further comprising: an output unit that outputs the detection result in the projection range as a signal corresponding to a position in the projection range. In the projection method of the projection device,
By the projection device,
A first irradiation step of irradiating a subject with light of a first wavelength;
A first imaging step of imaging the subject when irradiated with light of the first wavelength;
A second irradiation step of irradiating the object with light having a second wavelength that is longer than the first wavelength;
A second imaging step of imaging the subject when irradiated with light of the second wavelength;
A first captured image obtained by imaging when the light of the first wavelength is irradiated and a second imaging obtained by imaging when the subject is irradiated with the light of the second wavelength A detection step of detecting a skin area of the subject based on an image;
A projection step of projecting a projection image that is changed according to the detection result of the detection step. On the computer,
A first irradiation step of irradiating the subject with light of a first wavelength;
A first imaging step for imaging the subject when the light of the first wavelength is irradiated;
A second irradiation step of irradiating the object with light having a second wavelength that is longer than the first wavelength;
A second imaging step for performing imaging of the subject when irradiated with light of the second wavelength;
A first captured image obtained by imaging when the light of the first wavelength is irradiated and a second imaging obtained by imaging when the subject is irradiated with the light of the second wavelength A detection step of detecting a skin area of the subject based on an image;
And a projection step of projecting a projection image that is changed according to the detection result of the detection step.

Images