US20110142349A1 - Information processing apparatus and information processing method - Google Patents

Information processing apparatus and information processing method Download PDF

Info

Publication number
US20110142349A1
US20110142349A1 US13/058,948 US201013058948A US2011142349A1 US 20110142349 A1 US20110142349 A1 US 20110142349A1 US 201013058948 A US201013058948 A US 201013058948A US 2011142349 A1 US2011142349 A1 US 2011142349A1
Authority
US
United States
Prior art keywords
image
pixels
luminance values
interest
captured
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/058,948
Other languages
English (en)
Inventor
Nobuhiro Saijo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SAIJO, NOBUHIRO
Publication of US20110142349A1 publication Critical patent/US20110142349A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/143—Sensing or illuminating at different wavelengths
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof

Definitions

  • the present invention relates to an information processing apparatus and an information processing method, and more particularly to, an information processing apparatus and an information processing method that are suitable in a case where a shape of a hand of a user or the like is extracted from a captured image obtained by capturing an image of the user, for example.
  • data is input by using, for example, a gesture or a posture of a hand of a user in the data input technique, it is necessary to precisely extract a shape of the hand of the user from a captured image obtained by capturing an image of the user.
  • extraction techniques for extracting the shape of the hand of the user there are a pattern matching method using pattern matching of images, a skin area extraction method of extracting a skin area of the user, and the like.
  • a plurality of shape images obtained by capturing images of hands having various shapes and sizes are learned in advance, and a shape of a hand represented in a shape image that is most similar to the captured image (for example, shape image having a minimum sum of differences between pixel values of corresponding pixels) is extracted as the shape of the hand of the user.
  • the shape of the hand is difficult to be precisely extracted as compared to a case where a shape of the face is extracted, for example.
  • a skin area representing a skin of the user within the captured image is extracted using skin information expressing colors of the human skin.
  • an information processing apparatus for detecting a plurality of pixels of interest within an image.
  • the information processing apparatus includes a first memory configured to store a first image captured using light of a first wavelength and a second image captured using light of a second wavelength, which is different from the first wavelength.
  • the information processing apparatus further includes at least one processor configured to detect a plurality of pixels of interest within the first captured image based on luminance values of the stored first and second captured images.
  • an information processing apparatus includes a memory and at least one processor.
  • the memory is configured to store a processed image that is generated from an image and includes a plurality of pixels of interest.
  • the at least one processor is configured to determine frequencies of luminance values of the plurality of pixels of interest in the processed image, and to determine a range of luminance values corresponding to a predetermined object within the processed image based on the determined frequencies of the luminance values.
  • FIG. 1 is a block diagram showing a structure example of an information processing system.
  • FIG. 2 is a block diagram showing a structure example of an information processing apparatus.
  • FIG. 3 is a diagram showing an example of reflection characteristics of a human skin.
  • FIG. 4 are diagrams showing examples of first and second captured images.
  • FIG. 5 is a diagram showing an example of a binarized skin image generated by a binarization section.
  • FIG. 6 is a diagram showing an example of a skin image extracted by a skin extraction section.
  • FIG. 7 is a diagram showing an example of a histogram of a skin image.
  • FIG. 8 is a diagram showing an example of a mask image generated by a mask image generation section.
  • FIG. 9 is a diagram showing an example of an extracted image generated by a shape extraction section.
  • FIG. 10 is a flowchart for explaining shape extraction processing.
  • FIG. 11 is a diagram showing the first captured image that is used in FFT threshold value determination processing.
  • FIG. 12 is a flowchart for explaining the FFT threshold value determination processing.
  • FIG. 13 is a diagram showing relative sensitivity characteristics of a camera.
  • FIG. 14 is a diagram showing an arrangement method for LEDs.
  • FIG. 15 is a block diagram showing a structure example of a computer.
  • FIG. 1 shows a structure example of an information processing system 1 of this embodiment.
  • the information processing system 1 executes predetermined processing in accordance with a gesture (or posture) made by using a hand of a user and includes an information processing apparatus 21 , a camera 22 , and a light-emitting apparatus 23 .
  • the user changes a shape of his/her own hand (in front of lens surface of camera 22 ).
  • the information processing system 1 recognizes the shape of the hand of the user and executes the predetermined processing in accordance with the recognition result.
  • the user changes the shape of the hand in front of the lens surface of the camera 22 and makes a gesture (or posture) by moving his/her hand toward a position closer to the lens surface of the camera 22 than his/her face, chest, or the like.
  • the information processing apparatus 21 controls the camera 22 and the light-emitting apparatus 23 . Further, the information processing apparatus 21 recognizes the shape of the hand of the user based on a captured image captured by the camera 22 , and executes the predetermined processing in accordance with the recognition result.
  • the camera 22 includes a lens used for capturing an image of a subject such as a user, and a front surface of the lens is covered with a visible light cut filter 22 a that cuts off visible light.
  • the camera 22 receives only reflected light of invisible light that is irradiated onto a subject by the light-emitting apparatus 23 , except infrared components of fluorescent light or sunlight, and supplies the resultant captured image to the information processing apparatus 21 .
  • the camera 22 receives only reflected light of light having a first wavelength, the light being invisible light irradiated onto a subject by the light-emitting apparatus 23 (for example, near-infrared light of 870 nm), and supplies the resultant first captured image to the information processing apparatus 21 .
  • the light-emitting apparatus 23 for example, near-infrared light of 870 nm
  • the camera 22 receives only reflected light of light having a second wavelength different from the first wavelength, the light being invisible light irradiated onto the subject by the light-emitting apparatus 23 (for example, near-infrared light of 950 nm), and supplies the resultant second captured image to the information processing apparatus 21 .
  • the light-emitting apparatus 23 for example, near-infrared light of 950 nm
  • the light-emitting apparatus 23 includes LEDs (light emitting diodes) 23 a 1 and 23 a 2 that emit light having the first wavelength and LEDs 23 b 1 and 23 b 2 that emit light having the second wavelength.
  • LEDs light emitting diodes
  • the LEDs 23 a 1 and 23 a 2 need not to be distinguished from each other hereinafter, the LEDs 23 a 1 and 23 a 2 are referred to simply as LEDs 23 a. Further, in a case where the LEDs 23 b 1 and 23 b 2 need not to be distinguished from each other, the LEDs 23 b 1 and 23 b 2 are referred to simply as LEDs 23 b.
  • the LEDs 23 a and 23 b alternately emit light under control of the information processing apparatus 21 .
  • outputs of the LEDs 23 a and LEDs 23 b are adjusted so that intensities (amounts of light) of the reflected light received by the camera 22 become equal in the reflected light of the light having the first wavelength and the reflected light of the light having the second wavelength.
  • the LEDs 23 a and LEDs 23 b are alternately arranged in a grid as shown in FIG. 1 and a diffuser plate 23 c that uniformly diffuses light emitted from the LEDs 23 a and LEDs 23 b is provided in front of the LEDs 23 a and LEDs 23 b.
  • a diffuser plate 23 c that uniformly diffuses light emitted from the LEDs 23 a and LEDs 23 b is provided in front of the LEDs 23 a and LEDs 23 b.
  • the light-emitting apparatus 23 is arranged at a position where the light emitted from the LEDs 23 a or LEDs 23 b is reliably irradiated onto at least a hand of a user.
  • the user changes a shape of a hand in front of the lens surface of the camera 22 , and accordingly the light-emitting apparatus 23 is arranged close to the camera 22 , for example.
  • FIG. 2 shows a structure example of the information processing apparatus 21 .
  • the information processing apparatus 21 includes a controller 41 , a binarization section 42 , a skin extraction section 43 , a threshold value determination section 44 , a mask image generation section 45 , and a shape extraction section 46 .
  • the controller 41 controls the light-emitting apparatus 23 and causes the LEDs 23 a and LEDs 23 b to emit light alternately.
  • the binarization section 42 is supplied with the first captured image and the second captured image from the camera 22 . Based on the first and second captured images supplied from the camera 22 , the binarization section 42 extracts (detects) pixels of interest.
  • the pixels of interest correspond to one or more skin areas representing the skin of the user and an area excluding the skin area from the first captured image.
  • the binarization section 42 generates a binarized skin image obtained by binarizing pixel values of pixels constituting the extracted skin area and pixel values of pixels constituting the area excluding the skin area into different values (for example, 0 and 1), and supplies the binarized skin image to the skin extraction section 43 and the shape extraction section 46 .
  • the skin extraction section 43 and the mask image generation section 45 are supplied with the first captured image from the camera 22 .
  • the skin extraction section 43 Based on the binarized skin image supplied from the binarization section 42 , the skin extraction section 43 extracts an area corresponding to the skin area within the binarized skin image (area representing skin area of user) from the first captured image supplied from the camera 22 .
  • the skin extraction section 43 generates a skin image including the extracted area and supplies the skin image to the threshold value determination section 44 . It should be noted that the skin extraction section 43 may supply the extracted area as a skin image to the threshold value determination section 44 .
  • the threshold value determination section 44 creates a histogram of a processed image such as the skin image (luminance values of pixels constituting skin image) based on the skin image supplied from the skin extraction section 43 . Then, the threshold value determination section 44 determines a mask threshold value that is used for generating a mask image (described later) based on the created histogram of the skin image and supplies the mask threshold value to the mask image generation section 45 .
  • the mask image generation section 45 generates a mask image from the first captured image supplied from the camera 22 based on the mask threshold value supplied from the threshold value determination section 44 , and supplies the mask image to the shape extraction section 46 .
  • the mask image is an image obtained by binarizing the first captured image into a mask area constituted of the pixels having luminance values within a range of luminance values specified by the mask threshold value and a non-mask area excluding the mask area.
  • the shape extraction section 46 Based on the mask image from the mask image generation section 45 , the shape extraction section 46 extracts at least one predetermined object corresponding to a shape area representing the shape of the hand of the user, for example, as an area corresponding to the mask area within the mask image, from the binarized skin image supplied from the binarization section 42 .
  • the shape extraction section 46 recognizes the shape of the hand based on the extracted shape area, performs processing corresponding to the recognition result, and outputs the processing result to a subsequent stage.
  • the binarization section 42 extracts the skin area and the area excluding the skin area from the first captured image, but the binarization section 42 may extract a skin area and an area excluding the skin area from the second captured image.
  • the skin extraction section 43 and the mask image generation section 45 are supplied with the second captured image from the camera 22 , instead of the first captured image.
  • the skin extraction section 43 generates a skin image from the second captured image and the mask image generation section 45 generates a mask image from the second captured image.
  • FIGS. 3 and 4 the first captured image and the second captured image that are captured by the camera 22 will be described. Further, in FIG. 5 , a binarized skin image generated by the binarization section 42 based on the first captured image and the second captured image will be described.
  • FIG. 3 shows reflection characteristics of a human skin with respect to irradiation light having different wavelengths.
  • the reflection characteristics are universal irrespective of a difference in color of the human skin (difference in race) or a state of the skin (suntan or the like).
  • the horizontal axis represents a wavelength of light to be irradiated to the human skin
  • the vertical axis represents a reflectance of the light irradiated to the human skin
  • the reflectance of the light irradiated to the human skin sharply decreases from the vicinity of 900 nm with the vicinity of 800 nm as a peak, and increases again with the vicinity of 1,000 nm as a minimum value.
  • a reflectance of reflected light that is obtained by irradiating light having a wavelength of 870 nm to the human skin is 63% and a reflectance of reflected light that is obtained by irradiating light having a wavelength of 950 nm to the human skin is 50%.
  • the above phenomenon is peculiar to the human skin, and regarding objects other than the skin of humans (for example, hair or clothes), a change in reflectance often becomes gentle in the vicinity of 800 to 1,000 nm.
  • FIG. 4 show examples of a first captured image obtained by receiving reflected light of light that has a wavelength of 870 nm and is irradiated to a user, and a second captured image obtained by receiving reflected light of light that has a wavelength of 950 nm and is irradiated to a user.
  • FIG. 4A shows the first captured image in which a face 61 and a hand 62 of the user are shown as a skin area of the user, and a shirt 63 that the user wears and a background 64 are shown as an area excluding the skin area of the user.
  • FIG. 4B shows the second captured image in which a face 81 and a hand 82 of the user are shown as a skin area of the user, and a shirt 83 that the user wears and a background 84 are shown as an area excluding the skin area of the user.
  • the reflectance of the light having the wavelength of 870 nm is larger than the reflectance of the light having the wavelength of 950 nm.
  • luminance values of pixels constituting the skin area of the user (face 61 and hand 62 ) within the first captured image take larger values than luminance values of pixels constituting the skin area of the user (face 81 and hand 82 ) within the second captured image.
  • differences obtained by subtracting the luminance values of the pixels constituting the skin area of the user within the second captured image from the luminance values of the pixels constituting the corresponding skin area of the user within the first captured image take positive values.
  • the reflectance of the light having the wavelength of 870 nm is equal to or smaller than that of the light having the wavelength of 950 nm in some cases.
  • the light having the wavelength of 870 nm is irradiated to the user, as reflected light of the light irradiated to the portion excluding the skin portion of the user, light that is as bright as or darker than the reflected light of the light having the wavelength of 950 nm enters the lens of the camera 22 .
  • luminance values of pixels constituting the area excluding the skin area of the user (shirt 63 and background 64 ) within the first captured image take values equal to or smaller than luminance values of pixels constituting the area excluding the skin area of the user (shirt 83 and background 84 ) within the second captured image.
  • differences obtained by subtracting the luminance values of the pixels constituting the skin portion of the user within the second captured image from the luminance values of the pixels constituting the corresponding portion excluding the skin portion of the user within the first captured image take values equal to or smaller than 0 (values excluding positive values).
  • the binarization section 42 calculates differences between luminance values of corresponding pixels of the first captured image and the second captured image and extracts pixels of interest (e.g., the skin area) and the area excluding the skin area of the user based on the calculated differences. Then, the binarization section 42 generates a binarized skin image in which the extracted skin area of the user is represented by a value 1 and the area excluding the extracted skin area of the user is represented by a value 0.
  • the binarization section 42 extracts the corresponding pixels as those constituting the skin area of the user, and in a case where the calculated differences are not positive values, extracts the corresponding pixels as those constituting the area excluding the skin area of the user.
  • the binarization section 42 sets each of the values of the pixels extracted as those constituting the skin area of the user to 1, and each of the values of the pixels extracted as those constituting the area excluding the skin area of the user to 0 to thereby generate a binarized skin image, and supplies the binarized skin image to the skin extraction section 43 and the shape extraction section 46 .
  • the differences calculated for the portion excluding the skin portion are smaller than those calculated for the skin portion but take positive values may occur depending on a reflectance in the portion excluding the skin option of the user. Therefore, in a case where the differences take positive values but are smaller than a predetermined threshold value, it may be desirable to assume that the differences are those of the portion excluding the skin portion of the user and set the value 0 for that portion.
  • the binarization section 42 may calculate difference absolute values between luminance values of corresponding pixels of the first captured image and the second captured image, and based on whether the calculated difference absolute values are equal to or larger than a predetermined threshold value, extract the skin portion (skin area) of the user and the portion excluding the skin portion (area excluding the skin area) to generate a binarized skin image.
  • the above operation uses the fact that due to the reflection characteristics, the difference absolute values corresponding to the skin portion of the user take relatively large values and those corresponding to the portion excluding the skin portion of the user take relatively small values.
  • FIG. 5 shows an example of the binarized skin image generated by the binarization section 42 .
  • a portion shown in black indicates a skin area represented by the value 1.
  • the skin area includes a face area 101 indicating a skin portion of the face of the user, and a hand area 102 indicating a skin portion of the hand of the user.
  • the face area 101 shown in FIG. 5 includes eyebrows, eyes, hair, and the like in addition to the skin portion of the face for convenience of the illustration, but the face area 101 is constituted of only the skin portion of the face in actuality.
  • a portion shown in white indicates an area excluding the skin area and is represented by the value 0.
  • the binarization section 42 supplies the generated binarized skin image to the skin extraction section 43 and the shape extraction section 46 .
  • the skin extraction section 43 extracts, from the first captured image supplied from the camera 22 , an area corresponding to the face area 101 and the hand area 102 within the binarized skin image (area including face 61 and hand 62 ) based on the binarized skin image supplied from the binarization section 42 . Then, the skin extraction section 43 generates a skin image including the extracted area.
  • a processed image e.g., a skin image
  • FIG. 6 shows an example of the skin image extracted by the skin extraction section 43 .
  • the skin image shown in FIG. 6 shows the face 61 and the hand 62 of the user.
  • the skin image shown in FIG. 6 includes eyebrows, eyes, hair, and the like as the face 61 of the user in addition to the skin portion of the face for convenience of the illustration, but the face 61 shown in FIG. 6 represents only the skin portion of the face in actuality.
  • the skin extraction section 43 multiplies the luminance values of the pixels of the binarized skin image supplied from the binarization section 42 and those of corresponding pixels of the first captured image supplied from the camera 22 .
  • the skin extraction section 43 extracts, out of the pixels constituting the first captured image, an area constituted of pixels whose multiplication results are not 0 (area including face 61 and hand 62 ) and generates a skin image including the extracted area.
  • the face 61 included in the area corresponding to the face area 101 of the binarized skin image and the hand 62 included in the area corresponding to the hand area 102 of the binarized skin image are extracted as they are.
  • the area corresponding to the area excluding the skin area in the binarized skin image (shown in white in FIG. 6 ) is given a luminance value of 225, and then a skin image as shown in FIG. 6 is generated from the first captured image.
  • the skin extraction section 43 supplies the generated skin image to the threshold value determination section 44 .
  • the threshold value determination section 44 determines a mask threshold value used for generating a mask image based on the skin image supplied from the skin extraction section 43 .
  • FIG. 7 shows an example of a histogram of the skin image.
  • the horizontal axis indicates luminance values of pixels constituting the skin image. Further, the vertical axis indicates the number of pixels corresponding to the luminance values of the horizontal axis.
  • the number of pixels constituting the area shown in white and having the luminance values of 225 in the skin image of FIG. 6 is normally shown in the histogram of FIG. 7 , but illustration thereof is omitted because the number of pixels having the luminance values of 225 is not used for determining the mask threshold value.
  • the threshold value determination section 44 creates a histogram as shown in FIG. 7 regarding the luminance values of the pixels constituting the skin image supplied from the skin extraction section 43 .
  • a large number of pixels are concentrated between a luminance value 0 and a luminance value 54 and between a luminance value 55 and a luminance value 110. That is, in the histogram of FIG. 7 , a plurality of pixels of interest are grouped into two separate groups.
  • the hand is located close to the camera 22 and the face, chest, or the like is located far from the camera 22 .
  • the LEDs 23 a and LEDs 23 b of the light-emitting apparatus 23 emit light while being close to the camera 22 , a body part of the user (in this case, hand) that is located closer to the camera 22 (light-emitting apparatus 23 ) has a larger luminance value and a body part of the user (in this case, face or the like) that is located farther from the camera 22 has a smaller luminance value.
  • the luminance values of the pixels constituting the skin portion of the hand that is located close to the camera 22 takes larger values than those of the pixels constituting the skin portion of the face that is located far from the camera 22 .
  • the luminance values between the luminance value 0 and the luminance value 54 are those of the pixels constituting the face 61 (area thereof), and the luminance values between the luminance value 55 and the luminance value 110 are those of the pixels constituting a predetermined object such as the hand 62 .
  • the threshold value determination section 44 determines a minimum luminance value (in this example, luminance value 55) as a lower limit threshold value Th_L and a maximum luminance value (in this case, luminance value 110) as an upper limit threshold value Th_H.
  • the threshold value determination section 44 supplies the determined lower limit threshold value Th_L and upper limit threshold value Th_H, as mask threshold values, to the mask image generation section 45 .
  • the mask image generation section 45 Based on the mask threshold values (lower limit threshold value Th_L and upper limit threshold value Th_H) supplied from the threshold value determination section 44 , the mask image generation section 45 detects a mask area and a non-mask area from the first captured image supplied from the camera 22 , and generates a mask image in which the detected mask area and non-mask area are binarized into different values.
  • FIG. 8 shows an example of the mask image.
  • a mask area 121 shown in black is an area having luminance values of the lower limit threshold value Th_L or more and the upper limit threshold value Th_H or less within the corresponding first captured image.
  • non-mask area shown in white in the mask image shown in FIG. 8 is an area having luminance values that are lower than the lower limit threshold value Th_L or larger than the upper limit threshold value Th_H within the corresponding first captured image.
  • the mask image generation section 45 detects the pixels having such luminance values as pixels included in the mask area and converts each of those luminance values into the value 1.
  • the mask image generation section 45 detects the pixels having such luminance values as pixels included in the non-mask area and converts each of those luminance values into the value 0.
  • the mask image generation section 45 generates the mask image that is constituted of the mask area 121 (shown in black) constituted of the pixels each having the value 1 and the non-mask area (shown in white) constituted of the pixels each having the value 0, and supplies the mask image to the shape extraction section 46 .
  • the shape extraction section 46 Based on the mask image supplied from the mask image generation section 45 , the shape extraction section 46 extracts, for example, a shape area representing the shape of the hand of the user as an area corresponding to the mask area 121 within the mask image, from the face area 101 and the hand area 102 within the binarized skin image supplied from the binarization section 42 .
  • FIG. 9 shows a display example of the extracted image including the shape area that is extracted by the shape extraction section 46 .
  • a shape area 141 is a shape of a hand of the user.
  • the shape extraction section 46 multiplies the luminance values of the pixels constituting the mask image supplied from the mask image generation section 45 and those of corresponding pixels constituting the binarized skin image supplied from the binarization section 42 .
  • the shape extraction section 46 extracts, as the shape area 141 , an area within the binarized skin image in which multiplication results are not 0, that is, out of the face area 101 and the hand area 102 within the binarized skin image ( FIG. 5 ), a portion overlapping the mask area 121 within the mask image ( FIG. 8 ).
  • the shape extraction section 46 recognizes the shape of the hand of the user based on the extracted shape area 141 , and performs processing corresponding to the recognition result.
  • the mask area 121 within the mask image shown in FIG. 8 includes the shirt that the user wears, in addition to the hand of the user.
  • the shape extraction section 46 can precisely extract the shape area 141 that represents only the shape of the hand without extracting the area representing the shape of the shirt.
  • FIG. 10 is a flowchart for explaining the shape extraction processing. It should be noted that the shape extraction processing is repeatedly performed from a time when a power of the information processing system 1 is turned on.
  • Step S 1 the controller 41 controls the LEDs 23 a of the light-emitting apparatus 23 to start emitting the light having the first wavelength. It should be noted that in a case where the LEDs 23 b are emitting light, the controller 41 stops the emission of the light of the LEDs 23 b and then causes the LEDs 23 a to start emitting light.
  • Step S 2 the camera 22 captures an image of the user irradiated with the light having the first wavelength, and supplies the resultant first captured image to the information processing apparatus 21 .
  • Step S 3 the controller 41 controls the LEDs 23 a of the light-emitting apparatus 23 to stop emitting the light having the first wavelength, and controls the LEDs 23 b of the light-emitting apparatus 23 to start emitting the light having the second wavelength.
  • Step S 4 the camera 22 captures an image of the user irradiated with the light having the second wavelength, and supplies the resultant second captured image to the information processing apparatus 21 .
  • Step S 5 the binarization section 42 generates a binarized skin image shown in FIG. 5 based on the differences between luminance values of corresponding pixels of the first captured image and the second captured image that are supplied from the camera 22 , and supplies the binarized skin image to the skin extraction section 43 and the shape extraction section 46 .
  • Step S 6 the skin extraction section 43 extracts an area corresponding to the skin area (area representing skin portion of user) within the binarized skin image from the first captured image supplied from the camera 22 , based on the binarized skin image supplied from the binarization section 42 .
  • the skin extraction section 43 generates a skin image including the extracted area and supplies the skin image to the threshold value determination section 44 .
  • Step S 7 the threshold value determination section 44 creates a histogram of the skin image as shown in FIG. 7 based on the luminance values of the pixels constituting the skin image supplied from the skin extraction section 43 .
  • Step S 8 the threshold value determination section 44 determines a luminance value with a minimal number of pixels as a lower limit threshold value Th_L and a maximum luminance value as an upper limit threshold value Th_H, based on the created histogram of the skin image.
  • the threshold value determination section 44 supplies the determined lower limit threshold value Th_L and upper limit threshold value Th_H, as mask threshold values, to the mask image generation section 45 .
  • Step S 9 the mask image generation section 45 binarizes the first captured image supplied from the camera 22 based on the mask threshold values (lower limit threshold value Th_L and upper limit threshold value Th_H) supplied from the threshold value determination section 44 to generate a mask image as shown in FIG. 8 , and supplies the mask image to the shape extraction section 46 .
  • the mask threshold values lower limit threshold value Th_L and upper limit threshold value Th_H
  • Step S 10 based on the mask image supplied from the mask image generation section 45 , the shape extraction section 46 extracts, for example, an extraction area representing a shape of a hand of the user as an area corresponding to the mask area within the mask image, from the binarized skin image supplied from the binarization section 42 .
  • the shape extraction section 46 recognizes the shape of the hand by the extracted area thus extracted, performs processing corresponding to the recognition result, and outputs the processing result to a subsequent stage.
  • the mask image is generated from the first captured image captured by one camera 22 based on the mask threshold values, and the shape of the hand of the user is extracted from the binarized skin image based on the generated mask image.
  • the mask image that includes the mask area 121 including only a skin portion of the hand as a skin portion without including the skin portion of the face and the non-mask area is generated.
  • the mask area 121 includes, as a skin portion, only the skin portion of the hand without including that of the face, with the result that only the hand area 102 can be extracted from the binarized skin image.
  • the user since the user cannot visually recognize the light emitted from the LEDs 23 a and LEDs 23 b, the user does not feel uncomfortable due to bright light emitted from the LEDs 23 a and LEDs 23 b.
  • the diffuser plate 23 c is provided in front of the LEDs 23 a and LEDs 23 b in the light-emitting apparatus 23 of the information processing system 1 .
  • the invisible light emitted from the LEDs 23 a and LEDs 23 b is uniformly diffused. Therefore, uniform light without unevenness caused by an amount of light is irradiated to a subject.
  • reflected light of the invisible light irradiated to the subject is received by the camera 22 as uniform light without unevenness caused by an amount of light, with the result that the first and second captured images without unevenness caused by the amount of light can be obtained by the camera 22 .
  • first and second captured image without unevenness caused by the amount of light are used for extracting the shape of the hand or the like in the information processing system 1 , it becomes possible to extract the shape of the hand or the like more precisely than a case where first and second captured images with unevenness caused by the amount of light are used, for example.
  • the information processing system 1 it is desirable to extract the shape of the hand in about 80 ms from a start of the shape extraction processing so that the shape of the hand after being changed can be recognized each time the user changes the shape of the hand.
  • the skin image is extracted and the mask threshold values (lower limit threshold value Th_L and upper limit threshold value Th_H) are determined based on the histogram of the extracted skin image through the processing of Steps S 6 to S 8 every time the shape extraction processing is performed, but the shape extraction processing is not limited thereto.
  • the mask threshold values previously determined in Steps S 6 to S 8 may be used as they are when the shape extraction processing is performed.
  • Steps S 6 to S 8 since the processing in Steps S 6 to S 8 can be omitted, it is possible to rapidly extract the shape of the hand or the like by the shape extraction processing.
  • Steps S 6 to S 8 by performing the same processing as the processing in Steps S 6 to S 8 before performing the shape extraction processing to determine mask threshold values in advance, it is also possible to omit the processing in Steps S 6 to S 8 in the shape extraction processing.
  • FFT Fast Fourier Tansform
  • FIG. 11 shows an example of a first captured image obtained by capturing an image of the user irradiated with light having a wavelength of 870 nm.
  • the threshold value determination section 44 is supplied, from the camera 22 , with a plurality of first captured images obtained by capturing images of a user waving the hand by the camera 22 .
  • the threshold value determination section 44 performs the FFT processing on the plurality of first captured images and detects a hand area within the first captured image, the hand area moving at a constant frequency.
  • the threshold value determination section 44 calculates an average value ave_L of luminance values of pixels constituting a rectangular area 161 that is a part of the detected hand area.
  • the threshold value determination section 44 determines a value ave_L ⁇ a obtained by subtracting an adjustment value a from the average value ave_L as a lower limit threshold value Th_L and a value ave_L+b obtained by adding an adjustment value b to the average value ave_L as an upper limit threshold value Th_H.
  • adjustment values a and b are values used for adjusting the average value ave_L and determining the lower limit threshold value Th_L and the upper limit threshold value Th_H.
  • the adjustment values a and b are variables calculated in accordance with intensities of light (amounts of light) emitted from the LEDs 23 a and LEDs 23 b, a distance from the camera 22 to the user, and light sensitivity of a CCD (Charge Coupled Device Image Sensor) used in the camera 22 , but the variables are experimentally calculated in actuality in many cases.
  • the FFT threshold value determination processing in which the threshold value determination section 44 determines mask threshold values based on the average value of the luminance values of the pixels constituting the hand area of the user will be described.
  • FIG. 12 is a flowchart for explaining the FFT threshold value determination processing.
  • the FFT threshold value determination processing is started, for example, when a power of the information processing system is turned on and before the shape extraction processing is performed.
  • Step S 31 the controller 41 controls the LEDs 23 a of the light-emitting apparatus 23 to start emitting the light having the first wavelength.
  • Step S 32 the controller 41 controls a display, a speaker, or the like (not shown) provided in the information processing apparatus 21 to instruct a user to wave the hand.
  • Step S 33 the camera 22 captures images of the user waving the hand and supplies the resultant first captured images to the threshold value determination section 44 of the information processing apparatus 21 .
  • Step S 34 the threshold value determination section 44 performs the FFT processing on the first captured images and detects a hand area within the first captured image, the hand area moving at a constant frequency.
  • Step S 35 the threshold value determination section 44 calculates an average value ave_L of the luminance values of the pixels constituting the rectangular area 161 that is a part of the detected hand area.
  • Step S 36 the threshold value determination section 44 determines a value ave_L ⁇ a obtained by subtracting an adjustment value a from the average value ave_L as a lower limit threshold value Th_L and a value ave_L+b obtained by adding an adjustment value b to the average value ave_L as an upper limit threshold value Th_H.
  • the FFT threshold value determination processing is terminated.
  • the mask threshold values are determined before the shape extraction processing is performed in the FFT threshold value determination processing, with the result that it is also possible to omit the processing in Steps S 6 to S 8 and extract the shape of the hand or the like more rapidly in the shape extraction processing.
  • the FFT processing is performed on the plurality of first captured images to detect the hand area within the first captured image and the mask threshold values (lower limit threshold value Th_L and upper limit threshold value Th_H) based on the average value of the luminance values of the pixels within the hand area, but the FFT threshold value determination processing is not limited thereto.
  • the FFT threshold value determination processing by performing the FFT processing on a plurality of second captured images obtained by capturing images of the user waving the hand by the camera 22 , it may be possible to detect a hand area within the second captured image and determine mask threshold values based on an average value of luminance values of pixels within the hand area.
  • the binarization section 42 extracts the skin area of the user and the area excluding the skin area of the user from the first captured image and supplies a binarized skin image constituted of the extracted skin area and area excluding the skin area to the skin extraction section 43 and the shape extraction section 46 , but the present invention is not limited thereto.
  • the binarization section 42 may extract a skin area of the user from the first captured image and supply a binarized skin image including at least the extracted skin area to the skin extraction section 43 and the shape extraction section 46 .
  • the skin extraction section 43 extracts from the first captured image captured by the camera 22 an area corresponding to the skin area included in the binarized skin image supplied from the binarization section 42 . Further, the shape extraction section 46 extracts a shape area from the skin area included in the binarized skin image supplied from the binarization section 42 .
  • the mask image generation section 45 detects a mask area and a non-mask area from the first captured image, for example, and generates a mask image constituted of the detected mask area and non-mask area, but the present invention is not limited thereto.
  • the mask image generation section 45 may detect only the mask area as an extraction area for extracting a shape area from the binarized skin image and generate a mask image including at least the detected mask area. In this case, out of the skin area within the binarized skin image supplied from the binarization section 42 , an area corresponding to the mask area within the mask image is extracted as a shape area in the shape extraction section 46 .
  • the mask image generation section 45 may detect only the non-mask area as an extraction area and generate a mask image including at least the detected non-mask area. In this case, out of the skin area within the binarized skin image supplied from the binarization section 42 , an area corresponding to the area excluding the non-mask area within the mask image is extracted as a shape area in the shape extraction section 46 .
  • the applicant of the present invention used a video camera manufactured by Sony Corporation as the camera 22 .
  • the camera 22 has a model number XC-EI50 and includes a 1/2 IT-type CCD as an image pickup device. Further, the camera 22 has effective pixels of 768 ⁇ 494, and adopts a C mount as a lens mount and a scanning method of interlacing 525 lines as a scanning method.
  • the sensitivity is F11 (400 1 ⁇ ) and a lowest depth of field is 0.1 1 ⁇ . Further, an S/N (signal to noise) ratio of a captured image captured by the camera 22 is 60 dB.
  • a shutter speed by a shutter button (normal shutter) provided to the camera 22 in advance is 1/100 to 1/10,000 sec
  • a shutter speed by a release switch (external trigger shutter) externally connected to the camera 22 is 1 ⁇ 4 to 1/10,000 sec.
  • the camera 22 has an outer dimension of 29 (width) ⁇ 29 (height) ⁇ 32 (depth) mm and a weight of about 50 g. Furthermore, the camera 22 has a vibration resistance of 70 G.
  • the camera 22 has a sensitivity within a range from a visible region of 400 nm to a near-infrared region of 1,000 nm.
  • FIG. 13 shows an example of relative sensitivity characteristics of the camera 22 .
  • the horizontal axis indicates a wavelength that is incident to a lens of the camera 22 and the vertical axis indicates a relative sensitivity corresponding to the wavelength.
  • the applicant of the present invention used, as the light-emitting apparatus 23 , eight LEDs 23 a and eight LEDs 23 b that were alternately arranged in a grid as shown in FIG. 14 .
  • LEDs 23 a actually used by the applicant of the present invention LEDs that emit light having a wavelength of 870 nm were used, and as the LEDs 23 b, LEDs that emit light having a wavelength of 950 nm were used.
  • LEDs having a DC forward current (absolute maximum rating) of 100 mA and a forward voltage of 1.6 V were used as the LEDs 23 a and LEDs 23 b.
  • the applicant of the present invention actually carried out the shape extraction processing and the FFT threshold value determination processing while using the camera 22 having the performance described above and the LEDs 23 a and LEDs 23 b arranged as shown in FIG. 14 , and accordingly could find the evident operational effect described above.
  • the mask image generation section 45 generates a mask image from the first captured image supplied from the camera 22 based on the mask threshold value supplied from the threshold value determination section 44 , but the method of generating a mask image is not limited to the above.
  • the mask image generation section 45 can perform stereo processing of generating a distance image expressing a distance from a camera to a user based on captured images captured by a plurality of cameras that capture images in different directions and adopt the resultant distance image as a mask image.
  • the shape extraction section 46 extracts a portion in which an area expressing a distance from the camera to the hand within the distance image supplied from the mask image generation section 45 overlaps the face area 101 and hand area 102 within the binarized skin image supplied from the binarization section 42 , as a shape area 141 representing the shape of the hand of the user.
  • the method of generating the distance image as a mask image in addition to the stereo processing it is possible to generate a distance image of the user while using a laser range finder or the like that calculates a distance to the user based on a time during which infrared rays are irradiated to the user and returned by being reflected on the user.
  • the first wavelength emitted from the LEDs 23 a is set to 870 nm and the second wavelength emitted from the LEDs 23 b is set to 950 nm, but the combination of the wavelengths is not limited thereto.
  • any combination of wavelengths may be set as long as the combination leads to a sufficiently larger difference absolute value between a reflectance in the first wavelength and a reflectance in the second wavelength than an difference absolute value between reflectances obtained for an object other than the skin of the user.
  • a combination of 800 nm and 950 nm, that of 870 nm and 1,000 nm, and that of 800 nm and 1,000 nm may be possible in addition to the combination of 870 nm and 950 nm.
  • the LEDs 23 a and LEDs 23 b emit light individually in the shape extraction processing. However, it is possible to acquire a first captured image and a second captured image by causing the LEDs 23 a and LEDs 23 b to emit light simultaneously.
  • two cameras having the same function as the camera 22 are provided close to each other in place of the camera 22 .
  • a filter to pass only the light having the first wavelength is provided in front of one camera out of the two cameras, and a filter to pass only the light having the second wavelength is provided in front of the other camera.
  • the number of LEDs 23 a and the number of LEDs 23 b are each set to two, but the number of them is not limited to the above.
  • the hand shape thereof as an object representing a body part of the user is changed to cause the information processing apparatus 21 to execute the predetermined processing, but it is possible to adopt a foot of the user or the like as an object, in addition to the hand.
  • a series of processing described above can be executed by dedicated hardware or software.
  • programs constituting the software are installed from a recording medium in a so-called built-in computer or a general-purpose personal computer that can execute various functions by installing various programs.
  • FIG. 15 shows a structure example of a personal computer that executes the series of processing described above by programs.
  • the sections, or each of the sections, of the information processing apparatus 21 illustrated in FIG. 2 may be implemented by at least one processor, such as the Central Processing Unit 201 illustrated in FIG. 15 .
  • the binarization section 42 , the skin extraction section 43 , the threshold value determination section 44 , the mask image generation section 45 , and the shape extraction section 46 may be implemented by a single processor or a plurality of different processors.
  • a CPU (Central Processing Unit) 201 executes various types of processing in accordance with programs stored in a ROM (Read Only Memory) 202 or a storage section 208 .
  • a RAM (Random Access Memory) 203 stores programs to be executed by the CPU 201 , data, and the like as appropriate. Those CPU 201 , ROM 202 , and RAM 203 are connected to each other via a bus 204 .
  • the CPU 201 is connected with an input/output interface 205 via the bus 204 .
  • the input/output interface 205 is connected with an input section 206 such as a keyboard, a mouse, and a microphone and an output section 207 such as a display and a speaker.
  • the CPU 201 executes various types of processing in accordance with commands that are input from the input section 206 . Then, the CPU 201 outputs results of the processing to the output section 207 .
  • the storage section 208 connected to the input/output interface 205 is constituted of, for example, a hard disk and stores programs to be executed by the CPU 201 and various types of data.
  • the communication section 209 communicates with an external apparatus via a network such as the Internet and a local area network.
  • a program may be acquired via the communication section 209 and stored in the storage section 208 .
  • a drive 210 connected to the input/output interface 205 drives a removable medium 211 such as a magnetic disc, an optical disc, a magneto-optical disc, and a semiconductor memory when the removable medium 211 is mounted thereto, and acquires programs and data stored in the removable medium 211 .
  • the acquired programs and data are transferred to the storage section 208 as necessary and stored therein.
  • a recording medium that records (stores) programs installed in and executed by the computer is constituted of, as shown in FIG. 15 , the removable medium 211 that is a package medium such as a magnetic disc (including a flexible disc), an optical disc (including a CD-ROM (Compact Disc-Read Only Memory) and a DVD (Digital Versatile Disc)), a magneto-optical disc (including an MD (Mini-Disc)), and a semiconductor memory, the ROM 202 in which programs are temporarily or permanently stored, or a hard disk constituting the storage section 208 .
  • the programs are recorded on the recording medium via the communication section 209 as an interface such as a router and a modem as appropriate while using a wireless or wired communication medium such as a local area network, the Internet, and a digital broadcast.
  • steps describing the above series of processing include, in addition to processing that are performed in time series in the described order, processing that are executed in parallel or individually though not processed chronologically.
  • system herein represents the overall apparatuses constituted of a plurality of apparatuses.
US13/058,948 2009-06-30 2010-05-21 Information processing apparatus and information processing method Abandoned US20110142349A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2009-154921 2009-06-30
JP2009154921A JP4548542B1 (ja) 2009-06-30 2009-06-30 情報処理装置、情報処理方法及びプログラム
PCT/JP2010/003427 WO2011001593A1 (en) 2009-06-30 2010-05-21 Skin detection using multi-band near-infrared illumination

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/003427 A-371-Of-International WO2011001593A1 (en) 2009-06-30 2010-05-21 Skin detection using multi-band near-infrared illumination

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/089,047 Division US8285054B2 (en) 2009-06-30 2011-04-18 Information processing apparatus and information processing method

Publications (1)

Publication Number Publication Date
US20110142349A1 true US20110142349A1 (en) 2011-06-16

Family

ID=42457840

Family Applications (3)

Application Number Title Priority Date Filing Date
US13/058,948 Abandoned US20110142349A1 (en) 2009-06-30 2010-05-21 Information processing apparatus and information processing method
US13/059,422 Expired - Fee Related US8107706B2 (en) 2009-06-30 2010-05-24 Information processing apparatus, information processing method, program, and electronic apparatus
US13/089,047 Active US8285054B2 (en) 2009-06-30 2011-04-18 Information processing apparatus and information processing method

Family Applications After (2)

Application Number Title Priority Date Filing Date
US13/059,422 Expired - Fee Related US8107706B2 (en) 2009-06-30 2010-05-24 Information processing apparatus, information processing method, program, and electronic apparatus
US13/089,047 Active US8285054B2 (en) 2009-06-30 2011-04-18 Information processing apparatus and information processing method

Country Status (7)

Country Link
US (3) US20110142349A1 (zh)
EP (2) EP2384485A1 (zh)
JP (2) JP4548542B1 (zh)
KR (2) KR20120031309A (zh)
CN (1) CN102138148B (zh)
TW (1) TW201112168A (zh)
WO (2) WO2011001593A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110293140A1 (en) * 2010-05-28 2011-12-01 Qualcomm Incorporated Dataset Creation For Tracking Targets With Dynamically Changing Portions
US20140254870A1 (en) * 2013-03-11 2014-09-11 Lenovo (Singapore) Pte. Ltd. Method for recognizing motion gesture commands
US20160180148A1 (en) * 2009-12-18 2016-06-23 Tandent Vision Science, Inc. Method and system for generating intrinsic images using a single reflectance technique
US20190043444A1 (en) * 2017-08-07 2019-02-07 Seiko Epson Corporation Display device and control method for display device
US20190340762A1 (en) * 2016-12-30 2019-11-07 Skinio, Llc Skin Abnormality Monitoring Systems and Methods
US11100348B2 (en) * 2017-07-17 2021-08-24 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method of living body detection and terminal device

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4910923B2 (ja) * 2007-07-20 2012-04-04 ソニー株式会社 撮像装置、撮像方法及び撮像プログラム
JP2011039812A (ja) * 2009-08-12 2011-02-24 Sony Corp 画像処理装置及び電子機器
JP2012063824A (ja) * 2010-09-14 2012-03-29 Sony Corp 情報処理装置、情報処理方法、及びプログラム
EP2512121B1 (en) * 2011-04-13 2013-06-05 Axis AB Illumination device
JP2013084228A (ja) * 2011-10-12 2013-05-09 Sony Corp 情報処理装置、情報処理方法、プログラム、及び電子機器
JP2013164834A (ja) 2012-01-13 2013-08-22 Sony Corp 画像処理装置および方法、並びにプログラム
TWI479430B (zh) * 2012-10-08 2015-04-01 Pixart Imaging Inc 以自然影像進行的手勢辨識方法
CN103777741B (zh) * 2012-10-19 2017-08-01 原相科技股份有限公司 基于物件追踪的手势辨识方法及系统
CN108334204B (zh) * 2012-12-10 2021-07-30 因维萨热技术公司 成像装置
JP5971108B2 (ja) 2012-12-18 2016-08-17 富士通株式会社 画像処理装置、画像処理方法および画像処理プログラム
CN103268499B (zh) * 2013-01-23 2016-06-29 北京交通大学 基于多光谱成像的人体皮肤检测方法
US20140240477A1 (en) * 2013-02-26 2014-08-28 Qualcomm Incorporated Multi-spectral imaging system for shadow detection and attenuation
CN110045824B (zh) * 2014-02-10 2022-06-17 苹果公司 使用光学传感器检测到的运动手势输入
WO2015188146A2 (en) 2014-06-05 2015-12-10 Edward Hartley Sargent Sensors and systems for the capture of scenes and events in space and time
US9692968B2 (en) 2014-07-31 2017-06-27 Invisage Technologies, Inc. Multi-mode power-efficient light and gesture sensing in image sensors
US10736517B2 (en) * 2014-10-09 2020-08-11 Panasonic Intellectual Property Management Co., Ltd. Non-contact blood-pressure measuring device and non-contact blood-pressure measuring method
EP3276448B1 (en) * 2015-05-20 2020-01-15 Konica Minolta, Inc. Wearable electronic device, gesture detection method for wearable electronic device, and gesture detection program for wearable electronic device
JP6501806B2 (ja) 2017-01-05 2019-04-17 キヤノン株式会社 情報処理装置、操作検出方法、及びコンピュータプログラム
CN108416333B (zh) * 2018-03-30 2020-01-17 百度在线网络技术（北京）有限公司 图像处理的方法和装置
CN109145803B (zh) * 2018-08-14 2022-07-22 京东方科技集团股份有限公司 手势识别方法及装置、电子设备、计算机可读存储介质
CN109124587A (zh) * 2018-08-14 2019-01-04 上海常仁信息科技有限公司 一种基于机器人的皮肤检测系统和方法
CN113614487A (zh) * 2019-03-26 2021-11-05 索尼集团公司 图像处理装置、图像处理方法和图像处理程序

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050084141A1 (en) * 2003-08-29 2005-04-21 Fuji Xerox Co., Ltd. Action recognition apparatus and apparatus for recognizing attitude of object
US20050226505A1 (en) * 2004-03-31 2005-10-13 Wilson Andrew D Determining connectedness and offset of 3D objects relative to an interactive surface
US20060104488A1 (en) * 2004-11-12 2006-05-18 Bazakos Michael E Infrared face detection and recognition system
US20090021596A1 (en) * 2007-07-18 2009-01-22 Sony Corporation Imaging apparatus, image processing apparatus, and program
US20100245609A1 (en) * 2009-03-26 2010-09-30 Texas Instruments Incorporated Digital Image Segmentation Using Flash

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5031049A (en) * 1984-05-25 1991-07-09 Canon Kabushiki Kaisha Automatic object image follow-up device
US5418574A (en) * 1992-10-12 1995-05-23 Matsushita Electric Industrial Co., Ltd. Video signal correction apparatus which detects leading and trailing edges to define boundaries between colors and corrects for bleeding
JPH06304142A (ja) * 1993-04-22 1994-11-01 Canon Inc 視線検出装置
EP0957750A1 (en) * 1995-10-23 1999-11-24 Cytometrics, Inc. Method and apparatus for reflected imaging analysis
US5771033A (en) * 1996-05-24 1998-06-23 Microsoft Corporation Method and system for dissolving an image displayed on a computer screen
JPH10150572A (ja) * 1996-09-18 1998-06-02 Fuji Xerox Co Ltd 画像処理装置、画像処理方法、画像処理プログラムを記録した媒体
US6292576B1 (en) * 2000-02-29 2001-09-18 Digital Persona, Inc. Method and apparatus for distinguishing a human finger from a reproduction of a fingerprint
JP2003216955A (ja) * 2002-01-23 2003-07-31 Sharp Corp ジェスチャ認識方法、ジェスチャ認識装置、対話装置及びジェスチャ認識プログラムを記録した記録媒体
JP4074464B2 (ja) * 2002-02-26 2008-04-09 学校法人 中央大学 周期性を有する動きの検出方法および装置
KR101035667B1 (ko) * 2002-05-09 2011-05-19 소니 주식회사 생체 인식 패턴 검출 장치, 개인 인증 장치 및 방법
ATE476908T1 (de) * 2003-04-18 2010-08-15 Medispectra Inc System und diagnoseverfahren zur optischen detektion von verdächtigen stellen einer gewebeprobe
JP4102318B2 (ja) * 2004-03-03 2008-06-18 日本電信電話株式会社 道具動作認識装置および道具動作認識方法
US7427981B2 (en) * 2004-04-15 2008-09-23 Avago Technologies General Ip (Singapore) Pte. Ltd. Optical device that measures distance between the device and a surface
JP4537143B2 (ja) * 2004-07-30 2010-09-01 キヤノン株式会社 画像処理装置及びその方法、撮像装置、プログラム
JP4442472B2 (ja) * 2005-03-07 2010-03-31 株式会社豊田中央研究所 対象物の部位判別装置
US20070140553A1 (en) * 2005-12-19 2007-06-21 Olympus Corporation Dental colorimetry apparatus
US9696808B2 (en) * 2006-07-13 2017-07-04 Northrop Grumman Systems Corporation Hand-gesture recognition method
JP2008182360A (ja) * 2007-01-23 2008-08-07 Funai Electric Co Ltd 皮膚領域検出撮像装置
US20080304736A1 (en) * 2007-02-20 2008-12-11 Masahiro Nakagawa Method of estimating a visual evaluation value of skin beauty
JP5156365B2 (ja) 2007-12-27 2013-03-06 ライオン株式会社 プラスチックボトル

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050084141A1 (en) * 2003-08-29 2005-04-21 Fuji Xerox Co., Ltd. Action recognition apparatus and apparatus for recognizing attitude of object
US20050226505A1 (en) * 2004-03-31 2005-10-13 Wilson Andrew D Determining connectedness and offset of 3D objects relative to an interactive surface
US20060104488A1 (en) * 2004-11-12 2006-05-18 Bazakos Michael E Infrared face detection and recognition system
US20090021596A1 (en) * 2007-07-18 2009-01-22 Sony Corporation Imaging apparatus, image processing apparatus, and program
US20100245609A1 (en) * 2009-03-26 2010-09-30 Texas Instruments Incorporated Digital Image Segmentation Using Flash

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160180148A1 (en) * 2009-12-18 2016-06-23 Tandent Vision Science, Inc. Method and system for generating intrinsic images using a single reflectance technique
US9754155B2 (en) * 2009-12-18 2017-09-05 Tandent Vision Science, Inc. Method and system for generating intrinsic images using a single reflectance technique
US20110293140A1 (en) * 2010-05-28 2011-12-01 Qualcomm Incorporated Dataset Creation For Tracking Targets With Dynamically Changing Portions
US9256956B2 (en) * 2010-05-28 2016-02-09 Qualcomm Incorporated Dataset creation for tracking targets with dynamically changing portions
US20160098600A1 (en) * 2010-05-28 2016-04-07 Qualcomm Incorporated Dataset creation for tracking targets with dynamically changing portions
US9785836B2 (en) * 2010-05-28 2017-10-10 Qualcomm Incorporated Dataset creation for tracking targets with dynamically changing portions
US20140254870A1 (en) * 2013-03-11 2014-09-11 Lenovo (Singapore) Pte. Ltd. Method for recognizing motion gesture commands
US20190340762A1 (en) * 2016-12-30 2019-11-07 Skinio, Llc Skin Abnormality Monitoring Systems and Methods
US11100348B2 (en) * 2017-07-17 2021-08-24 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method of living body detection and terminal device
US20190043444A1 (en) * 2017-08-07 2019-02-07 Seiko Epson Corporation Display device and control method for display device
US10657925B2 (en) * 2017-08-07 2020-05-19 Seiko Epson Corporation Display device and control method for conversion of luminance information

Also Published As

Publication number Publication date
WO2011001593A1 (en) 2011-01-06
EP2384485A1 (en) 2011-11-09
US8107706B2 (en) 2012-01-31
KR20120031309A (ko) 2012-04-02
US20110194774A1 (en) 2011-08-11
EP2378759A1 (en) 2011-10-19
CN102138148B (zh) 2013-07-24
JP4548542B1 (ja) 2010-09-22
US20110216941A1 (en) 2011-09-08
WO2011001761A1 (ja) 2011-01-06
EP2378759A4 (en) 2012-11-07
JP2014064047A (ja) 2014-04-10
JPWO2011001761A1 (ja) 2012-12-13
KR20120039498A (ko) 2012-04-25
CN102138148A (zh) 2011-07-27
US8285054B2 (en) 2012-10-09
TW201112168A (en) 2011-04-01
JP4831267B2 (ja) 2011-12-07

Similar Documents

Publication Publication Date Title
US8285054B2 (en) Information processing apparatus and information processing method
US20210334526A1 (en) Living body detection device, living body detection method, and recording medium
JP5800175B2 (ja) 画像処理装置、画像処理方法、プログラム、及び電子機器
US20110298909A1 (en) Image processing apparatus, image processing method, program and electronic apparatus
US20120224042A1 (en) Information processing apparatus, information processing method, program, and electronic apparatus
WO2019163065A1 (ja) なりすまし検知装置、なりすまし検知方法、及びコンピュータ読み取り可能な記録媒体
WO2019163066A1 (ja) なりすまし検知装置、なりすまし検知方法、及びコンピュータ読み取り可能な記録媒体
US9117114B2 (en) Image processing device, image processing method, program, and electronic device for detecting a skin region of a subject
JP5287792B2 (ja) 情報処理装置、情報処理方法及びプログラム
US11645864B2 (en) Imaging device, authentication device, and biometric imaging method
KR101146017B1 (ko) 정보 처리 장치 및 정보 처리 방법
WO2019006707A1 (zh) 虹膜采集方法、电子装置和计算机可读存储介质
JP7207506B2 (ja) なりすまし検知装置、なりすまし検知方法、及びプログラム
JP2011158447A (ja) 画像処理装置、画像処理方法、プログラム、及び電子機器
JP2012000147A (ja) 画像処理装置、画像処理方法、プログラム、及び電子装置

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SAIJO, NOBUHIRO;REEL/FRAME:025824/0499

Effective date: 20101028

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION