WO2016017096A1 - 情報処理装置、情報処理方法及びプログラム - Google Patents
情報処理装置、情報処理方法及びプログラム Download PDFInfo
- Publication number
- WO2016017096A1 WO2016017096A1 PCT/JP2015/003560 JP2015003560W WO2016017096A1 WO 2016017096 A1 WO2016017096 A1 WO 2016017096A1 JP 2015003560 W JP2015003560 W JP 2015003560W WO 2016017096 A1 WO2016017096 A1 WO 2016017096A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- contact
- target object
- information processing
- image
- processing apparatus
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
- G06F3/0426—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking fingers with respect to a virtual keyboard projected or printed on the surface
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Definitions
- the present technology relates to an information processing apparatus capable of detecting contact, an information processing method, and a program.
- a stereo camera, a 3D sensor, or the like is used for contact of the operating body with the target object.
- the UI recognizes the target object and the operating body, and detects the position of the operating body and the planar shape of the target object based on the three-dimensional information acquired from a stereo camera, a 3D sensor, or the like.
- the UI can determine that a contact has occurred when the distance between the target object and the operating tool is a certain value or less.
- the above-described method requires detection means such as a stereo camera and a 3D sensor for contact detection, and there are problems such as an increase in device size, an increase in power consumption, and an increase in cost. Further, due to the characteristics of stereo cameras and 3D sensors, it is difficult to use under sunlight. Furthermore, even when the operating body approaches the target object and is not in contact with the target object, it is determined that the operation body is in contact, and an input unintended by the user may be made.
- an object of the present technology is to provide an information processing apparatus, an information processing method, and a program capable of performing contact detection with high accuracy from an image captured by a normal camera.
- an information processing apparatus includes a contact determination unit.
- the contact determination unit determines contact of the operating body with respect to the target object based on a color change in an image including the target object and the operating body.
- color is synonymous with pixel value and includes brightness.
- the information processing apparatus An object recognition processing unit for recognizing an operation tool included in the image,
- the contact determination unit may determine contact of the operating body with the target object based on a change in color of the operating body.
- the contact determination unit can detect a contact by using a change in the color of the recognized operation body. It is.
- the object recognition processing unit may recognize the operation body using an object detection dictionary.
- the object recognition processing unit can recognize the operating body by comparing the image with an object detection dictionary in which information on an object that can be the operating body is stored.
- the object recognition processing unit may recognize an object that has entered the image as an operating body.
- the object recognition processing unit can recognize an object as an operating object when a new object enters the image (imaging range of the camera).
- the contact determination unit may determine contact of the operating body with the target object based on a predetermined color change.
- the contact determination unit can detect the contact using the change in the color. Is possible.
- the predetermined color may be a color corresponding to the blood color of the fingertip.
- the blood color of the fingertip changes when the finger contacts any object. For this reason, the contact determination unit can determine contact with the target object when the operation body is a finger by using a color change corresponding to the blood color of the fingertip.
- the contact determination unit may determine contact of the operating body with respect to the target object based on a change in color of the operating body.
- the contact determination unit determines contact based on a change in the color of the operating body Is possible.
- the contact determination unit may determine contact of the operating body with the target object based on a change in the color of the target object.
- the contact determination unit may determine contact using the change in the color of the target object.
- the contact determination unit may determine contact of the operating body with the target object based on a change in color of the target object due to a shadow of the operating body.
- the contact determination unit may determine contact by using a change in the color of the target object due to the shadow of the operating body. For example, when the operating body approaches the target object, the shadow becomes darker, and it is possible to determine the contact using the shadow.
- the contact determination unit can determine the contact strength (pressing force) using the color change.
- the contact determination unit may further determine a contact direction of the operating body with respect to the target object based on a color change in the image.
- the contact determination unit can determine the contact direction (pressing direction) using the color change.
- the information processing apparatus may further include an input control unit that controls an operation input based on a determination result of the contact determination unit.
- the input control unit can accept an operation input according to the contact position and the number of contacts and supply it to the OS or the like when the contact determination unit determines that the operation object and the target object are in contact with each other.
- the information processing apparatus further includes an image output unit that generates an operation target image to be superimposed on the target object,
- the input control unit may control an operation input based on a contact position between the operation body and the operation target object in the operation target image.
- the image output unit generates an operation target image that is displayed on an HMD (Head Mounted Display), a projector, or the like and superimposed on the target object.
- the input control unit calculates the positional relationship between the operation target image and the contact position based on the contact position in the operation target image and the superimposed position of the operation target image on the target object, and the contents of the operation target image superimposed on the contact position An operation input corresponding to (icon etc.) can be accepted.
- An object recognition processing unit for recognizing an operation tool included in the image may control the operation input based on the shape of the operating body recognized by the object recognition processing unit and the determination result of the contact determination unit.
- the object recognition processing unit recognizes the shape of the operating body (for example, the shape of a finger during a pinch operation) and supplies it to the input control unit, whereby the input control unit receives an operation input corresponding to the shape of the operating body It can be.
- An object recognition processing unit for recognizing a target object included in the image;
- the contact determination unit determines contact of the operating body with the target object based on a color change in the image when the target object is a rigid body, and normals when the target object is not a rigid body. You may determine the contact of the said operating body with respect to the said target object based on the direction of the normal line with respect to the surface of the said target object acquired by the direction detection sensor.
- the normal direction on the object surface (direction perpendicular to the object surface) can be detected by a normal direction detection sensor, and contact can be determined using deformation of the target object due to contact.
- the contact determination unit determines contact of the operating body with respect to the target object based on a color change in the image when a distance between the target object acquired by the depth sensor and the operating body is a predetermined value or less. Also good.
- the depth sensor can measure the distance between the sensor and the object using an infrared projection pattern or the like.
- the contact determination unit determines the contact based on the color change in the image when the distance between the target object and the operation body is equal to or less than the predetermined value, the contact determination is performed only when the contact possibility is high. Is possible.
- a program provides information as a contact determination unit that determines contact of the operating body with the target object based on a color change in an image including the target object and the operating body. Operate the processor.
- the contact determination unit performs contact of the operation body with the target object based on a color change in an image including the target object and the operation body. judge.
- the present technology provides an information processing apparatus, an information processing method, and a program capable of performing contact detection with high accuracy from an image captured by a general camera.
- the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.
- FIG. 1 is a schematic diagram illustrating a functional configuration of the information processing apparatus 100 according to the first embodiment of the present technology.
- the information processing apparatus 100 includes a voice input unit 101, a voice recognition processing unit 102, an image input unit 103, an object recognition processing unit 104, a contact determination unit 105, an input control unit 106, and an image output unit 107.
- a voice input unit 101 a voice recognition processing unit 102
- an image input unit 103 an object recognition processing unit 104
- a contact determination unit 105 an input control unit 106
- an image output unit 107 an image output unit 107.
- the information processing apparatus 100 can be connected to the eyewear 10. Further, the information processing apparatus 100 may be mounted on the eyewear 10.
- the eyewear 10 includes a display 11, a camera 12, and a microphone (not shown).
- the display 11 can be a transmissive HMD (Head-Mounted Display), and the camera 12 can be a general image (moving image) imaging camera.
- HMD Head-Mounted Display
- the voice input unit 101 is connected to the microphone of the eyewear 10 and receives the voice signal of the voice collected by the microphone.
- the voice input unit 101 outputs the acquired voice signal to the voice recognition processing unit 102.
- the voice recognition processing unit 102 performs voice recognition processing on the voice signal supplied from the voice input unit 101 to recognize the operation voice by the user. If the operation voice is recognized, the voice recognition processing unit 102 supplies the recognition result to the input control unit 106.
- the image input unit 103 is connected to the camera 12, and an image (moving image) captured by the camera 12 is input.
- the image input unit 103 supplies the acquired image (hereinafter, acquired image) to the object recognition processing unit 104.
- the object recognition processing unit 104 collates the acquired image supplied from the image input unit 103 with the information stored in the object detection dictionary D to recognize the object.
- the object recognition processing unit 104 can recognize an object such as an operation body or a target object described later.
- the object recognition processing unit 104 supplies the acquired image and the object recognition result to the contact determination unit 105.
- the object detection dictionary D may be stored in the information processing apparatus 100, or the object recognition processing unit 104 may acquire it from a network or the like.
- the contact determination unit 105 performs contact determination on the acquired image supplied from the object recognition processing unit 104 using the object recognition result and the contact determination threshold T. Details of this processing will be described later.
- the contact determination unit 105 supplies the determination result to the input control unit 106.
- the contact determination threshold value T may be stored in the information processing apparatus 100, or the contact determination unit 105 may acquire it from a network or the like.
- the input control unit 106 controls the operation input by the user based on the recognition result by the voice recognition processing unit 102 or the determination result by the contact determination unit 105. Specifically, the input control unit 106 accepts operation inputs such as touch, multi-touch, dragging, and pinching from the contact position, pressing force, pressing direction, movement of the operation body after contact, and the like with respect to the target object. The input control unit 106 supplies the received operation input to the OS or the like of the information processing apparatus 100.
- the image output unit 107 generates an image (moving image) displayed on the display 11.
- the image output unit 107 can generate an operation image including icons and operation buttons.
- the display 11 is a transmissive HMD, and the user visually recognizes a field of view in which an image displayed on the display 11 and a real object are superimposed.
- FIG. 2 schematically shows the user's field of view through the display 11.
- the user's field of view includes the user's palm H, the user's finger Y, and the operation image G displayed on the display 11.
- the operation image G includes an icon P.
- FIG. 3 is a schematic diagram illustrating a state in which the operation image G is superimposed on the magazine Z. As shown in the figure, when the user touches the place where the icon P on the magazine Z is superimposed with the finger Y, the information processing apparatus 100 accepts the selection of the icon P. The user can also indicate the icon P with something other than a finger such as a stylus.
- an object finger Y or stylus that touches the target object and points to the object P (an object such as a finger Y or a stylus) is projected as an operation object G projected by the user and designated by the user (palm H or magazine Z) And That is, the information processing apparatus 100 detects contact of the operating body with respect to the target object, and accepts an operation input according to the contact.
- FIG. 4 shows a state where the camera 12 is provided independently of the eyewear 10.
- the operation image G is displayed on the display 11 and superimposed on the magazine Z.
- the camera 12 is provided at a position different from the eyewear 10 (for example, above the magazine Z) and is connected to the image input unit 103.
- the information processing apparatus 100 may be connected to an image projection apparatus different from the eyewear 10.
- the projector may be connected to a projector including a camera that can project an image on a desk or a wall surface and can capture an image of the projection surface.
- the image output unit 107 is connected to the image projection mechanism of the projector, and projects an image on a target object (desk or wall).
- the image input unit 103 is connected to a camera and acquires an image including a target object and a projection image.
- the information processing apparatus 100 can display or project an image on a target object, and can be connected to a device including a camera that can capture the target object. is there.
- the functional configuration of the information processing apparatus 100 as described above may be connected to or built in an image projection apparatus such as eyewear or a projector, or may be mounted on a PC or a smartphone other than the image projection apparatus. . A part or all of the functional configuration may be configured on a network.
- FIG. 5 is a schematic diagram illustrating a hardware configuration of the information processing apparatus 100.
- the configuration of the information processing apparatus 100 described above is a functional configuration realized by the cooperation of the hardware configuration illustrated in FIG. 5 and the program.
- the information processing apparatus 100 includes a CPU 121, a memory 122, a storage 123, and an input / output IF 124 as hardware configurations. These are connected to each other by a bus 125.
- a CPU (Central Processing Unit) 121 controls other configurations according to a program stored in the memory 122, performs data processing according to the program, and stores the processing result in the memory 122.
- the CPU 121 can be a microprocessor.
- the memory 122 stores programs executed by the CPU 121 and data.
- the memory 122 can be a RAM (Random Access Memory).
- the storage 123 stores programs and data.
- the storage 123 may be an HDD (Hard disk drive) or an SSD (solid state drive).
- the storage 123 stores the object detection dictionary D and the contact determination threshold value T described above.
- the input / output IF (interface) 124 receives an input to the information processing apparatus 100 and supplies the output of the information processing apparatus 100 to the display 11 or the like.
- the input / output IF 124 includes input devices such as a mouse and a touch panel, output devices such as the display 11, and connection interfaces such as a network.
- the information processing apparatus 100 can accept a user operation input based on the image acquired by the image input unit 103. In this case, the input is accepted without going through the input / output IF 124.
- the image generated by the image output unit 107 is output to the display 11 through the input / output IF 124 and displayed.
- the hardware configuration of the information processing apparatus 100 is not limited to that shown here, and any hardware configuration that can realize the functional configuration of the information processing apparatus 100 may be used.
- FIG. 6 is a flowchart showing the operation of the information processing apparatus 100.
- an image is captured by the camera 12, and the image input unit 103 acquires an image (acquired image).
- the image input unit 103 supplies the acquired image to the object recognition processing unit 104.
- the object recognition processing unit 104 recognizes the target object (St101).
- the object recognition processing unit 104 recognizes the target object by collating the acquired image acquired from the image input unit 103 with the object detection dictionary D. Thereby, for example, if a palm or a magazine is included in the acquired image, it is recognized as a target object.
- the image output unit 107 supplies the operation image G (see FIG. 2) to the display 11 for display (St103).
- recognition of the target object by the object recognition processing unit 104 (St101) is executed again.
- the object recognition processing unit 104 recognizes the operating body (St104).
- the object recognition processing unit 104 recognizes the operating tool by collating the acquired image with the object detection dictionary D. Thereby, for example, if a finger, a stylus, or the like is included in the acquired image, it is recognized as an operating tool.
- the object recognition processing unit 104 may recognize a plurality of operating objects.
- the contact determination unit 105 calculates a change in the color of the operating tool (St106).
- a color means a pixel value (RGB value), and includes brightness (color brightness).
- the contact determination unit 105 captures a change in the color of the operating body with the passage of the imaging time.
- the target recognition by the object recognition processing unit 104 (St101) is executed again.
- FIG. 7 shows an example when the finger Y is recognized as the operating tool.
- the contact determination unit 105 can calculate a change in the blood color of the fingertip (inside the nail) of the finger Y.
- FIG. 7A shows a state where the finger Y is not in contact with the target object
- FIG. 7B shows a state where the finger Y is in contact with the target object.
- the gray area represents the color of the fingertip.
- the contact determination unit 105 compares the color change of the operating tool with the contact determination threshold value T, and when the color change is a change in a predetermined pattern defined in the contact determination threshold value T (St107: Yes). Then, it is determined that the operating body has touched the target object (St108). For example, when the blood color of the fingertip shown in FIG. 7A changes as shown in FIG. 7B, the contact determination unit 105 can determine that the finger Y has touched the target object.
- the contact determination unit 105 can perform contact determination for each operation body when a plurality of operation bodies are recognized by the object recognition processing unit 104.
- the color change pattern (contact determination threshold value T) used for determination by the contact determination unit 105 may be determined according to some rule (for example, it is determined that contact occurs when the color of the toe becomes light as shown in FIG. 7). It may be determined by machine learning that repeats contact determination for a large number of subjects. Further, the contact determination unit 105 may accumulate the blood color tendency of the fingertip of the user who is using the information processing apparatus 100 and execute the calibration of the contact determination threshold value T.
- the contact determination unit 105 supplies a contact position with respect to the acquired image to the input control unit 106 when determining contact with the target object by the operating body.
- the input control unit 106 controls the operation input based on the position of the icon P displayed on the operation image G and the contact position supplied from the contact determination unit 105. For example, the input control unit 106 can select the icon P superimposed on the contact position.
- the information processing apparatus 100 operates as described above. As described above, the information processing apparatus 100 performs contact determination using an image captured by a normal camera. That is, no contact determination means such as a 3D sensor or a stereo camera is required, and the device can be reduced in size and cost. Further, the color change of the target object does not occur unless the operating tool is in contact with the target object. That is, the information processing apparatus 100 performs the contact determination only when the operating tool is reliably in contact with the target object. In order to execute, operation input can be received with high accuracy.
- the operating body is not limited to the finger.
- the contact may be determined using a color change accompanying the deformation of the stylus due to the contact with the target object.
- a stylus that changes color or emits light when touching something may be used as the operating body.
- the contact determination unit 105 determines the contact using the color change of the operating body.
- the present invention is not limited to this, and the contact determination unit 105 determines the contact using the color change of the target object. Also good.
- the target object is a palm
- the blood color of the palm changes when the operating body comes into contact with it
- the contact determination unit 105 may determine contact using the palm.
- Some liquid crystal displays change color when they touch the liquid crystal surface when no image is displayed.
- the contact determination unit 105 may determine contact using the color change of the liquid crystal surface.
- deformation compression
- the contact determination unit 105 may determine contact using the color change caused by the deformation.
- the contact determination unit 105 may determine contact using the shadow of the operating tool. When the operating body approaches the target object to some extent, a shadow of the operating body is generated on the target object. The contact determination unit 105 can also determine from the change in the color of the target object due to the shadow of the operation object that the operation object has contacted the target object when the shadow and the operation object come into contact.
- the object recognition processing unit 104 collates the acquired image with the object detection dictionary D and recognizes the object (target object and operation body), the present invention is not limited to this.
- the object may be recognized as an operating body.
- the contact determination unit 105 can determine contact by using a change in the color of the object.
- the camera 12 of the eyewear 10 (or other device) connected to the information processing apparatus 100 is a camera capable of polarization imaging.
- Polarization imaging is a technique that can acquire the luminance image of the imaging range and the direction of the normal line (line perpendicular to the surface of the object) included in the imaging range.
- the camera 12 may acquire the normal direction of the object by a method different from the polarization image, and the eyewear 10 may include a sensor capable of acquiring the normal direction separately from the camera 12.
- FIG. 8 is a flowchart showing another operation of the information processing apparatus 100.
- an image is captured by the camera 12, and the image input unit 103 acquires an image (acquired image).
- the image input unit 103 supplies the acquired image to the object recognition processing unit 104.
- the object recognition processing unit 104 recognizes the target object (St111).
- the object recognition processing unit 104 recognizes the target object by collating the acquired image with the object detection dictionary D from the image input unit 103. Thereby, for example, if a palm or a magazine is included in the acquired image, it is recognized as a target object.
- the image output unit 107 supplies the operation image G (see FIG. 2) to the display 11 for display (St113).
- the target object recognition (St111) by the object recognition processing unit 104 is executed again.
- the object recognition processing unit 104 recognizes the operating body (St114).
- the object recognition processing unit 104 recognizes the operating tool by collating the acquired image with the object detection dictionary. Thereby, for example, if a finger, a stylus, or the like is included in the acquired image, it is recognized as an operating tool.
- the object recognition processing unit 104 may recognize a plurality of operating objects.
- the object recognition processing unit 104 determines whether or not the target object is a rigid body (St116).
- “Rigid body” means an object that is not deformed by contact with another object, such as a desk. Palms and magazines are judged not to be rigid.
- the contact determination unit 105 calculates a change in the color of the operating body (St117).
- the contact determination unit 105 compares the color change of the operating tool with the contact determination threshold value T, and the color change is a change in a predetermined pattern defined in the contact determination threshold value T. If it is found (St118: Yes), it is determined that the operating body has come into contact with the target object (St119). When the change in the color of the operating body does not change the predetermined pattern (St118: No), the object recognition processing unit 104 recognizes the target object (St111) again.
- the contact determination unit 105 calculates the normal direction of the target object surface around the operating body from the acquired image (St120). As described above, when the camera 12 has the polarization imaging function, the normal direction of the surface of the target object can be acquired from the acquired image. The object recognition processing unit 104 may calculate the normal direction of the surface of the target object from the output of a sensor different from the camera 12.
- the contact determination unit 105 determines contact of the operating body with the target object based on the change in the normal direction of the surface of the target object (St121).
- the contact determination part 105 can determine the contact of the operating body with respect to a target object based on the change of the normal direction in the surface of a target object. That is, when the normal direction on the surface of the target object changes to the direction of the operating body (St121: Yes), the contact determination unit 105 determines that the operating body has contacted the target object (St119). If the normal direction on the surface of the target object does not change to the direction of the operating body (St121: No), recognition of the target object by the object recognition processing unit 104 (St111) is executed again.
- the contact determination unit 105 supplies a contact position with respect to the acquired image to the input control unit 106 when determining contact with the target object by the operating body.
- the input control unit 106 controls the operation input based on the position of the icon P displayed on the operation image G and the contact position supplied from the contact determination unit 105. For example, the input control unit 106 can select the icon P superimposed on the contact position.
- the information processing apparatus 100 can also perform the above operations. If the target object is a rigid body, the surface of the target object is not deformed even if the operating body comes into contact with the target object, and contact determination using the normal direction is impossible. Therefore, the information processing apparatus 100 determines contact using the color change of the acquired image when the target object is a rigid body, and determines contact using the normal direction if the target object is not a rigid body. Can be.
- the eyewear 10 (or other device) connected to the information processing apparatus 100 is provided with a depth sensor capable of acquiring depth information.
- the depth information is distance information between the object and the depth sensor.
- FIG. 9 is a flowchart showing another operation of the information processing apparatus 100.
- an image is captured by the camera 12, and the image input unit 103 acquires an image (acquired image).
- the image input unit 103 supplies the acquired image to the object recognition processing unit 104.
- the object recognition processing unit 104 recognizes the target object (St131).
- the object recognition processing unit 104 recognizes the target object by collating the acquired image acquired from the image input unit 103 with the object detection dictionary D. Thereby, for example, if a palm or a magazine is included in the acquired image, it is recognized as a target object.
- the image output unit 107 supplies the operation image G (see FIG. 2) to the display 11 for display (St133).
- the target object recognition (St131) by the object recognition processing unit 104 is executed again.
- the object recognition processing unit 104 recognizes the operating body (St134).
- the object recognition processing unit 104 recognizes the operating tool by collating the acquired image with the object detection dictionary D. Thereby, for example, if a finger, a stylus, or the like is included in the acquired image, it is recognized as an operating tool.
- the object recognition processing unit 104 may recognize a plurality of operating objects.
- the contact determination unit 105 calculates the distance between the operating tool and the target object using the depth information (St136).
- the contact determination unit 105 can acquire depth information from the depth sensor as described above.
- the object recognition processing unit 104 recognizes the target object (St131) again.
- the contact determination unit 105 calculates a change in the color of the operating tool (St138).
- the target recognition by the object recognition processing unit 104 (St131) is executed again.
- the contact determination unit 105 compares the color change of the operating tool with the contact determination threshold value T, and the color change is a change in a predetermined pattern set in the place contact determination threshold value T. When it becomes (St139: Yes), it determines with the operating body having contacted the target object (St140). When the change in the color of the operating body does not change the predetermined pattern (St139: No), the recognition of the target object (St131) by the object recognition processing unit 104 is repeated.
- the contact determination unit 105 supplies a contact position with respect to the acquired image to the input control unit 106 when determining contact with the target object by the operating body.
- the input control unit 106 controls the operation input based on the position of the icon P displayed on the operation image G and the contact position supplied from the contact determination unit 105. For example, the input control unit 106 can select the icon P superimposed on the contact position.
- the information processing apparatus 100 can also perform the above operations.
- the information processing apparatus 100 can acquire the distance between the operating tool and the target object by using the depth information, and is acquired only when the distance between the operating tool and the target object is sufficiently small and the possibility of contact is high. Contact can be determined using the change in color of the image.
- the information processing apparatus 100 can determine not only the contact between the operating body and the target object but also the type of operation by the contact (dragging, pinching, etc.).
- FIG. 10 is a flowchart illustrating the contact and operation determination operation of the information processing apparatus 100.
- the contact determination step (St151) is the same flow as the contact determination operation shown in FIG. Further, the same step (St151) may be the same flow as the contact determination operation combined with the use of the normal direction shown in FIG. 8, and is the same as the contact reaction operation combined with the use of the depth information shown in FIG. This flow may be used.
- the contact determination unit 105 determines the strength and direction of contact (pressing) of the operating body with respect to the target object when determined to be contact in the contact determination step (St152: Yes) (St153). When it is not determined to be contact in the contact determination step (St152: No), the contact determination step (St151) is repeated.
- the contact determination unit 105 can determine the strength and direction of the contact (pressing) of the operating body with respect to the target object using the color change of the operating body.
- FIG. 11 is a schematic diagram showing an aspect of color change depending on the strength of contact of the operating body (finger) with the target object. As shown in the figure, when the operating body is the finger Y, when the pressing force shown in FIG. 11A is small and when the pressing force shown in FIG. Different.
- the contact determination unit 105 can determine the strength of pressing of the operating body against the target object by comparing the color change with a blood color change pattern (contact determination threshold T) according to the pressing force. Is possible.
- the operating body is not limited to the finger Y, and any object may be used as long as the color changes according to the pressure of the other object.
- FIG. 12 is a schematic diagram showing an aspect of color change depending on the direction of contact of the operating body (finger) with the target object.
- 12A shows a case where the pressing direction is perpendicular to the target object
- FIG. 12B shows a case where the pressing direction is the right direction
- FIG. 12C shows a case where the pressing direction is the left direction.
- 12 (d) shows a case where the pressing direction is upward
- FIG. 12 (e) shows a case where the pressing direction is downward.
- the contact determination unit 105 can determine the pressing direction of the operating body with respect to the target object by comparing the color change with the blood color change pattern (contact determination threshold T) corresponding to the pressing direction. is there.
- the operating body is not limited to the finger Y, and may be any one that changes in color according to the pressing direction of another object.
- the contact determination unit 105 supplies the input control unit 106 with the contact position with respect to the acquired image, the magnitude of the pressing force with respect to the target object of the operating body, and the pressing direction.
- the input control unit 106 determines whether or not the contact position between the operating tool and the target object has changed (St154). When the contact position has changed (St154: Yes), the input control unit 106 determines whether or not the number of operating bodies is two (St155). When the contact position has not changed (St155: No), the contact determination step (St151) is executed again.
- the input control unit 106 determines that the operation by the operating body is a pinch. Further, when the number of operation objects is not two, the input control unit 106 determines that the operation by the operation object is a drag. Note that, when the number of operating bodies is two, the input control unit 106 may determine that the operation is a drag (multi-touch drag) by a plurality of operating bodies if the moving direction of the contact location is substantially parallel. . Further, when the operation by the operating body is a pinch, the input control unit 106 may determine that the operation is a pinch-in or a pinch-out according to the moving direction of the contact location.
- FIG. 13 is a schematic diagram illustrating the shape of a finger as an operation body.
- the object recognition processing unit 104 supplies information to the input control unit 106 when a specific shape (such as finger orientation) of the operating tool is registered in the object recognition dictionary D in advance.
- the input control unit 106 can use the shape of the operating tool for determining the type of operation. For example, in the example illustrated in FIG.
- the object recognition processing unit 104 can recognize that the shape of the finger is a pinch operation shape (a U shape with the thumb and forefinger), and the input control unit 106 has a finger shape of Only when the shape is a pinch operation, it is possible to determine that the operation is a pinch operation.
- a pinch operation shape a U shape with the thumb and forefinger
- the input control unit 106 controls the operation input according to the type of operation by the operating body. At this time, the input control unit 106 may change the degree of the effect of the drag (the moving speed of the cursor, the scrolling speed, etc.) according to the magnitude of the pressing force on the target object of the operating body. Further, the input control unit 106 may control the direction of the operation input according to the direction of the pressing force of the operating body with respect to the target object even when the contact position between the operating body and the target object has not moved. In this case, the user can perform an operation input as if operating a pointing stick without moving the contact position of the operating tool.
- FIG. 14 is an example of a music playback application UI based on the above-described operation of the information processing apparatus 100. As shown in the figure, the information processing apparatus 100 can select the playback volume and the music to be played, depending on the direction and intensity of pressing the target object by the operating tool.
- FIG. 15 is a schematic diagram illustrating a functional configuration of an information processing device 200 according to the second embodiment of the present technology.
- the information processing apparatus 200 includes a voice input unit 201, a voice recognition processing unit 202, an image input unit 203, a contact determination unit 204, an input control unit 205, and an image output unit 206.
- the information processing apparatus 200 according to the present embodiment does not have an object recognition processing unit.
- the information processing apparatus 200 can be connected to the eyewear 10. Since the configuration of the eyewear 10 is the same as that of the first embodiment, description thereof is omitted.
- the audio input unit 201 is connected to the microphone of the eyewear 10 and receives an audio signal of the sound collected by the microphone.
- the voice input unit 201 outputs the acquired voice signal to the voice recognition processing unit 202.
- the voice recognition processing unit 202 performs voice recognition processing on the voice signal supplied from the voice input unit 201 and recognizes the operation voice by the user. If the operation voice is recognized, the voice recognition processing unit 202 supplies the voice recognition result to the input control unit 205.
- the image input unit 203 is connected to the camera 12 and an image (moving image) captured by the camera 12 is input.
- the image input unit 203 supplies the acquired image (hereinafter, acquired image) to the contact determination unit 204.
- the contact determination unit 204 performs contact determination on the acquired image supplied from the image input unit 203 using the contact determination threshold T. Details of this processing will be described later.
- the contact determination unit 204 supplies the determination result to the input control unit 205.
- the contact determination threshold value T may be stored in the information processing apparatus 200, or the contact determination unit 204 may acquire it from a network or the like.
- the input control unit 205 controls the operation input by the user based on the recognition result by the voice recognition processing unit 202 or the determination result by the contact determination unit 204. Specifically, the input control unit 205 accepts operation inputs such as touch, multi-touch, dragging, and pinching from the contact position, pressing force, pressing direction, movement of the operating body after contact, and the like with respect to the target object. The input control unit 205 supplies the received operation input to the OS or the like of the information processing apparatus 200.
- the image output unit 206 generates an image (moving image) displayed on the display 11.
- the image output unit 206 can generate an operation image including icons and operation buttons.
- the user visually recognizes the field of view (see FIGS. 2 and 3) in which the image displayed on the display 11 and the real object are superimposed.
- the information processing apparatus 200 is capable of displaying or projecting an image on a target object and connected to an apparatus including a camera capable of capturing the target object. Is possible.
- the functional configuration of the information processing apparatus 200 as described above may be connected to or built in an image projection apparatus such as an eyewear or a projector, or may be mounted on a PC or smartphone other than the image projection apparatus. . A part or all of the functional configuration may be configured on a network.
- the functional configuration of the information processing apparatus 200 described above can be realized by the hardware configuration shown in the first embodiment.
- FIG. 16 is a flowchart showing the operation of the information processing apparatus 200.
- an image is captured by the camera 12, and the image input unit 203 acquires an image (acquired image).
- the image input unit 203 supplies the acquired image to the contact determination unit 204.
- the image output unit 206 supplies the operation image G (see FIG. 2) to the display 11 for display (St201). Unlike the first embodiment, the target object recognition process is not executed, but the user can project the operation image G onto the target object by pointing the display 11 toward the target object (desktop, wall surface, or the like). Is possible.
- the contact determination unit 204 calculates a change in the color of the operating tool (St202).
- the contact determination unit 204 can calculate a change in a predetermined color (hereinafter referred to as a determination target color).
- the determination target color is a color corresponding to the blood color of a finger, for example.
- the color of the stylus that is assumed to be used may be used as the determination target color.
- the contact determination unit 204 can specify the color for which the change is to be calculated without performing the operation body recognition process.
- the determination target color can be a color having a certain width (pixel value range). Also, a plurality of colors may be set as the determination target colors.
- the contact determination unit 204 compares the change in the color of the operating body (determination target color) with the contact determination threshold value T, and the change in color is a change in a predetermined pattern set in the place contact determination threshold value T. In the case (St203: Yes), it is determined that the operating body has touched the target object (St204).
- the contact determination unit 204 can determine contact for each operation object by regarding that there are a plurality of operation objects when there are a plurality of determination target color regions.
- the color change pattern (contact determination threshold T) used by the contact determination unit 204 for determination may be determined according to some rule, or may be determined by machine learning that repeats contact determination for a large number of subjects. Further, the contact determination unit 204 may accumulate the blood color tendency of the fingertip of the user who is using the information processing apparatus 200 and execute the calibration of the contact determination threshold value T.
- the contact determination unit 204 supplies a contact position with respect to the acquired image to the input control unit 205 when determining contact with the target object by the operating tool.
- the input control unit 205 controls the operation input based on the contact determination result supplied from the contact determination unit 204.
- the information processing apparatus 200 operates as described above. As described above, the information processing apparatus 200 performs contact determination using an image captured by a normal camera. That is, no contact determination means such as a 3D sensor or a stereo camera is required, and the device can be reduced in size and cost. Further, the color change of the target object does not occur unless the operating tool is in contact with the target object. That is, the information processing apparatus 200 performs the contact determination only when the operating tool is reliably in contact with the target object. In order to execute, operation input can be received with high accuracy.
- the operating body is not limited to the finger.
- the contact may be determined using a color change accompanying the deformation of the stylus due to the contact with the target object.
- a stylus that changes color or emits light when touching something may be used as the operating body.
- the contact determination unit 204 determines contact using the color change of the operating tool.
- the present invention is not limited to this, and the contact determination unit 204 determines contact using the color change of the target object. Also good.
- the contact determination unit 204 may determine contact using the blood color of the palm or the color change of the liquid crystal display.
- the contact determination unit 204 may determine contact using a color change caused by deformation of the target object.
- the contact determination unit 204 may determine the contact using the color of the shadow of the operating body as a determination target color and using the change in the color. For example, the contact determination unit 204 can determine contact by using a color (shadow) darker than the surrounding as a determination target color.
- the information processing apparatus 200 operates as described above. Note that the information processing apparatus 200 may determine contact (see FIG. 9) using the color change and the depth information, as in the first embodiment. Further, the information processing apparatus 200 may determine the type of operation by the contact (see FIG. 10) after determining the contact.
- An information processing apparatus comprising: a contact determination unit that determines contact of the operation object with respect to the target object based on a color change in an image including the target object and the operation object.
- the object recognition processing unit recognizes the operation body using an object detection dictionary.
- the object recognition processing unit recognizes an object that has entered the image as an operating body.
- the information processing apparatus according to any one of (1) to (5) above, The information processing apparatus, wherein the predetermined color is a color corresponding to a blood color of a fingertip.
- the said contact determination part determines the contact of the said operation body with respect to the said target object based on the change of the color of the said target object by the shadow of the said operation body.
- the contact determination unit further determines the contact strength of the operating body with respect to the target object based on a color change in the image.
- the contact determination unit further determines a contact direction of the operating body with respect to the target object based on a color change in the image.
- An information processing apparatus further comprising: an input control unit that controls an operation input based on a determination result of the contact determination unit.
- the information processing apparatus according to any one of (1) to (13) above, An object recognition processing unit for recognizing an operation tool included in the image, The input control unit controls an operation input based on the shape of the operation body recognized by the object recognition processing unit and the determination result of the contact determination unit.
- the information processing apparatus according to any one of (1) to (14) above, An object recognition processing unit for recognizing a target object included in the image; The contact determination unit determines contact of the operating body with the target object based on a color change in the image when the target object is a rigid body, and normals when the target object is not a rigid body. An information processing apparatus that determines contact of the operating body with the target object based on a direction of a normal to the surface of the target object acquired by a direction detection sensor.
- the contact determination unit determines contact of the operation body with the target object based on a color change in the image when a distance between the target object and the operation body acquired by a depth sensor is equal to or less than a predetermined value. Processing equipment.
- a program that causes an information processing apparatus to operate as a contact determination unit that determines contact of the operation object with respect to the target object based on a color change in an image including the target object and the operation object.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
上記接触判定部は、対象物体と操作体を含む画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する。
上記画像に含まれる操作体を認識する物体認識処理部をさらに具備し、
上記接触判定部は、上記操作体の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定してもよい。
をさらに具備し、
上記入力制御部は、上記操作対象画像における上記操作体と上記操作対象物体の接触位置に基づいて操作入力を制御してもよい。
上記入力制御部は、上記物体認識処理部によって認識された操作体の形状と、上記接触判定部の判定結果に基づいて、操作入力を制御してもよい。
上記接触判定部は、上記対象物体が剛体である場合には上記画像における色の変化に基づいて上記対象物体に対する上記操作体の接触を判定し、上記対象物体が剛体ではない場合には法線方向検出センサによって取得された上記対象物体の表面に対する法線の方向に基づいて上記対象物体に対する上記操作体の接触を判定してもよい。
本技術の第1の実施形態に係る情報処理装置について説明する。
図1は、本技術の第1の実施形態に係る情報処理装置100の機能的構成を示す模式図である。同図に示すように、情報処理装置100は、音声入力部101、音声認識処理部102、画像入力部103、物体認識処理部104、接触判定部105、入力制御部106及び画像出力部107を備える。
情報処理装置100の動作について説明する。図6は情報処理装置100の動作を示すフローチャートである。上記のようにカメラ12によって画像が撮像され、画像入力部103は画像(取得画像)を取得する。画像入力部103は取得画像を物体認識処理部104に供給する。
情報処理装置100の別の動作について説明する。なお、情報処理装置100と接続されるアイウェア10(又は他の装置)のカメラ12は偏光イメージングが可能なカメラであるものとする。偏光イメージングは、撮像範囲の輝度画像と、撮像範囲に含まれる法線(物体に表面に垂直な線)の方向を取得できる技術である。また、カメラ12は偏光イメージとは異なる方法によって物体の法線方向を取得してもよく、アイウェア10はカメラ12とは別に法線方向の取得が可能なセンサを備えていてもよい。
情報処理装置100の別の動作について説明する。なお、情報処理装置100と接続されるアイウェア10(又は他の装置)には、デプス情報の取得が可能なデプスセンサが設けられている。デプス情報は、物体とデプスセンサの間の距離情報である。
情報処理装置100は、操作体と対象物体の接触のみならず、接触による操作の種別(ドラッグやピンチ等)を判定することも可能である。図10は、情報処理装置100の接触及び操作判定動作を示すフローチャートである。
本技術の第2の実施形態に係る情報処理装置について説明する。
図15は、本技術の第2の実施形態に係る情報処理装置200の機能的構成を示す模式図である。同図に示すように、情報処理装置200は、音声入力部201、音声認識処理部202、画像入力部203、接触判定部204、入力制御部205、画像出力部206を備える。本実施形態に係る情報処理装置200は、物体認識処理部を有しない。
情報処理装置200の動作について説明する。図16は情報処理装置200の動作を示すフローチャートである。上記のようにカメラ12によって画像が撮像され、画像入力部203が画像(取得画像)を取得する。画像入力部203は取得画像を接触判定部204に供給する。
対象物体と操作体を含む画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する接触判定部
を具備する情報処理装置。
上記(1)に記載の情報処理装置であって、
上記画像に含まれる操作体を認識する物体認識処理部をさらに具備し、
上記接触判定部は、上記操作体の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)又は(2)に記載の情報処理装置であって、
上記物体認識処理部は、物体検出辞書を利用して上記操作体を認識する
情報処理装置。
上記(1)から(3)のいずれか一つに記載の情報処理装置であって、
上記物体認識処理部は、上記画像に進入した物体を操作体として認識する
情報処理装置。
上記(1)から(4)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、所定の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)から(5)のいずれか一つに記載の情報処理装置であって、
上記所定の色は、指先の血色に相当する色である
情報処理装置。
上記(1)から(6)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、上記操作体の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)から(7)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、上記対象物体の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)から(8)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、上記操作体の影による上記対象物体の色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)から(9)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、さらに、上記画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触の強さを判定する
情報処理装置。
上記(1)から(10)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、さらに、上記画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触の方向を判定する
情報処理装置。
上記(1)から(11)のいずれか一つに記載の情報処理装置であって、
上記接触判定部の判定結果に基づいて、操作入力を制御する入力制御部
をさらに具備する情報処理装置。
上記(1)から(12)のいずれか一つに記載の情報処理装置であって、
上記対象物体に重畳される操作対象画像を生成する画像出力部
をさらに具備し、
上記入力制御部は、上記操作対象画像における上記操作体と上記操作対象物体の接触位置に基づいて操作入力を制御する
情報処理装置。
上記(1)から(13)のいずれか一つに記載の情報処理装置であって、
上記画像に含まれる操作体を認識する物体認識処理部をさらに具備し、
上記入力制御部は、上記物体認識処理部によって認識された操作体の形状と、上記接触判定部の判定結果に基づいて、操作入力を制御する
情報処理装置。
上記(1)から(14)のいずれか一つに記載の情報処理装置であって、
上記画像に含まれる対象物体を認識する物体認識処理部をさらに具備し、
上記接触判定部は、上記対象物体が剛体である場合には上記画像における色の変化に基づいて上記対象物体に対する上記操作体の接触を判定し、上記対象物体が剛体ではない場合には法線方向検出センサによって取得された上記対象物体の表面に対する法線の方向に基づいて上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
上記(1)から(15)のいずれか一つに記載の情報処理装置であって、
上記接触判定部は、デプスセンサによって取得された上記対象物体と上記操作体の距離が所定値以下の場合に、上記画像における色の変化に基づいて上記対象物体に対する上記操作体の接触を判定する
情報処理装置。
対象物体と操作体を含む画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する接触判定部
として情報処理装置を動作させるプログラム。
接触判定部が、対象物体と操作体を含む画像における色の変化に基づいて、上記対象物体に対する上記操作体の接触を判定する
情報処理方法。
101、201…音声入力部
102、202…音声認識処理部
103、203…画像入力部
104…物体認識処理部
105、204…接触判定部
106、205…入力制御部
107、206…画像出力部
Claims (18)
- 対象物体と操作体を含む画像における色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する接触判定部
を具備する情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記画像に含まれる操作体を認識する物体認識処理部をさらに具備し、
前記接触判定部は、前記操作体の色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項2に記載の情報処理装置であって、
前記物体認識処理部は、物体検出辞書を利用して前記操作体を認識する
情報処理装置。 - 請求項2に記載の情報処理装置であって、
前記物体認識処理部は、前記画像に進入した物体を操作体として認識する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、所定の色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項5に記載の情報処理装置であって、
前記所定の色は、指先の血色に相当する色である
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、前記操作体の色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、前記対象物体の色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、前記操作体の影による前記対象物体の色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、さらに、前記画像における色の変化に基づいて、前記対象物体に対する前記操作体の接触の強さを判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、さらに、前記画像における色の変化に基づいて、前記対象物体に対する前記操作体の接触の方向を判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部の判定結果に基づいて、操作入力を制御する入力制御部
をさらに具備する情報処理装置。 - 請求項12に記載の情報処理装置であって、
前記対象物体に重畳される操作対象画像を生成する画像出力部
をさらに具備し、
前記入力制御部は、前記操作対象画像における前記操作体と前記操作対象物体の接触位置に基づいて操作入力を制御する
情報処理装置。 - 請求項12に記載の情報処理装置であって、
前記画像に含まれる操作体を認識する物体認識処理部をさらに具備し、
前記入力制御部は、前記物体認識処理部によって認識された操作体の形状と、前記接触判定部の判定結果に基づいて、操作入力を制御する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記画像に含まれる対象物体を認識する物体認識処理部をさらに具備し、
前記接触判定部は、前記対象物体が剛体である場合には前記画像における色の変化に基づいて前記対象物体に対する前記操作体の接触を判定し、前記対象物体が剛体ではない場合には法線方向検出センサによって取得された前記対象物体の表面に対する法線の方向に基づいて前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 請求項1に記載の情報処理装置であって、
前記接触判定部は、デプスセンサによって取得された前記対象物体と前記操作体の距離が所定値以下の場合に、前記画像における色の変化に基づいて前記対象物体に対する前記操作体の接触を判定する
情報処理装置。 - 対象物体と操作体を含む画像における色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する接触判定部
として情報処理装置を動作させるプログラム。 - 接触判定部が、対象物体と操作体を含む画像における色の変化に基づいて、前記対象物体に対する前記操作体の接触を判定する
情報処理方法。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP15826923.3A EP3176681A4 (en) | 2014-07-30 | 2015-07-14 | Information processing device, information processing method and program |
CN201580040986.8A CN106575184B (zh) | 2014-07-30 | 2015-07-14 | 信息处理装置、信息处理方法及计算机可读介质 |
JP2016537736A JP6686885B2 (ja) | 2014-07-30 | 2015-07-14 | 情報処理装置、情報処理方法及びプログラム |
US15/319,440 US10175825B2 (en) | 2014-07-30 | 2015-07-14 | Information processing apparatus, information processing method, and program for determining contact on the basis of a change in color of an image |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-155227 | 2014-07-30 | ||
JP2014155227 | 2014-07-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016017096A1 true WO2016017096A1 (ja) | 2016-02-04 |
Family
ID=55217021
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/003560 WO2016017096A1 (ja) | 2014-07-30 | 2015-07-14 | 情報処理装置、情報処理方法及びプログラム |
Country Status (5)
Country | Link |
---|---|
US (1) | US10175825B2 (ja) |
EP (1) | EP3176681A4 (ja) |
JP (1) | JP6686885B2 (ja) |
CN (1) | CN106575184B (ja) |
WO (1) | WO2016017096A1 (ja) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB201420859D0 (en) * | 2014-11-24 | 2015-01-07 | Cancer Res Inst Royal | Tumour analysis |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005165670A (ja) * | 2003-12-02 | 2005-06-23 | Olympus Corp | ヒューマンインタフェース装置及びヒューマンインタフェースシステム |
JP2012022664A (ja) * | 2010-07-13 | 2012-02-02 | Samsung Electro-Mechanics Co Ltd | タッチパネルの圧力センシングモジュール及びその作動方法 |
WO2013168508A1 (ja) * | 2012-05-09 | 2013-11-14 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
WO2014034527A1 (ja) * | 2012-08-27 | 2014-03-06 | シチズンホールディングス株式会社 | 情報入力装置 |
JP2014071505A (ja) * | 2012-09-27 | 2014-04-21 | Aisin Aw Co Ltd | 電子機器、電子機器の制御方法及びコンピュータプログラム |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5260643B2 (ja) * | 2008-09-29 | 2013-08-14 | パナソニック株式会社 | ユーザインターフェース装置、ユーザインターフェース方法、及び記録媒体 |
JP2011070491A (ja) | 2009-09-28 | 2011-04-07 | Nec Personal Products Co Ltd | 入力方法、情報処理装置、タッチパネル及びプログラム |
CN102200881B (zh) * | 2010-03-24 | 2016-01-13 | 索尼公司 | 图像处理装置以及图像处理方法 |
US9030425B2 (en) * | 2011-04-19 | 2015-05-12 | Sony Computer Entertainment Inc. | Detection of interaction with virtual object from finger color change |
JPWO2012173001A1 (ja) * | 2011-06-13 | 2015-02-23 | シチズンホールディングス株式会社 | 情報入力装置 |
JP6037900B2 (ja) * | 2013-03-11 | 2016-12-07 | 日立マクセル株式会社 | 操作検出装置及び操作検出方法 |
CN103246351B (zh) * | 2013-05-23 | 2016-08-24 | 刘广松 | 一种用户交互系统和方法 |
-
2015
- 2015-07-14 EP EP15826923.3A patent/EP3176681A4/en not_active Withdrawn
- 2015-07-14 US US15/319,440 patent/US10175825B2/en not_active Expired - Fee Related
- 2015-07-14 JP JP2016537736A patent/JP6686885B2/ja not_active Expired - Fee Related
- 2015-07-14 WO PCT/JP2015/003560 patent/WO2016017096A1/ja active Application Filing
- 2015-07-14 CN CN201580040986.8A patent/CN106575184B/zh not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005165670A (ja) * | 2003-12-02 | 2005-06-23 | Olympus Corp | ヒューマンインタフェース装置及びヒューマンインタフェースシステム |
JP2012022664A (ja) * | 2010-07-13 | 2012-02-02 | Samsung Electro-Mechanics Co Ltd | タッチパネルの圧力センシングモジュール及びその作動方法 |
WO2013168508A1 (ja) * | 2012-05-09 | 2013-11-14 | ソニー株式会社 | 情報処理装置、情報処理方法、及びプログラム |
WO2014034527A1 (ja) * | 2012-08-27 | 2014-03-06 | シチズンホールディングス株式会社 | 情報入力装置 |
JP2014071505A (ja) * | 2012-09-27 | 2014-04-21 | Aisin Aw Co Ltd | 電子機器、電子機器の制御方法及びコンピュータプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3176681A4 * |
Also Published As
Publication number | Publication date |
---|---|
EP3176681A4 (en) | 2018-04-18 |
US20170139545A1 (en) | 2017-05-18 |
JP6686885B2 (ja) | 2020-04-22 |
CN106575184B (zh) | 2020-05-19 |
EP3176681A1 (en) | 2017-06-07 |
US10175825B2 (en) | 2019-01-08 |
JPWO2016017096A1 (ja) | 2017-04-27 |
CN106575184A (zh) | 2017-04-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8972902B2 (en) | Compound gesture recognition | |
US9092058B2 (en) | Information processing apparatus, information processing method, and program | |
JP6528774B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
US8581869B2 (en) | Information processing apparatus, information processing method, and computer program | |
US20120105367A1 (en) | Methods of using tactile force sensing for intuitive user interface | |
US9612736B2 (en) | User interface method and apparatus using successive touches | |
WO2011142317A1 (ja) | ジェスチャー認識装置、方法、プログラム、および該プログラムを格納したコンピュータ可読媒体 | |
US20080134078A1 (en) | Scrolling method and apparatus | |
CN109643213B (zh) | 用于协同编辑工具的触摸屏用户界面的系统和方法 | |
JP2012221073A (ja) | 情報処理装置、情報処理方法およびコンピュータプログラム | |
US20150193037A1 (en) | Input Apparatus | |
KR20130052749A (ko) | 터치 기반 사용자 인터페이스 장치 및 방법 | |
KR20040063153A (ko) | 제스쳐에 기초를 둔 사용자 인터페이스를 위한 방법 및 장치 | |
JP6253284B2 (ja) | 情報処理装置およびその制御方法、プログラム、記録媒体 | |
WO2017029749A1 (ja) | 情報処理装置、その制御方法、プログラム、及び記憶媒体 | |
JP5374564B2 (ja) | 描画装置、描画制御方法、及び描画制御プログラム | |
Bonnet et al. | Extending the vocabulary of touch events with ThumbRock | |
JP2014182814A (ja) | 描画装置、描画方法および描画プログラム | |
US10656746B2 (en) | Information processing device, information processing method, and program | |
CN106325726B (zh) | 触控互动方法 | |
JP2013186702A (ja) | 入力処理装置、入力処理プログラム及び入力処理方法 | |
WO2009119716A1 (ja) | 情報処理システム、情報処理装置、方法及びプログラム | |
JP6686885B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
US20150268736A1 (en) | Information processing method and electronic device | |
TW201246019A (en) | Electronic apparatus, display method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15826923 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15319440 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2016537736 Country of ref document: JP Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2015826923 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |