WO2019054037A1 - Dispositif de traitement d'informations, procédé de traitement d'informations et programme - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et programme Download PDF

Info

Publication number
WO2019054037A1
WO2019054037A1 PCT/JP2018/026799 JP2018026799W WO2019054037A1 WO 2019054037 A1 WO2019054037 A1 WO 2019054037A1 JP 2018026799 W JP2018026799 W JP 2018026799W WO 2019054037 A1 WO2019054037 A1 WO 2019054037A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
gesture
information processing
recognition unit
processing apparatus
Prior art date
Application number
PCT/JP2018/026799
Other languages
English (en)
Japanese (ja)
Inventor
賢次 杉原
真里 斎藤
Original Assignee
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニー株式会社 filed Critical ソニー株式会社
Publication of WO2019054037A1 publication Critical patent/WO2019054037A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the present disclosure relates to an information processing device, an information processing method, and a program.
  • Patent Document 1 discloses a technique for performing gesture recognition based on the posture of a user with respect to an imaging device.
  • Patent Document 1 corrects the movement of the target site based on the relative posture of the user with respect to the imaging device. For this reason, with the technique described in Patent Document 1, it is difficult to improve the accuracy of gesture recognition when the posture of the user with respect to the imaging device does not change.
  • the present disclosure proposes a new and improved information processing apparatus, an information processing method, and a program capable of realizing more accurate gesture recognition.
  • a gesture recognition unit that recognizes a gesture based on a user's operation, and an operation control unit that generates a control signal related to operation control of an operation target object by the gesture based on the recognized gesture
  • An information processing apparatus wherein the gesture recognition unit determines a coordinate axis of space coordinates related to recognition of the gesture based on an orientation of the user.
  • a processor recognizes a gesture based on a user's operation, and generates a control signal related to operation control of an operation target by the gesture based on the recognized gesture. And C., wherein the recognizing further includes determining a coordinate axis of space coordinates related to recognition of the gesture based on an orientation of the user.
  • a computer a gesture recognition unit that recognizes a gesture based on a user's operation, and a control signal related to operation control of an operation target based on the gesture are generated based on the recognized gesture.
  • FIG. 1 is a block diagram illustrating an exemplary system configuration and an exemplary functional configuration of an information processing apparatus according to an embodiment of the present disclosure. It is a figure for demonstrating determination of the coordinate axis of the space coordinate which concerns on gesture recognition in, when the inclination of the user based on the embodiment is less than a threshold value.
  • Embodiment 1.1 Outline of embodiment 1.2.
  • System configuration example 1.3 Functional configuration example of information processing apparatus 10 1.4. Details of gesture recognition 1.5. Flow of operation 2.
  • a technology has been developed which recognizes a gesture by a user and controls the operation of an object (hereinafter sometimes referred to as an operation object) based on the gesture.
  • a terminal such as an HMD (Head Mount Display: HMD) recognizes an action of a user's hand or finger as a gesture, and causes an object to execute various actions according to the recognized gesture.
  • HMD Head Mount Display: HMD
  • the user can intuitively use various objects such as a device existing in real space or a virtual object displayed using VR (Virtual Reality) or AR (Augmented Reality) technology. It is possible to operate and give instructions.
  • VR Virtual Reality
  • AR Augmented Reality
  • a terminal such as an HMD
  • a predetermined coordinate axis used to determine the inclination may be referred to as a reference axis.
  • FIGS. 1A and 1B are diagrams for explaining a deviation between a predetermined coordinate axis involved in gesture recognition and an actual gesture direction.
  • the visual field FV of the user U who gives an operation instruction to the object 20 by a gesture is shown, and in the lower part, the positional relationship between the object 20 and the user U and the direction of the gesture are shown. ing.
  • the user U is making a gesture of projecting the hand UH to the back with respect to the object 20 which is an illumination device operable by a gesture, which is present in the real space.
  • the gesture may be, for example, a gesture corresponding to a switching instruction related to turning on / off the object 20.
  • the user U to perform the gesture using hand UH on the basis of user's direction, such as a user's gaze direction SD.
  • a user's gaze direction SD such as a user's gaze direction SD.
  • FIG. 1A the true gesture direction MD intended by the user U is shown along the viewing direction SD.
  • the gesture direction by the hand UH is often recognized on the basis of a preset default coordinate axis.
  • the general information processing terminal 90 recognizes a gesture with a predetermined coordinate axis based on the direction DD of the information processing terminal 90 in the horizontal direction. For this reason, a shift occurs between the gesture direction FMD detected by the general information processing terminal 90 and the true gesture direction MD intended by the user U.
  • the user U turns the line of sight upward to view the illumination fixture, and the gesture direction FMD and the gesture direction MD There may be a gap between them.
  • the user U makes a gesture of moving the hand UH in the downward direction with respect to the object 20 which is the illumination device existing in the real space.
  • the gesture may be, for example, a gesture that instructs the object 20 to reduce the illuminance.
  • the user U makes the above gesture based on the direction of the user such as the gaze direction SD.
  • the general information processing terminal 90 in order to perform gesture recognition based on a predetermined coordinate axis, a shift occurs between the gesture direction FMD detected by the information processing terminal 90 and the true gesture direction MD intended by the user U. It becomes.
  • the gesture recognition is performed using the predetermined coordinate axes set in advance without considering the direction of the user U, the true gesture direction FD intended by the user U is accurately determined. It may be difficult to recognize. Therefore, in the general information processing terminal 90, there may occur a situation in which the gesture intended by the user U can not be correctly recognized or a situation in which a gesture different from the operation instruction intended by the user U is erroneously recognized.
  • FIG. 2 is a diagram for explaining a change in the gesture direction accompanying the inclination of the user U.
  • FIG. 2 shows an example of the case where the user U wearing the general information processing terminal 90 gives an operation instruction by a gesture to the target 20 disposed on the ceiling. Note that FIG. 2 shows an example in which the user U makes a gesture on the object 20 in a state of lying on a floor or the like.
  • the user U makes a gesture of moving the hand UH in the direction of gravity as shown in FIG. 1B based on the normal standing state and the posture of the object 20.
  • the general information processing terminal 90 recognizes a gesture based on a predetermined coordinate axis, it is difficult to correctly recognize the true gesture direction MD intended by the user U.
  • the general information processing terminal 90 there is a possibility that the wrong gesture direction is recognized, and the object 20 is caused to execute an unintended operation of the user U.
  • An information processing apparatus, an information processing method, and a program according to an embodiment of the present disclosure are conceived based on the above points, and are more accurate even when the user's orientation changes. It is possible to realize gesture recognition.
  • an information processing apparatus, an information processing method, and a program according to an embodiment of the present disclosure dynamically determine coordinate axes of space coordinates related to recognition of a gesture based on detected user orientations. Is one of the features.
  • an information processing apparatus, an information processing method, and a configuration and a function for realizing the above-described feature of a program according to an embodiment of the present disclosure will be described in detail.
  • FIG. 3 is a block diagram showing an example of the system configuration of this embodiment and an example of the functional configuration of the information processing apparatus 10.
  • the information processing system according to the present embodiment may include an information processing device 10, an object 20, and a sensor device 30. Further, the information processing apparatus 10 and the object 20, and the information processing apparatus 10 and the sensor apparatus 30 are connected via the network 40 so as to be able to communicate with each other.
  • the information processing apparatus 10 has a function of recognizing a gesture by the user and controlling the operation of the target 20 based on the gesture. Under the present circumstances, the information processing apparatus 10 which concerns on this embodiment has a function which determines the coordinate axis of the space coordinate which concerns on recognition of a gesture dynamically based on the detected direction of the user.
  • the orientation of the user according to the present embodiment may be, for example, the orientation of the head of the user. More specifically, the orientation of the user according to the present embodiment includes the gaze direction of the user.
  • the information processing apparatus 10 according to the present embodiment can detect the gaze direction of the user based on, for example, the eye movement of the user and the direction of the information processing apparatus 10.
  • the information processing apparatus 10 may be, for example, a wearing terminal such as an HMD or a glasses-type wearable device worn by the user on the head.
  • the information processing apparatus 10 according to the present embodiment may be various apparatuses that perform gesture recognition based on sensor information collected by the attached terminal or the like.
  • the target 20 performs various operations based on control by the information processing apparatus 10.
  • the information processing apparatus 10 can cause the target 20 to execute the operation corresponding to the gesture based on the recognized gesture of the user.
  • the target 20 according to the present embodiment may be various devices in real space.
  • the object 20 may be, for example, a home appliance, a game machine, a general-purpose computer, an autonomous mobile robot, or the like.
  • the object 20 according to the present embodiment may be a virtual object displayed using VR or AR technology.
  • the sensor device 30 is a device that collects sensor information related to the direction of the user.
  • the sensor device 30 transmits the collected sensor information to the information processing device 10 via the network 40.
  • the sensor device 30 may be a device having an imaging function.
  • the information processing apparatus 10 can estimate the direction of the user such as the gaze direction based on the image of the user captured by the sensor device 30.
  • the sensor apparatus 30 which concerns on this embodiment may be equipped with the infrared sensor which detects the infrared rays which an infrared marker which a user mounts
  • the information processing apparatus 10 can acquire the direction of the user based on the change in position of the infrared marker detected by the sensor device 30.
  • the sensor device 30 may be a device equipped with an acceleration sensor, a gyro sensor, a geomagnetic sensor, and the like and worn by the user.
  • the information processing apparatus 10 can estimate the direction of the user based on the information such as the acceleration, the angular velocity, and the direction collected by the sensor device 30.
  • calibration is performed so that the information processing device 10 and the sensor device 30 share in advance a coordinate axis serving as a reference.
  • the network 40 has a function of connecting the information processing device 10 to the target 20 and the information processing device 10 to the sensor device 30.
  • the network 40 may include the Internet, a public line network such as a telephone network, a satellite communication network, various LANs (Local Area Networks) including Ethernet (registered trademark), a WAN (Wide Area Network), and the like.
  • the network 40 may include a dedicated line network such as an Internet Protocol-Virtual Private Network (IP-VPN).
  • IP-VPN Internet Protocol-Virtual Private Network
  • the network 40 may also include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).
  • the configuration example of the information processing system according to the present embodiment has been described above.
  • the configuration described above with reference to FIG. 3 is merely an example, and the configuration of the information processing system according to the present embodiment is not limited to such an example.
  • the information processing system according to the present embodiment may not necessarily include the sensor apparatus 30.
  • the configuration of the information processing system according to the present embodiment can be flexibly deformed according to the specification and the operation.
  • the information processing apparatus 10 includes a sensor unit 110, a user recognition unit 120, an object recognition unit 130, a gesture recognition unit, an operation control unit 150, and a communication unit 160.
  • the sensor unit 110 has a function of collecting sensor information related to the direction of the user.
  • the orientation of the user according to the present embodiment includes the gaze direction of the user.
  • the sensor unit 110 according to the present embodiment may include an acceleration sensor, a gyro sensor, a geomagnetic sensor, etc. .
  • the sensor unit 110 is a light source that emits light to the user's eye and reflection from the corneal surface. You may provide the imaging sensor which detects light.
  • the sensor unit 110 inputs the collected various sensor information to the user recognition unit 120.
  • the user recognition unit 120 has a function of performing recognition related to the user. Specifically, the user recognition unit 120 according to the present embodiment functions as a direction detection unit that detects the direction of the user based on the sensor information collected by the sensor unit 110 or the sensor device 30.
  • the user recognition unit 120 may detect the gaze direction based on the eye movement of the user using, for example, the image captured by the sensor unit 110.
  • the user recognition unit 120 calculates the direction of the information processing apparatus 10 worn on the head by the user based on the acceleration information, the angular velocity information, the geomagnetic information, etc. collected by the sensor unit 110 as the user's gaze direction. It may be detected.
  • the user recognition unit 120 may detect the direction of the user based on the relative position of the object 20 and the user.
  • the user recognition unit 120 can also detect, for example, the direction in which the target 20 is present based on the position of the user as the direction of the user.
  • the user recognition unit 120 has a function of recognizing the user's operation.
  • the user recognition unit 120 may detect, for example, the motion of the user's hand or the like from the image captured by the sensor unit 110. At this time, the user recognition unit 120 can recognize the three-dimensional direction, the size of the motion, and the like related to the motion of the user. Note that, when the sensor unit 110 that captures the user's hand is provided in a wearable device mounted on the user's head, an image may be generated according to the user's orientation even if the user intends the same gesture. For example, the shape of the user's hand inside may be different.
  • templates such as the shape of the user's hand for recognizing a gesture differ according to the orientation of the user. You may Alternatively, depending on the orientation of the user, a template such as the shape of the user's hand for recognizing a gesture may be machine-learned.
  • the user recognition unit 120 inputs, to the gesture recognition unit 140, information related to the detected orientation and operation of the user.
  • the target recognition unit 130 has a function of performing recognition related to the target 20. Specifically, the object recognition unit 130 has a function of specifying the object 20 to be controlled.
  • the target recognition unit 130 according to the present embodiment may specify the target 20 to be controlled based on the direction of the user detected by the user recognition unit 120, for example.
  • the object recognition unit 130 can specify, as the object 20, a device or a virtual object targeted for the user's gaze direction or gaze point.
  • the object recognition unit 130 may specify the object 20 by performing image recognition based on the image captured by the sensor unit 110.
  • the target recognition unit 130 may specify the target 20 based on an input act including the user's utterance.
  • the target recognition unit 130 can also execute, for example, voice recognition related to the user's speech, and specify the target 20 intended by the user.
  • the object recognition unit 130 can also specify the object 20 based on identification information and the like transmitted from the object 20.
  • the object recognition unit 130 may recognize a preset device, a virtual object, or the like as the object 20.
  • the object recognition unit 130 has a function of detecting the tilt of the object 20.
  • the object recognition unit 130 detects the tilt of the object 20 based on, for example, the difference between the upper and lower axes of the predetermined coordinate axes and the upper and lower axes determined based on the state of the object 20.
  • the object recognition unit 130 may detect the head top direction of the object 20 as the vertical axis of the object 20.
  • the object recognition unit 130 can detect the tilt of the object 20 based on the change in the vertical axis determined for each object 20 regardless of the above example.
  • the information on the tilt of the object 20 may be transmitted from the object 20. That is, the object 20 can also transmit information on the detected inclination of itself to the information processing apparatus 10 together with the identification information.
  • the target recognition unit 130 can specify the target 20 and the tilt of the target 20 based on the received information.
  • the target recognition unit 130 inputs, to the gesture recognition unit 140, the specified target 20 and information on the tilt of the target 20.
  • the gesture recognition unit 140 has a function of recognizing a gesture based on the user's operation detected by the user recognition unit 120. Under the present circumstances, the gesture recognition part 140 which concerns on this embodiment makes one of the characteristics the coordinate axis of the space coordinate which concerns on recognition of a gesture based on the direction of the user which the user recognition part 120 detected.
  • the gesture recognition unit 140 may determine the longitudinal axis, the horizontal axis, and the vertical axis in the above coordinate axes based on the direction of the user detected by the user recognition unit 120. Also, the gesture recognition unit 140 executes gesture recognition based on the user's action on the coordinate axes determined as described above.
  • the orientation of the user according to the present embodiment may include the gaze direction of the user. That is, the gesture recognition unit 140 according to the present embodiment can determine the coordinate axes of the space coordinates related to the recognition of the gesture based on the gaze direction of the user detected by the user recognition unit 120. According to the above-described function of the gesture recognition unit 140 according to the present embodiment, the coordinate axis according to the user's direction is dynamically determined even when the user's direction such as the gaze direction changes each time. It is possible to always realize highly accurate gesture recognition.
  • the gesture recognition unit 140 may determine the coordinate axes of the space coordinates related to the recognition of the gesture based on the inclination of the user or the target 20.
  • the above-described functions of the gesture recognition unit 140 according to the present embodiment will be separately described in detail.
  • the gesture recognition unit 140 inputs, to the operation control unit 150, information on the gesture recognized as described above.
  • the operation control unit 150 generates a control signal related to the operation control of the object 20 based on the gesture recognized by the gesture recognition unit 140.
  • the operation control unit 150 according to the present embodiment can cause the object 20, which is a device existing in the real space, to execute various operations based on the gesture recognized by the gesture recognition unit 140, for example.
  • the operation control unit 150 may perform display control or the like related to the behavior of the target 20 which is a virtual object, based on the gesture recognized by the gesture recognition unit 140.
  • the operation control unit 150 may be configured to include various display devices that display visual information.
  • the operation control unit 150 may not necessarily perform only the operation control related to the object 20.
  • the operation control unit 150 according to the present embodiment can also control the output of visual information and sound information corresponding to the gesture based on the gesture recognized by the gesture recognition unit 140, for example.
  • the communication unit 160 has a function of performing information communication with the target 20 and the sensor device 30 via the network 40. Specifically, the communication unit 160 receives various sensor information collected by the sensor device 30. The communication unit 160 also transmits the control signal generated by the operation control unit 150 to the object 20. In addition, the communication unit 160 may receive, from the object 20, identification information, information on the tilt of the object 20, and the like.
  • the functional configuration of the information processing apparatus 10 according to the present embodiment has been described above.
  • the above-described functional configuration described using FIG. 3 is merely an example, and the functional configuration of the information processing apparatus 10 according to the present embodiment is not limited to such an example.
  • the information processing apparatus 10 according to the present embodiment may be a server provided separately from the HMD.
  • the information processing apparatus 10 can also perform gesture recognition based on the direction of the user or the direction of the user by receiving sensor information collected by the HMD worn by the user via the network 40. is there.
  • the functional configuration of the information processing apparatus 10 according to the present embodiment can be flexibly changed according to the specification and the operation.
  • gesture recognition by the gesture recognition unit 140 according to the present embodiment will be described in detail.
  • the gesture recognition unit 140 according to the present embodiment can determine coordinate axes of space coordinates related to gesture recognition based on the direction of the user recognized by the user recognition unit 120.
  • the coordinate axis according to the user's direction is dynamically determined even when the user's direction such as the gaze direction changes each time. It is possible to always realize highly accurate gesture recognition.
  • the gesture by the user is not always performed based on the orientation of the user.
  • coordinate axes in a normal state such as standing position or coordinate axes based on the object 20 (that is, the viewpoint of the object 20)
  • a gesture taking into account the coordinate axes defined by when the coordinate axis of the space coordinate which concerns on gesture recognition is determined based only on a user's direction, such as a gaze direction, the condition where a user does not recognize a gesture which a user intended correctly may arise.
  • the gesture recognition unit 140 may determine the coordinate axes of the space coordinates related to the gesture recognition based on the tilt of the user or the tilt of the target 20. According to the above-described function of the gesture recognition unit 140 according to the present embodiment, the above-mentioned coordinate axes can be determined dynamically according to the user or the state of the target 20, and gesture recognition with higher accuracy is realized. It becomes possible.
  • FIGS. 4 to 6 are diagrams for explaining determination of coordinate axes of space coordinates related to gesture recognition when the inclination of the user is less than the threshold.
  • the gesture recognition unit 140 is an upper and lower axis that approximates a predetermined upper and lower axis set in advance by the user. It is determined that the space is recognized by the axis, and the coordinate axis is determined based on the direction of the user.
  • the standing position and the sitting position may be regarded as a reference state (reference posture) for determining the inclination of the user.
  • the predetermined coordinate axis includes at least one of an axis substantially parallel to the gravity direction (vertical direction in real space) and an axis substantially parallel to the direction perpendicular to the axis (horizontal direction). obtain.
  • the gesture recognition unit 140 calculates the inclination of the user based on the difference between the vertical axis determined based on the orientation of the user detected by the user recognition unit 120 and the vertical axis in the default coordinate axis.
  • the gesture recognition unit 140 can calculate the inclination of the user based on the difference between the vertical axis determined based on the direction of the user and the vertical direction in the real space.
  • the upper and lower axes in the predetermined coordinate axes may be axes substantially parallel to the vertical direction in real space.
  • the direction or inclination of the user may be information acquired by the object 20 or the sensor device 30.
  • the gesture recognition unit 140 can determine the coordinate axes of the space coordinates related to the gesture recognition based on the above information received from the object 20 or the sensor device 30.
  • FIG. 4 shows an example in which the user U makes a gesture on the object 20 in a state where the object 20 which is a robot and the user U stand upright.
  • the gesture recognition unit 140 first determines the vertical axis US of the user based on the gaze direction SD of the user U.
  • the gesture recognition unit 140 may determine the upper and lower axes US based on, for example, a normal vector extending perpendicularly to the vertical direction in the field of view of the user U detected from the gaze direction SD.
  • the gesture recognition unit 140 calculates the difference between the vertical axis in a predetermined coordinate axis, such as the vertical direction in real space, and the vertical axis US of the determined user U.
  • a predetermined coordinate axis such as the vertical direction in real space
  • the gesture recognition unit 140 uses the coordinate axis having the vertical axis US, the horizontal axis, and the longitudinal axis determined based on the user's gaze direction SD, and makes the gesture direction.
  • MD may recognize MD.
  • the above threshold may be, for example, 45 °. If the inclination of the user U is less than 45 °, it is assumed that the user U recognizes space with coordinate axes similar to the predefined coordinate axes.
  • the example in case the target object 20 is located above the user U is shown.
  • the line of sight SD of the user U is directed upward as compared to the horizontal direction in the real space, as illustrated.
  • a threshold for example, 45 °
  • the gesture recognition unit 140 can correctly recognize the user's gesture direction MD based on the coordinate axis by determining the coordinate axis of the space coordinate related to the gesture recognition based on the gaze direction SD.
  • the difference between the user's vertical axis US and the predetermined vertical axis is less than the threshold, and the tilt of the target 20 from the vertical axis TS of the target 20 defined from the top of the head of the target 20 etc. If it is determined that it is less than the threshold, the gesture recognition unit 140 determines that the user U and the object 20 share similar upper and lower axes, and recognizes a gesture using a coordinate axis based on the direction of the user U. Run.
  • the coordinate axis related to the recognition of the gesture can be dynamically determined based on the direction of the user U such as the line of sight direction SD. It is possible to realize high gesture recognition.
  • FIG. 6 shows an example in which the user U makes a gesture on the object 20 while the user U is in the standing position and the object 20 is lying down.
  • the viewing direction SD of the user U is directed downward as compared to the horizontal direction in the real space, as illustrated.
  • the difference between the user's vertical axis US determined based on the gaze direction SD and the vertical axis in the predetermined coordinate axis is less than a threshold (for example, 45 °) as in the case of FIG. 4 and FIG. I understand.
  • the gesture recognition unit may determine coordinate axes of space coordinates related to gesture recognition based on the gaze direction SD, and may recognize the gesture direction MD of the user based on the coordinate axes.
  • FIGS. 7 and 8 are diagrams for explaining determination of coordinate axes of space coordinates related to gesture recognition when the inclination of the user is equal to or greater than a threshold.
  • FIG. 7 shows an example where the user U makes a gesture on the object 20 when the user U is lying down and the object 20 is in the standing position.
  • the difference between the vertical axis US of the user U and the predetermined vertical axis is equal to or more than the threshold as illustrated.
  • the user U performs a gesture in consideration of the vertical axis in the normal state such as the standing state or the vertical axis TS of the object 20.
  • the gesture recognition unit 140 may determine the coordinate axes of the space coordinates related to the gesture recognition based on the tilt of the object 20 when the tilt of the user U is equal to or greater than the threshold. In the example shown in FIG.
  • the gesture recognition unit 140 determines the coordinate axis based on the predetermined vertical axis,
  • the gesture direction MD can be recognized based on the coordinate axes.
  • the gesture recognition unit 140 may recognize the gesture direction MD based on a coordinate axis based on the vertical axis TS of the object 20.
  • the gesture recognition unit 140 According to the above-described function of the gesture recognition unit 140 according to the present embodiment, it becomes possible to correctly recognize a change in the gesture direction that changes according to the inclination of the user U, and a gesture with high accuracy according to the user's state Recognition can be realized.
  • FIG. 8 shows an example of the case where the user U makes a gesture on the object 20 while both the user U and the object 20 are lying down.
  • the user U since the user U has the upper and lower axes US similar to the upper and lower axes TS of the object 20, it is assumed that the user U performs a gesture based on the upper and lower axes US.
  • the gesture recognition unit 140 relates to gesture recognition based on the gaze direction SD of the user U. Coordinate axes of space coordinates may be determined.
  • the gesture recognition unit 140 even when the user U is inclined, in the case where the target 20 has the vertical axis TS similar to the user U, Assuming that the user U performs a gesture based on the vertical axis US of itself, it is possible to recognize the gesture with high accuracy.
  • FIG. 9 is a flowchart showing the flow of the operation of the information processing apparatus 10 according to the present embodiment.
  • the gesture recognition unit 140 determines whether the program related to gesture recognition has ended (S1101). At this time, the gesture recognition unit 140 may perform the above determination, for example, based on the user's input of the program end instruction.
  • the information processing apparatus 10 ends the process relating to the gesture recognition.
  • the gesture recognition unit 140 determines to continue the program (S1101: NO)
  • the user recognition unit 120 detects the direction of the user (S1102).
  • the user recognition unit 120 can detect the direction of the user based on the sensor information collected by the sensor unit 110 or the sensor device 30.
  • the direction of the user according to the present embodiment includes the direction of the user's line of sight.
  • the gesture recognition unit 140 calculates the inclination ⁇ 1 of the user based on the direction of the user detected by the user recognition unit 120 in step S1102 (S1103). At this time, the gesture recognition unit 140 may calculate, for example, the inclination ⁇ 1 of the user from the difference between the vertical axis determined based on the direction of the user and the predetermined vertical axis.
  • the gesture recognition unit 140 determines whether the inclination ⁇ 1 of the user calculated in step S1103 is equal to or greater than a threshold (for example, 45 °) (S1104).
  • the object recognition unit 130 calculates the inclination ⁇ 2 of the object 20 (S1105). At this time, the object recognition unit 130 determines, for example, the inclination ⁇ 2 of the object 20 based on the difference between the reference upper and lower axes determined for each object 20 and the upper and lower axes detected according to the state of the object 20. May be calculated. In addition, the inclination ⁇ 2 of the object 20 may be calculated by the gesture recognition unit 140.
  • the gesture recognition unit 140 determines whether the inclination ⁇ 2 of the object 20 calculated by the object recognition unit 130 in step S1105 is equal to or greater than a threshold (for example, 45 °) (S1106).
  • the gesture recognition unit 140 maintains a predetermined coordinate axis set in advance.
  • gesture recognition The unit 140 determines the coordinate axes of the space coordinates related to gesture recognition based on the direction of the user detected by the user recognition unit 120 in step S1102 (S1107).
  • the gesture recognition unit 140 executes gesture recognition based on the predetermined coordinate axis or the coordinate axis newly determined in step S1107 (S1108).
  • the operation control unit 150 generates a control signal related to the operation control of the object 20 or the like based on the gesture recognized by the gesture recognition unit 140 in step S1108, and causes the operation control based on the gesture to be executed (S1109) .
  • FIG. 10 is a block diagram showing an example of the hardware configuration of the information processing apparatus 10 according to an embodiment of the present disclosure.
  • the information processing apparatus 10 includes, for example, a CPU 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, and an output device 879.
  • Storage 880, drive 881, connection port 882, and communication device 883 Note that the hardware configuration shown here is an example, and some of the components may be omitted. In addition, components other than the components shown here may be further included.
  • the CPU 871 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation or a part of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable recording medium 901.
  • the ROM 872 is a means for storing a program read by the CPU 871, data used for an operation, and the like.
  • the RAM 873 temporarily or permanently stores, for example, a program read by the CPU 871 and various parameters appropriately changed when the program is executed.
  • the CPU 871, the ROM 872, and the RAM 873 are mutually connected via, for example, a host bus 874 capable of high-speed data transmission.
  • host bus 874 is connected to external bus 876, which has a relatively low data transmission speed, via bridge 875, for example.
  • the external bus 876 is connected to various components via an interface 877.
  • Input device 8708 For the input device 878, for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, and the like are used. Furthermore, as the input device 878, a remote controller (hereinafter, remote control) capable of transmitting a control signal using infrared rays or other radio waves may be used.
  • the input device 878 also includes a voice input device such as a microphone.
  • the output device 879 is a display device such as a CRT (Cathode Ray Tube), an LCD, or an organic EL, a speaker, an audio output device such as a headphone, a printer, a mobile phone, or a facsimile. It is a device that can be notified visually or aurally. Also, the output device 879 according to the present disclosure includes various vibration devices capable of outputting haptic stimulation.
  • the storage 880 is a device for storing various data.
  • a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used.
  • the drive 881 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information on the removable recording medium 901, for example.
  • a removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory
  • the removable recording medium 901 is, for example, DVD media, Blu-ray (registered trademark) media, HD DVD media, various semiconductor storage media, and the like.
  • the removable recording medium 901 may be, for example, an IC card equipped with a non-contact IC chip, an electronic device, or the like.
  • connection port 882 is, for example, a port for connecting an externally connected device 902 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • an externally connected device 902 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. is there.
  • the external connection device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, or the like.
  • the communication device 883 is a communication device for connecting to a network.
  • a communication card for wired or wireless LAN Bluetooth (registered trademark) or WUSB (Wireless USB), a router for optical communication, ADSL (Asymmetric Digital) (Subscriber Line) router, or modem for various communications.
  • Bluetooth registered trademark
  • WUSB Wireless USB
  • ADSL Asymmetric Digital
  • Subscriber Line Subscriber Line
  • the information processing apparatus 10 that implements the information processing method according to an embodiment of the present disclosure has a function of recognizing a gesture based on a detected user's operation. At this time, the information processing apparatus 10 has a feature of determining the coordinate axes of the space coordinates related to the recognition of the gesture based on the detected orientation of the user. In addition, the information processing apparatus 10 has a function of generating a control signal related to the operation control of the object 20 of the gesture based on the recognized gesture. According to such a configuration, it is possible to realize more accurate gesture recognition.
  • the information processing apparatus 10 determines the coordinate axes in the three-dimensional space based on the direction of the user.
  • the present technology is not limited to such an example.
  • the information processing apparatus 10 according to an embodiment of the present disclosure may determine coordinate axes in a two-dimensional space, and perform gesture recognition based on the coordinate axes.
  • each step related to the process of the information processing apparatus 10 in the present specification does not necessarily have to be processed in time series in the order described in the flowchart.
  • each step relating to the processing of the information processing apparatus 10 may be processed in an order different from the order described in the flowchart or may be processed in parallel.
  • a gesture recognition unit that recognizes a gesture based on a user's operation;
  • a motion control unit that generates a control signal related to motion control of the operation target based on the gesture based on the recognized gesture; Equipped with The gesture recognition unit determines a coordinate axis of space coordinates related to recognition of the gesture based on an orientation of the user.
  • Information processing device (2)
  • the gesture recognition unit determines a longitudinal axis, a horizontal axis, and a vertical axis in the coordinate axes based on the direction of the user, and recognizes the gesture based on the user's motion in the coordinate axes.
  • the orientation of the user includes the viewing direction of the user,
  • the gesture recognition unit determines the coordinate axis based on a gaze direction of the user.
  • the gaze direction of the user is detected based on eye movement of the user,
  • the gaze direction of the user is detected based on the orientation of the wearing terminal worn by the user on the head,
  • the orientation of the user is detected based on the relative position of the user and the operation object.
  • the information processing apparatus according to any one of the above (1) to (5).
  • the gesture recognition unit determines the coordinate axis based on an inclination of the user with respect to a reference axis.
  • the information processing apparatus determines the coordinate axis based on the orientation of the user when the inclination of the user is less than a threshold.
  • the information processing apparatus according to (7).
  • the gesture recognition unit determines the coordinate axis based on the inclination of the operation target with respect to the reference axis when the inclination of the user is equal to or greater than a threshold.
  • the information processing apparatus according to (7) or (8).
  • the gesture recognition unit determines the coordinate axis based on the inclination of the operation object with respect to the reference axis.
  • the information processing apparatus according to any one of the above (7) to (9).
  • the gesture recognition unit determines the coordinate axis based on the direction of the user when the inclination of the user is equal to or greater than a threshold and the inclination of the operation target with respect to the reference axis is equal to or greater than a threshold.
  • the information processing apparatus according to any one of the above (7) to (10).
  • the gesture recognition unit uses the reference axis as a coordinate axis of space coordinates related to recognition of the gesture. Recognize the gesture by making a decision
  • the information processing apparatus according to any one of the above (7) to (11).
  • the gesture recognition unit calculates the inclination of the user based on the difference between the vertical axis determined based on the direction of the user and the reference axis.
  • the reference axis is an axis substantially parallel to the vertical direction in real space, The information processing apparatus according to (13).
  • a direction detection unit that detects the direction of the user; Further include, The information processing apparatus according to any one of the above (1) to (14).
  • (16) A mounting terminal that the user wears on the head, The information processing apparatus according to any one of the above (1) to (15).
  • the processor recognizes a gesture based on the user's action; Generating a control signal related to motion control of the operation target by the gesture based on the recognized gesture; Including The recognition may include determining coordinate axes of space coordinates related to recognition of the gesture based on an orientation of the user. Further include, Information processing method. (18) Computer, A gesture recognition unit that recognizes a gesture based on a user's operation; A motion control unit that generates a control signal related to motion control of the operation target based on the gesture based on the recognized gesture; Equipped with The gesture recognition unit determines a coordinate axis of space coordinates related to recognition of the gesture based on an orientation of the user. Information processing device, Program to function as.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Le problème décrit par la présente invention est d'obtenir une reconnaissance de geste d'une plus grande précision. La solution proposée par la présente invention porte sur un dispositif de traitement d'informations qui comprend : une unité de reconnaissance de geste qui reconnaît un geste sur la base d'un actionnement d'un utilisateur; et une unité de commande d'actionnement qui produit un signal de commande pour une commande d'actionnement d'un objet devant être actionné par le geste sur la base du geste reconnu, l'unité de reconnaissance de geste déterminant des axes de coordonnées spatiales par rapport à la reconnaissance du geste sur la base d'une direction de l'utilisateur. De plus, l'invention concerne un procédé de traitement d'informations comprenant, par un processeur : la reconnaissance d'un geste sur la base d'un actionnement d'un utilisateur; et la production d'un signal de commande pour une commande d'actionnement d'un objet devant être commandé par le geste sur la base du geste reconnu, la reconnaissance comprenant en outre la détermination d'axes de coordonnées spatiales par rapport à la reconnaissance du geste sur la base d'une direction de l'utilisateur.
PCT/JP2018/026799 2017-09-12 2018-07-18 Dispositif de traitement d'informations, procédé de traitement d'informations et programme WO2019054037A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017174878 2017-09-12
JP2017-174878 2017-09-12

Publications (1)

Publication Number Publication Date
WO2019054037A1 true WO2019054037A1 (fr) 2019-03-21

Family

ID=65722716

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/026799 WO2019054037A1 (fr) 2017-09-12 2018-07-18 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Country Status (1)

Country Link
WO (1) WO2019054037A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111880652A (zh) * 2020-07-16 2020-11-03 北京悉见科技有限公司 用于移动ar对象位置的方法、装置及存储介质
CN116214522A (zh) * 2023-05-05 2023-06-06 中建科技集团有限公司 基于意图识别的机械臂控制方法、系统及相关设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011054118A (ja) * 2009-09-04 2011-03-17 Sony Corp 表示制御装置、表示制御方法及び表示制御プログラム
JP2011175617A (ja) * 2010-01-29 2011-09-08 Shimane Prefecture 画像認識装置および操作判定方法並びにプログラム
WO2014080829A1 (fr) * 2012-11-22 2014-05-30 シャープ株式会社 Dispositif d'entrée de données
JP2015149051A (ja) * 2014-01-08 2015-08-20 富士通株式会社 入力装置、入力方法および入力プログラム
JP2015176253A (ja) * 2014-03-13 2015-10-05 オムロン株式会社 ジェスチャ認識装置およびジェスチャ認識装置の制御方法
JP2016118868A (ja) * 2014-12-19 2016-06-30 任天堂株式会社 情報処理装置、情報処理プログラム、情報処理システム、および情報処理方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011054118A (ja) * 2009-09-04 2011-03-17 Sony Corp 表示制御装置、表示制御方法及び表示制御プログラム
JP2011175617A (ja) * 2010-01-29 2011-09-08 Shimane Prefecture 画像認識装置および操作判定方法並びにプログラム
WO2014080829A1 (fr) * 2012-11-22 2014-05-30 シャープ株式会社 Dispositif d'entrée de données
JP2015149051A (ja) * 2014-01-08 2015-08-20 富士通株式会社 入力装置、入力方法および入力プログラム
JP2015176253A (ja) * 2014-03-13 2015-10-05 オムロン株式会社 ジェスチャ認識装置およびジェスチャ認識装置の制御方法
JP2016118868A (ja) * 2014-12-19 2016-06-30 任天堂株式会社 情報処理装置、情報処理プログラム、情報処理システム、および情報処理方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111880652A (zh) * 2020-07-16 2020-11-03 北京悉见科技有限公司 用于移动ar对象位置的方法、装置及存储介质
CN116214522A (zh) * 2023-05-05 2023-06-06 中建科技集团有限公司 基于意图识别的机械臂控制方法、系统及相关设备
CN116214522B (zh) * 2023-05-05 2023-08-29 中建科技集团有限公司 基于意图识别的机械臂控制方法、系统及相关设备

Similar Documents

Publication Publication Date Title
JP7095602B2 (ja) 情報処理装置、情報処理方法及び記録媒体
JP7092028B2 (ja) 情報処理装置、情報処理方法、及びプログラム
CN113728293B (zh) 用于基于位所的设备控制的系统和界面
US11373650B2 (en) Information processing device and information processing method
US9829708B1 (en) Method and apparatus of wearable eye pointing system
JP2015095802A (ja) 表示制御装置、表示制御方法、およびプログラム
US11151804B2 (en) Information processing device, information processing method, and program
US11886643B2 (en) Information processing apparatus and information processing method
JPWO2015198688A1 (ja) 情報処理装置、情報処理方法及びプログラム
CN108369451B (zh) 信息处理装置、信息处理方法及计算机可读存储介质
JP7059934B2 (ja) 情報処理装置、情報処理方法、及びプログラム
JP6627775B2 (ja) 情報処理装置、情報処理方法およびプログラム
US20230162450A1 (en) Connecting Spatially Distinct Settings
WO2019102680A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
JP2016194763A (ja) 画像処理装置、画像処理プログラム、及び画像処理方法
WO2018146922A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
WO2019054037A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
JP6575518B2 (ja) 表示制御装置、表示制御方法およびプログラム
KR101695695B1 (ko) 이동단말기 및 그 제어방법
WO2019176218A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et support de stockage
US20230226460A1 (en) Information processing device, information processing method, and recording medium
JP2010086367A (ja) 位置情報入力装置、位置情報入力方法、プログラム、情報処理システム、及び電子装置
JP2021140195A (ja) 情報処理装置、情報処理方法およびプログラム
JP2019117994A (ja) 情報処理装置、情報処理方法および情報処理システム
JP2016170584A (ja) 情報処理装置、情報処理方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18855219

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18855219

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP