JP2008135033A - Hand posture operation detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a hand posture operation detector that detects operation of an operator's hand with a simple system configuration, performs analysis based on this operation, and exactly recognizes an operation input intended by the operator. <P>SOLUTION: This hand posture operation detector comprises a hand's back detecting means for detecting the motion and posture of the back of the hand of the operator with respect to the gravity acceleration direction with an acceleration sensor and angular velocity sensor that are attached to the back of the hand of the operator and are arranged on three mutually orthogonal axes, and finger posture detecting means that are attached to a plurality of fingers including the operator's thumb and detects the posture of the operator's fingers. The finger posture detecting means include a first finger posture detecting means formed of an angular velocity sensor for detecting the bending operation and posture in the bending direction of the first joint of the thumb, a second finger posture detecting means formed of the angular velocity sensor for detecting the bending operation and posture from the palm side by the third joint of the thumb, and a third finger posture detecting means formed of the angular velocity sensor for detecting the bending operation and posture in the bending direction of the first joint of a finger other than the thumb. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an operation input device used in computers, multimedia, TV game machines, and the like, and in particular, a three-dimensional (3D) such as a space operation mouse for providing a man-machine interface environment with good operability. The present invention relates to a hand posture motion detection device applied to an input device.

  Furthermore, the present invention relates to a hand posture motion detection device applied to a gesture (spatial motion pattern) input system that realizes an extended input function based on an operation pattern or motion pattern of an operator.

  Conventionally, as a computer input device to which this type of operation input device is applied, an acceleration sensor or the like is added to a conventional two-dimensional mouse, such as a 3D mouse, a spatial operation mouse, etc. For example, Japanese Patent Application Laid-Open No. 7-28591 discloses an operation system in which the function of a mouse is extended so that it can be used as a three-dimensional input device.

  In addition, as a method of measuring the bending degree of each finger or palm by using an optical fiber or a resistor attached to the globe, a data glove is commercially available as a product (Data Grove of VPL Research) and the following patent document 1 and 2.

In addition, a technique for detecting the shape and movement of a hand by an image processing method is disclosed in Patent Document 3 below.
USP 4,937,444 USP 5,097,252 JP-A-9-102046

  However, in the operation system such as the 3D mouse described above, since the operation of the two-dimensional mouse is expanded, the operator has to newly learn a specific operation method for the operation command, This places a new burden on the operator.

  In addition, in the case of a device such as the above-mentioned data glove, many fingers are attached to an optical fiber or a pressure-sensitive resistance element attached to the finger joint and the change in the light joint or the resistance value is observed. Since it is necessary to measure the joint, the device and its control system become complicated.

  Furthermore, in the case of a device such as a data glove, a calibration operation or initialization operation is required at the time of wearing in order to correspond to the shape of an individual hand, and it can be used immediately by anyone. It is not something that can be done, and if the size of the glove is different from the size of the hand, it cannot be used.

  In addition, in the case of such a device, because of the glove shape, there is a sense of restraint at the time of wearing, and since the fingertips are covered, delicate operations and operations using the fingertips are hindered. However, there is a problem that it cannot always be worn on the fingertip.

  In addition, in the detection method of the shape and movement of the hand by the above-described image processing, there is a problem of the camera position for capturing the image and restriction of the captured image depending on the position. In addition to difficulties, there are problems that the processing apparatus and processing system for image processing are also complicated.

  As described above, each conventional operation system has various problems.

  Therefore, the present invention has been made in view of the above circumstances, and can detect the motion of the operator's hand with a simple system configuration and can grasp various motions of the hand using the detection result. An object of the present invention is to provide a hand posture motion detection device that makes it possible to accurately grasp an operation input intended by an operator if analysis is performed based on this motion.

According to the present invention, in order to solve the above problems,
Back detection means for detecting movement and posture of the back of the operator with respect to the gravitational acceleration direction by means of an acceleration sensor and an angular velocity sensor that are mounted on the back of the operator's hand and arranged on three axes that are orthogonal to each other;
A finger posture detecting unit that is attached to a plurality of fingers including the operator's thumb and detects the posture of the operator's finger, and an angular velocity sensor that detects a bending operation and posture of the first joint of the thumb. First finger posture detecting means, second finger posture detecting means comprising an angular velocity sensor for detecting a bending action or posture from the back of the hand to the palm side by the third joint of the thumb, and the first joint of the other finger that is not the thumb A finger posture detecting means including a third finger posture detecting means comprising an angular velocity sensor for detecting a bending operation and posture in the bending direction of
Analyzing the hand gesture by determining the relative position and posture of the back of the hand from the detection information obtained from the back detection means and determining the angular position of the fingertip relative to the back of the hand based on the detection information obtained from the finger posture detection means Thus, there is provided a hand posture motion detecting device characterized in that it can be converted into operation input command information data.

  Therefore, as described above, according to the present invention, it is possible to detect the movement of the operator's hand with a simple system configuration, and to grasp various movements of the hand using the detection result. By performing the analysis based on the above, it is possible to provide a hand posture motion detection device that makes it possible to accurately grasp the operation input intended by the operator.

  Embodiments of the present invention will be described below with reference to the drawings.

"Main configuration of operation input device"
FIG. 1 is a block diagram showing a configuration of a main part of an operation input device according to an embodiment of the present invention.

  That is, the operation input device according to the present invention is connected to the finger posture detecting means 1, the finger shape estimating means 2 connected to the finger posture detecting means 1, and the finger shape estimating means 2 as shown in FIG. The hand shape estimating means 3 is provided.

  Further, as shown in FIG. 1, the operation input device according to the present invention comprises a back detection means 5, a spatial coordinate calculation means 6 connected to the back detection means 5 and the finger shape estimation means 2, and the spatial coordinates. A calculation means 6 and an operation input analysis means 4 connected to the hand shape estimation means 3 are provided.

  In this operation input device, the finger shape estimation means 2, the hand shape estimation means 3, the operation input analysis means 4, and the spatial coordinate calculation means 6 are constituted by a CPU 11 including a microcomputer and its peripheral circuits. And

"Model Description"
FIG. 2 shows a coordinate system and the like defined for each finger and joint when viewed from the back of the hand with the right hand extended forward.

  In FIG. 2, OXYZ indicates fixed space coordinates in this measurement space, and the Z-axis direction is the direction of gravity.

  Normally, the position that was initialized when the system was started is the origin.

  BXYZ represents a hand coordinate system with the back of the hand as the origin, and coincides with the fixed space coordinates OXYZ at the time of initialization.

  The fingertip direction is defined as + X axis, the thumb direction is defined as + Y axis, and the direction perpendicular to the XY plane of the back of the hand and directed upward is defined as + Z axis direction.

  The clockwise rotation directions with respect to the traveling directions (+ axis directions) of the coordinate axes of the XYZ axes are defined as + (roll) Ro11, + (pitch) Pitch, and + (yaw) Yaw directions, respectively.

  Furthermore, an independent coordinate system is similarly defined for each fingertip.

  The fingertip direction is the + X axis, the direction vertically upward from the nail is the + Z axis direction, and the left direction with respect to the XZ plane is the + Y axis.

  Thumb coordinate system TXYZ, index finger coordinate system IXYZ, middle finger coordinate system MXYZ, ring finger coordinate system DXYZ, little finger coordinate system LXYZ (however, each fingertip coordinate system is set to be a relative coordinate system with respect to the back coordinate system PXYZ) The

  FIG. 3 shows a skeleton model of a hand assumed in the present invention.

  A joint portion corresponding to each joint is a one-degree-of-freedom joint that rotates only in one axial direction.

  The operation points of the fingertip tip link are the thumb Tp, the index finger Ip, the middle finger Mp, the ring finger Dp, and the little finger Lp, respectively, and the same coordinate system as that in FIG. Yes, subscript jl. j2 and j3 mean the first, second and third joints (joints).

  In this model, each finger joint other than the thumb has one degree of freedom that rotates only in the Pitch direction, which is rotation about the Y axis in the BXYZ coordinate system, the thumb has two degrees of freedom, and the wrist has three degrees of freedom. .

  FIG. 4 shows a hand-shaped model in the case of the three-finger type operation input device according to this embodiment.

  The detection coordinates for detecting the position and posture of the back of the hand and the angle of each joint with respect to the back of the hand in this model are indicated by symbols.

  The fingertip tip angle is the thumb Y-axis rotation angle (θTy), the thumb X-axis rotation angle (θTx), the index finger Y-axis rotation angle (θI), and the middle finger Y-axis rotation angle (θM) with respect to the back of the hand. As shown in FIG. 5 when the hand is seen from the side, it is defined by relative angle information between adjacent joints.

  The joint angle at this time corresponds to the following equation (1).

θTy = Tj1 + Tj2, θTx = Tj3,
θM = Mj1 + Mj2 + Mj3,
θI = Ij1 + Ij2 + Ij3 (1)
As described above, the posture information of the operation point at the finger tip indicates a composite value of the first, second, and third joint information.

  Further, FIG. 6 shows that the angle information which is the posture information of the fingertip is arranged so that the angular velocity sensor detects the circumference of the fingertip coordinate system XYZ around the Y axis (Pitch direction), and further the position of the back of the hand (Xb, Yb, Zb) An image in which the inclination (Pitch, Roll) is arranged to be detected by the acceleration sensor 12 and the angular velocity sensor 14 is shown.

  In this case, as for the sensors 12 and 14, specifically, the angular velocity sensor 14 uses a vibration gyro-type sensor that detects the angular velocity momentum in the rotation direction of one axis, and in FIG. It shows the state of the mounting direction when using a vibration gyro-type sensor that detects rotation.

  The acceleration sensor 12 as the back detection means 5 is installed by combining three types of semiconductor acceleration sensors.

  FIG. 6 shows a state in which the biaxial acceleration sensors 12x and 12y and the uniaxial acceleration sensor 12z are arranged in combination.

"Detailed circuit configuration"
FIG. 8 is a block diagram showing detailed circuit configurations of the finger posture detecting means 1 and the back detecting means 5 of FIG.

  First, the finger posture detection means 1 includes an angular velocity sensor 7, an analog arithmetic circuit 8, and an analog / digital (A / D) converter 9 in order to detect the posture of the operator's finger tip.

  Here, the angular velocity sensor 7 is attached to the tip of the operator's fingertip, and functions as a sensor element that detects the angular velocity generated by the rotational motion caused by the bending and stretching of the fingertip.

  The angular velocity signal detected by the angular velocity sensor 7 is applied to the analog arithmetic circuit 8.

  In this analog arithmetic circuit 8, the angular velocity signal applied from the angular velocity sensor 7 is amplified and A / D converted so that the angular velocity signal applied from the angular velocity sensor 7 matches the conversion range of the A / D converter 9. Send to section 9.

  In the A / D converter 9, the analog signal from the analog arithmetic circuit 8 is converted into a digital signal.

  The angular velocity signal converted by the A / D converter 9 is applied to the CPU 11.

  At the same time, the analog arithmetic circuit 8 also has a band-pass filter function for cutting unnecessary signals in the low frequency region and high frequency region of the angular velocity signal.

  Further, the finger posture detecting means 1 (I, M, Tx, Ty) are installed in parallel by the number of the two axes of the thumb and the other fingers.

  Next, in order to detect the position and posture of the back of the operator's hand, the back detection means 5 includes three acceleration sensors 12 arranged on three orthogonal axes, a counter circuit 13, and the acceleration sensor 12. The angular velocity sensor 14, the analog arithmetic circuit 15, and the A / D conversion unit 16 are arranged on the same axis.

  The acceleration sensor 12 is attached to the back of the operator's hand, and one detects a signal proportional to the motion acceleration with respect to the direction of movement of the back of the hand.

  Furthermore, the acceleration sensor 12 functions as an inclination sensor that detects gravity acceleration (1G) that changes depending on the inclination of the back of the operator's hand.

  Since the signal detected by the acceleration sensor 12 is PWD (pulse width) modulated and output, the counter circuit 13 counts the duty ratio (H / L ratio of the pulse width) to detect the signal. The acceleration information thus obtained can be converted.

  The acceleration signal converted by the counter circuit 13 is applied to the CPU 11.

  The functions from the angular velocity sensor 14 to the analog arithmetic circuit 15 and the A / D converter 16 are substantially the same circuit configuration and operation as the finger posture detection means 1.

  However, the angular velocity sensor 14 is attached to the back of the operator's hand, and functions as a sensor element that detects the angular velocity generated by the rotational motion caused by the inclination of the back of the operator's hand.

  One of the angular velocity information is used to separate the inclination information obtained from the acceleration sensor 12 and the motion acceleration information.

  As described above, the back detection means 5 includes three acceleration sensors 12x, 12y, 12z that detect three axes orthogonal to detect an acceleration signal and an angular velocity signal in a three-dimensional space, and 3 It is composed of three counters 13x, 13y, 13z, three angular velocity sensors 14x, 14y, 14z, three analog arithmetic circuits 15x, 15y, 15z, and three A / D conversion units 16x, 16y, 16z.

  However, the A / D converters 16x, 16y, 16z, etc. may be configured to perform conversion while switching the input of one converter by a multiplex function, or the A / D conversion used in the finger posture detecting means 1 It can also be shared with the unit 16.

  The CPU 11 is connected to the memory 10 and the interface unit 17.

`` Back posture detection ''
Next, in the CPU 11, calculation processing for the spatial coordinate calculation means 6 to obtain the position / posture of the back of the operator's hand based on the three acceleration signals and angular velocity signals, which are information from the back detection means 5, respectively. Is done.

  This acceleration signal is information in which a motion acceleration component and a gravitational acceleration component are combined.

  When the acceleration sensor 12x for detecting the X-axis direction attached to the back of the operator's hand advances in the X-axis direction with the inclination θ and the motion acceleration (e) as shown in FIG. The acceleration component due to g) is a = g−sin θ (FIG. 9A), the motion acceleration component is b = e−cos θ (FIG. 9B), and the acceleration sensor 12x generates an acceleration signal a + b. Synthesized.

  Therefore, angular displacement information obtained by time-integrating angular velocity information obtained by the angular velocity sensor 14 is used for the calculation of separating the gravitational acceleration component that is the inclination component of the acceleration sensor 12.

  For the inclination in the X-axis direction, the angular velocity signal of the angular velocity sensor 14y that measures the rotational motion around the Y-axis is integrated over time to obtain the displacement of the rotational angle.

  In general, the angular velocity sensor 14 causes a drift due to an offset value in an output signal due to the influence of temperature, vibration, or the like.

  Therefore, an error is accumulated in the angle information obtained by time integration of this signal.

  On the other hand, since the acceleration information of the acceleration sensor 12 is a composite value of the inclination information that is a DC component and the motion acceleration information that is an AC component, the low frequency of the signal of the acceleration sensor 12 is low-passed as described above. No error is accumulated in the tilt angle information obtained by doing so.

  By comparing and referring to the tilt angle information of the acceleration sensor 12 and the angular displacement information of the angular velocity sensor 14, the tilt angle can be obtained with high accuracy.

  Further, as described above, since the gravitational acceleration component that is the tilt angle information of the acceleration sensor 12 can be separated, the motion acceleration information can also be separated.

  Therefore, this motion acceleration information can be used as velocity information representing displacement on three axes by time integration, and can be converted to translational position information by time integration.

"Finger posture angle"
Next, the finger shape estimation means 2 integrates the angular velocity signal, which is information from the finger posture detection means 1, with time, and converts it into angle information.

  However, information on the rotational motion of the wrist of the operator operating in the same direction as the attachment rotation detection direction may be combined with the angular velocity information of the angular velocity sensor 14 attached to the fingertip of the operator at this time.

  Therefore, the finger shape estimation means 2 can obtain only the angle information for the back of the hand at the tip of the fingertip of the operator by subtracting the inclination information obtained from the spatial coordinate calculation means 6 from this angle information.

  As a result, the finger shape estimation means 2 can determine the posture angle of the operator with respect to the back of each hand.

  However, this is a finger movement with the back of the operator's hand facing up or down, and a little more processing is required with the back of the hand facing out or the thumb. State.

  Next, the hand shape estimating means 3 estimates the hand shape of the operator based on the angle information of each finger obtained from the finger shape estimating means 2 and the positional relationship of the back of the hand.

  Further, the operation input analysis unit 4 analyzes the hand gesture of the operator based on the hand shape information obtained from the hand shape estimation unit 3 and the spatial posture and movement information of the hand from the space coordinate calculation unit 6. Then, the operation input command information data is transferred to the interface unit 17 shown in FIG.

"Thumb model"
When the finger posture detection means 1 is applied to the thumb of the operator, the attachment method changes with other fingers.

  As for the thumb, in addition to the function of bending the finger by the first and second joints, there is a function of opposing the palm by the third joint. Two posture detection means are provided.

  That is, the posture detection (θTy) of the first and second joints is detected by the first thumb posture detection means, and the posture detection (θTx) of the third joint is detected by the thumb second posture detection means.

  In this case, the internal configuration and functions are exactly the same as those of the finger posture detecting means 1 except that the mounting position and direction of the angular velocity sensor are slightly different.

  FIG. 7 illustrates the positional relationship between the fingertips when viewed from the tip of the hand toward the wrist.

  FIG. 7 particularly shows the positional relationship between the thumb coordinate system TXYZ and other fingers.

  Tp is a state in which the thumb and other fingers are in the same direction (this is defined as the same direction state), that is, a state in which the palm is open, and Tp ′ is a state in which the thumb and other fingers face each other (this is defined as an opposite state) That is, it shows the state when grasping an object.

  Therefore, the state of the minimum value and the maximum value when the thumb rotates around the X axis (θTx) of the thumb coordinate system TXYZ is shown.

  As shown in FIG. 7, the Y-axis direction of the thumb coordinate system TXYZ at this time substantially coincides with the −Z-axis direction of the back coordinate system BXYZ at Tp, and the −Y-axis of the back coordinate system BXYZ at Tp ′. It is arranged so as to almost coincide with the direction.

  That is, the angular velocity sensor is arranged so as to detect rotation about the Y axis of the thumb coordinate system TXYZ.

  This arrangement is easy to compare with the angular velocity signals of other fingers when the thumb is in the facing state (Tp ′).

  However, conversely, with respect to the bending direction of the thumb by the first and second joints, the axis is shifted from the detection axis of the angular velocity sensor 7 of the finger posture detection means 1.

  This is processed by correcting the angle of the deviation.

"Hand shape estimation"
As described above, in the hand shape estimation unit 3, the calculation processing method inside the CPU 11 differs depending on the posture of the hand and the state of the thumb.

  Based on the acceleration information and angular velocity information detected by the back detection means 5, the spatial coordinate calculation means 6 obtains the position and orientation of the back of the hand, but the inclination information from the acceleration sensor 12 is the Roll with respect to the fixed space coordinate system OXYZ. Only Pitch can be detected.

  That is, with respect to the Yaw direction, only the angular velocity information from the angular velocity sensor 14 can be detected, and therefore the rotation angle in the Yaw direction from which an error has been completely removed cannot be obtained.

  Therefore, the rotation axis that cannot be detected by the acceleration sensor changes depending on the posture of the hand in the back coordinate system BXYZ.

  FIG. 10 shows a combination of sensors to be handled and a flow according to the back state and the thumb state in the hand shape processing, and the wrist movement and finger movement at that time.

  When there is no wrist movement and only the finger tip moves, the processing is based on the angle information calculated by the time integration processing of the angular velocity information of the angular velocity sensor 7 (S29, S30).

  However, in this case, when the thumb is in an opposing state (S21, S27), the comparison processing (S30) between the angular velocity sensor 7 (Ty) of the thumb first posture detecting means and the angular velocity sensor 7 (I, M) of the fingertip (S30). Is possible.

Next, with the movement of the Yaw axis that moves to the left and right of the wrist,
In the state where the back of the hand is facing upward or the palm is facing upward (this is defined as the horizontal state) (Sll), the movement around the Z axis of the back coordinate system BXYZ is the movement of the Yaw axis (S19, S22). ,
In a state where the back of the hand is directed outward (this is defined as a vertical state) (S12), the movement around the Y axis of the back coordinate system BXYZ is the movement in the Yaw direction (S25, S28).

  Since the operation of the Yaw axis cannot be detected by the acceleration sensor 12, the Yaw angle information from the spatial coordinate calculation means 6 may include an error.

  Therefore, the accuracy can be improved by comparing the information of the angular velocity sensor 7 (Tx) of the fingertip and the angular velocity sensors (14y, 14z) for detecting the Yaw axis rotation of the back detector 5 in consideration of the direction at this time ( S31).

  For example, in the horizontal state where the thumb is directed (S13), in this case, the first thumb posture detection means of the thumb coincides with the Z axis of the back coordinate system BXYZ axis due to the left / right movement of the wrist (S19). Thus, a change in the angular velocity information of the angular velocity sensor 7 (Ty) occurs, so that the posture change of the first and second joints of the thumb is detected.

  In other words, it is impossible to distinguish whether the wrist is moved or the thumb is bent.

  In this case, by comparing with the information of the angular velocity sensor 14z that detects the Z-axis rotation of the back detector 5, the angular velocity of the opposite phase is generated if the movement of the wrist, so that the angular velocity can be subtracted.

  Next, when the wrist moves up and down, regardless of the horizontal state (S17, S20) and the vertical state (S23, S26), the inclination obtained by correctly calculating the information from the back detection unit 5 by the space posture calculation unit 6 The fingertip posture can be correctly calculated based on the angle information (S32).

  For example, when the thumb is in the horizontal state (S14), the angular velocity information by the rotational motion of the wrist is detected as the opposite-phase angular velocity signals by the angular velocity sensors 7 of the first thumb posture detecting means 1 and the finger posture detecting means 1. (S20).

  Further, the angular velocity sensor 7 of the index finger and middle finger posture detecting means 1 detects the angular velocity information by the rotational movement of the wrist as an in-phase angular velocity signal.

  By grasping each finger posture angle relatively, the relationship between the fingers is not changed, and the hand shape can be recognized regardless of the movement of the wrist, so that the accuracy can be further increased ( S32).

`` Operation input analysis ''
Next, the operation input analysis unit 3 has several operation input processing modes, and the processing operation and the transfer data information at that time are selected according to the operation mode.

  Usually, the operation input device of the present invention is used in a form of being connected as an input device of a personal computer (host PC) as a host for operation of a computer or the like.

  Further, the mode can be changed even during the processing operation in a certain operation input processing mode.

  The transfer data format for each of the following data processing modes is shown in FIG.

  First, there is a (1) hand shape raw data processing mode.

  The transfer data includes velocity information (Vbx.Vby.Vbz) and posture information (θbx, θby, θbz) that are the spatial posture of the back of the hand, and posture angle information (θTx, θTy, θI, θM) of each finger tip with respect to the back of the hand. is there.

  The operation input analysis means 4 simply converts the information from the hand shape estimation means 3 and the information from the spatial coordinate calculation means 6 into a transfer data format and transfers it.

  On the host PC side, it is possible to receive information from this operation input device and estimate the hand shape internally or perform 3D model computer graphics (CG) display of virtual reality (VR) hand. Become.

  In hand CG processing, it is possible to display a multi-joint 3D model of a hand only by knowing the angle information of the back of the hand and the tip of the hand by using inverse kinematics processing or the like.

  Next, there is (2) hand shape prediction data processing mode.

  The transfer data includes velocity information (Vbx.Vby.Vbz) and posture information (θbx, θby, θbz), which is the spatial posture of the back of the hand, and relative angle information (thumb: θT3, θT2, θT1) of the finger joints from the back of the hand. : ΘI3, θI2, θI 1. Middle finger: θM3, θM2, θM1).

  The operation input analysis unit 4 simply estimates the relative angle of each finger joint from the information from the hand shape estimation unit 3.

  Regarding this estimation, the angle information for each joint is selected by the operation input analysis means 4 for the finger posture information for each finger by the internal finger angle conversion table, and this information is transferred.

  FIG. 11 shows the locus of each joint point with respect to the operation points (IP, Tp) of the index finger and thumb in the grasping operation of the hand.

  The relative angles (θI1.θI2, θI3) of the first, second, and third joints of the index finger with respect to the angle between the operation point (Ip) of the index finger and the back of the hand (finger posture angle information: θI) are tabulated. The thing is FIG.

  Therefore, the finger angle conversion table process is performed based on the table of FIG.

  When the obtained finger posture angle information is between these data values, intermediate data is obtained by interpolation processing, and the angle information of the finger joint is estimated.

  Further, the conversion method is not limited to this, and the inverse kinematics method performed in the CG process can also be used.

  Further, the relationship between the angle data and the finger posture angle information can be converted into an n-dimensional function and estimated by substituting it into the function.

  On the host PC side, it is possible to receive information from this operation input device and perform 3D model CG display in a VR manner without estimating the hand shape inside the PC.

  Even in a PC system that cannot use inverse kinematics processing of an articulated 3D model at the time of CG processing, it is possible to know angle information for each finger of the operator, so that it can be displayed in CG It becomes.

  The two modes can be used for VR-type CG display or the like, and can be used for analyzing hand gestures on the host side.

  However, a command operation input device to a PC does not need to know such hand shape information, and may wish to perform processing using more abstract data.

  Next is (3) 3-finger related information data mode.

  The speed information (Vbx.Vby.VbZ) and posture information (θbx, θby, θbz), which are the spatial posture of the back of the hand of the transfer data, are the same as in the mode, but not the posture angle information of each finger. Status information (Tstat, Istat, Mstat) is attached.

  This is information such as which point each finger is in contact with which finger, or where no finger is touching.

  The code information for each finger uses the link symbol of FIG.

  An example of the hand shape and transfer status at that time is shown in FIG.

  FIG. 13 (a) shows a state in which the tip link portion of the thumb and the middle finger is in contact.

  At this time, the status information is (T-M1, IO, M-T1).

  FIG. 13 (b) shows a state where the thumb and the tip link portion of the index finger are in contact, and the middle finger is in contact with the third link portion of the thumb.

  Therefore, the status information at this time is (T-I1, I-T1.M-T3).

  When receiving information from the operation input device on the host PC side, work commands for combinations of status information for each finger are tabulated in advance, and internal processing is executed when the combination information is input. Can do.

  Further, when the information is input, it is possible to combine information such as that performed in a certain posture at a certain position.

  Thereby, the processing load on the host PC side becomes very light, and the operating side can also operate the PC with a simple hand shape or gesture.

  Next, there is (4) hand shape coded transfer mode.

  In the transfer data, the velocity information (Vbx.Vby.Vbz), which is the spatial posture of the back of the hand, and the posture information (θbx, θby, θbz) are the same as those in the above mode, but the rest is the hand shape code number (Hcode). Is attached.

  This transfers the hand shape code number determined to be the closest to the hand shapes registered in advance.

  Furthermore, the hand shape can be registered by the operator himself.

  For example, Janken Goo can be coded as 1, Choki is 2, Par is 3, and so on.

  The host PC receives information from this operation input device, and when a hand shape code number necessary for internal processing of the PC is input, or when it is performed in a certain posture at a certain position, etc. Internal processing can be performed simply by making a determination.

  Thereby, the processing load on the host PC side becomes very light, and the operation side can operate the PC with a simple hand shape and gesture.

  Next, there is (5) pseudo 2D mouse mode.

  The transfer data is attached with XY-axis position information (Pbx.Pby) obtained by time-integrating speed information, which is translational movement information on the back of the hand, and switch information (B1, B2) corresponding to mouse buttons.

  However, speed information (Vbx.Vby) can be transferred instead of position information.

  This is a pseudo 2D mouse mode in which the function of operating the mouse pointer and buttons is simulated by the position information of the back of the hand and the operation of the finger.

  Pointer operation is performed by moving a hand within a fixed coordinate space OXYZ within a two-dimensional plane (XY plane) of a certain height (Pbz) registered at the time of execution. The movement information is calculated as position information (Pbx, Pby) and transferred.

  At this time, the position information (Pbx, Pby) is not updated by moving the hand from the plane in the Z-axis direction and performing the same XY movement operation there.

  In other words, this corresponds to an operation performed when the 2D mouse does not move the pointer and the position of the mouse is changed, and is the same as the operation performed when the mouse is moved while floating.

  In this operation mode, a virtual plane can be assumed in the space, and the operation can be performed there. However, using an actual desk or knee, place your hand on it and move the pointer using the same operation method as a mouse. It can also be moved.

"Button action"
Regarding the button operation, the left button (B1) of the mouse detects the contact state between the thumb and the index finger, and the right button (B2) detects the contact state between the thumb and the middle finger.

  Further, when a desk or knee is used, it is recognized as a button operation in the same manner by a collision operation with the index finger or the middle finger.

  The detection of these button operations will be described later.

  That is, the switch information (B1, B2) is “1” when each finger is in contact with the thumb or each finger is in contact with a desk or the like, and “0” otherwise. "

  As described above, the host PC side receives information from the operation input device and processes it as 2D mouse data, so that a conventional application operated by the 2D mouse can be used as it is. .

  Even when such an application is operated, it is possible to use the application without having to switch to a new 2D mouse.

  Furthermore, it is not always limited to operations on a desk or the like as in a conventional mouse, and can be used without selecting an operation environment.

  Next, there is (6) pseudo joystick mode.

  The transfer data is attached with position information (Pitch. Roll. Yaw) obtained by full-scale adjustment of the tilt information, which is the spatial posture of the back of the hand, and switch information (B1) corresponding to the button.

  This is a pseudo joystick mode in which a joystick stick operation and a button operation are simulated by the tilt information of the back of the hand and the operation of a finger.

  FIG. 14 shows the hand shape and the state of the operation during this operation.

  This is a state in which the back posture is performed in a vertical state (the Z axis of the fixed coordinate system OXYZ and the Y axis of the back coordinate system BXYZ are in the same direction).

  Left / right tilt by the wrist (X-axis rotation of the back coordinate system BXYZ) or information according to the tilt amount is the left / right operation information of the joystick, and forward / backward tilt by the wrist (Z-axis rotation of the back coordinate system BXYZ) or its tilt Information corresponding to the amount is used as joystick front / rear operation information, and left / right rotation by the wrist (Y-axis rotation of the back coordinate system BXYZ) or information corresponding to the rotation amount is detected as joystick rotation operation information.

  In addition, the button operation is generated in a pseudo manner with the thumb standing and tilted forward.

  That is, the switch information (B1) is “1” or “O” in a state where the thumb is in contact with the third link portion (I3) of the index finger.

  As described above, on the host PC side, by receiving information from the operation input device and processing it as joystick data, a conventional application or the like that has been operated with the joystick can be used as it is. .

  Even when such an application is operated, it is possible to use the application without having to switch to a new joystick.

  Furthermore, it is not always limited to operations on a desk or the like as in a conventional joystick, and can be used without selecting an operation environment.

  It should be noted that the speed information (Vbx.Vby.Vbz) of the transfer data in the processing modes (1) to (4) can be transferred as position information (Pbx, Pby, Pbz) integrated with time. .

"Angle sensor collision detection"
The “collision detection means” of the angle sensor of the finger posture detection means 1 in the button operation process in the pseudo 2D mouse mode will be described.

  The angular velocity signal from the angular velocity sensor 7 can be detected as a characteristic motion signal when the tip of the fingertip collides with something or when an operation is performed to cause the tip of the fingertip to collide.

  One is the action of lifting a finger and then dropping it down, and when this is dropped, the angular velocity becomes zero at the next moment after the angular velocity that is higher than the normal angular velocity.

  FIG. 17 shows a graph of the angular velocity signal (ω) and the angular acceleration signal (ω ′) at this time.

  The graph shown in FIG. 17 shows a state where two consecutive button operations are input.

  FIG. 16 shows the flow of processing of this collision detection means.

  The finger shape estimation means 2 obtains angular acceleration information (ω ′) by differentiating the angular velocity information (ω) of the angular velocity sensor 7, and constantly monitors whether this value exceeds a certain threshold value (Th1).

  The finger shape estimation means 2 detects that the threshold value is exceeded (S1), waits for t1 seconds (S2), and then confirms that the angular velocity (ω) is within a certain threshold value (± Th0). (S3).

  Further, the finger shape estimation means 2 waits for t2 seconds if it can be confirmed (S4), and if it can be confirmed again, at that time, it detects that a collision with the angular velocity sensor 7 has been detected (S5). Inform.

  The operation input analyzing means 4 can determine the button input operation based on the collision detection information simultaneously with the gesture analysis of the finger swinging up and down operation.

  By capturing these phenomena for each finger, it is possible to detect an operation pattern as if the finger collided with something.

  Furthermore, if it is captured in relation to each finger, the contact relationship between the thumb and middle finger or the thumb and forefinger can be known.

  Therefore, it is possible to perform switch input using a desk or knee, virtual switch operation using the thumb, and the like without attaching a device such as a switch to the tip of the finger.

"All collision detection"
In addition to the finger, the “all-collision detection means” based on the same signal processing will be described for the angular velocity sensor 14 used for the back detection means 5.

  When all the angular velocity sensors of the back detecting means 5 and the finger posture detecting means 1 that generate an angular velocity signal by the rotation of the wrist simultaneously detect the collision operation pattern in the same process as the collision detecting means, the information is sent.

  That is, according to this, it is possible to detect a motion of hitting a hand in a state where the hand is fully opened (par) and a state where the hand is held (goo).

  Therefore, by using not only the finger but also the collision detection of the entire hand, a virtual switch based on the combination with the hand shape at that time can be used.

"Initial reset operation"
Next, “initial reset means” will be described.

  As described above, in the angle measurement using the angular velocity sensor, errors due to drift or the like are accumulated.

  Therefore, normally, it cannot be used for a long time unless some countermeasure against drift is taken.

  Further, in the vibration type gyro, an error is likely to occur due to vibration such as an impact, and therefore, the possibility that the error is generated by the operation of the “collision detection unit” increases.

  Hand shape information, which is an initial hand shape in a state where the hand is fully opened (par) and a state where the hand is held (goo), is stored in advance.

  In the first method, when a collision operation is performed with a hand shape such as goo or par, the hand shape data is forced to the initial hand shape closer to the current hand shape based on the detection information of the “all collision detection means”. This is a means for changing the angle information.

  Furthermore, the other method is a method of forcibly changing the angle information to the initial hand shape closer to the current hand shape when the hand shape vertebra 3 continuously detects the combination of the hand shapes. It is.

  That is, when the hand shape is input continuously with Goo Par, the hand shape data is changed to the angle information of the par by the detection of the “initial reset means”.

  Therefore, the error data of the hand shape due to drift or the like can be easily corrected, and can be used without installing a device such as a special sensor for correcting the drift of the angular velocity sensor.

  Furthermore, it is not necessary to perform a calibration operation or an initialization operation at the time of wearing in order to correspond to the shape of an individual person's hand, and anyone can immediately initialize and start using.

  In addition, the reset hand shape stored in advance is not limited to this pattern, and can be handled with a shape that does not easily affect other normal operations.

"Internal hand shape adaptive reset means"
Next, the “adaptive resetting means” of the internal hand shape will be described.

  This function is a function obtained by further developing the “initial reset means”.

  The "initial reset means" operates according to its function only when a signal from the "all collision detection means" is detected with respect to the initial hand shape, but this function is assumed in addition to the initial hand shape. The hand shape of the button operation is also stored in advance.

  For example, a hand shape at the moment of an assumed button operation or reset operation, such as a hand shape at the moment of hitting the desk with the index finger or a hand shape at the time of hitting the thumb and index finger, is stored.

  Next, when a button operation or a reset operation is performed, angle information is forcibly set to the internal memory hand shape value closest to the current hand shape.

  Therefore, the internal information can be reset by the operator's conscious or unconscious action, so that the correction operation can be performed at more timings for problems such as drift of the angular velocity sensor. .

  In addition, since this operation is a closed process inside the operation input device for the data processing modes (3) to (6), the operation does not affect the external host PC or the like. It is possible to continue.

“Attaching the angular velocity sensor”
FIG. 18 shows a state in which the angular velocity sensor 7 of the finger posture detecting means 1 is attached to the tip of each finger.

  FIG. 18A shows a state viewed from the front side when attached to a finger other than the thumb, and FIG. 18B shows a state viewed from the back side.

  As shown in FIG. 18, an angular velocity sensor is fixed to the upper surface of a ring (a thimble shape) with a part of the ring cut, and this can be put on the fingertip.

  Further, the thumb shows two types of attachment states shown in FIGS. 18C and 18D.

  As for the thumb, two angular velocity sensors 7 (Ty) of the first thumb posture detecting means and two angular velocity sensors 7 (Tx) of the second thumb posture detecting means are fixed on the finger-penetrating upper surface. Can do.

  Therefore, since it is only necessary to attach a sensor for detecting posture information of the fingertip to the tip of the fingertip, it can be used even if the size of an individual hand is different.

  In addition, since the fingertips are not covered as in the shape of a grope, it is possible to always use them while wearing them without hindering delicate operations and operations using the fingertips.

  Moreover, it is also possible to adhere | attach directly on the nail | claw of a fingertip with respect to the method of mounting | wearing like this ring or a thimble.

  In that case, there are no more objects covering the fingertips, and more delicate operations and operations using the fingertips can be performed, and it is possible to always use the devices while wearing them.

"Attaching the back detection means"
FIG. 19 shows the overall state of the operation input device attached.

  FIG. 19 particularly shows a state in which the fixtures of the various sensors of the back detector 5 and the angular velocity sensor are attached to the tips of the fingers.

  In FIG. 19A, various sensors 21 of the back detection means 5 and other control means 22 are fixed to the back of the hand of the fingerless glove 20.

  Since the fingerless gloves 20 are used to fix the fingerless gloves 20 to the back of the hand, they may be fixed at several places on the hand, and can be provided with a fixing tool and an adjustment function, and can be made free size. is there.

  Further, FIG. 19B shows another type of fixture.

  Various sensors 21 of the back detection means 5 are fixed to the back of the back fixing part 23, and other control means 22 are fixed to the wristband part 24 connected to the back fixing part 23 and also to the wrist. It is fixed.

  The wristband portion 24 and the back fixing portion 23 are connected by a member 25 made of an elastic deformable body.

  The member 25 made of this elastic deformable body is normally constructed so as to bend inward of the wrist, and always tries to deform in a direction in close contact with the back of the hand even if the wrist moves.

  Furthermore, since the back fixing part 23 extends to both sides of the hand and is in close contact with the palm side, it can be kept fixed even with respect to the left and right movements of the wrist.

  Further, the angular velocity sensor 7 of the finger posture detecting means 1 and the fixture for fixing these control means 22 are connected by a signal line 28 as shown in the figure.

  Therefore, since the sensor for detecting the posture information of the fingertip and the various sensors of the back detection means 5 can be separately mounted, it can be used even if the size of an individual hand is different.

  Further, since the fingertip is not covered like a glove, it is possible to always wear and use it without obstructing delicate work and operation using the fingertip.

"Other detection sensor combinations"
In the present embodiment, the position of the back of the hand by the back detection means 5 and the operation input device by three fingers have been described. However, the present invention is not limited to this embodiment. An input device, or an operation input device using two fingers, the position of the back of the hand and the index finger and the thumb, or an operation input device using all of the position of the back of the hand and the five fingers, and also using both left and right hands by a combination thereof An operation input device can be considered.

  Also, various sensor elements are not limited to the present embodiment, and various sensor elements can be used.

  The interface unit 17 is not described in detail, but can be configured in various forms.

  That is, the interface unit 17 can be a general RS232C standard communication port, or can be configured with a higher-speed interface standard such as USB to communicate with the host PC.

  The interface unit 17 may be a wireless system using light such as radio or IrDA.

  As described above, when the interface unit 17 is wireless, only the information on the finger posture detection unit 1 and the back detection unit 5 is transferred on the transfer unit side, and various computations and operations in the CPU 11 are performed in the reception unit on the host side. By performing calculation / estimation by the estimation means, the configuration on the side worn on the hand can be made smaller and smaller.

  The present invention shown in the embodiment as described above includes the inventions as shown in the following supplementary notes (1) to (20) in addition to the claims 1 to 3 shown in the claims. ing.

Appendix (1)
In an operation input device that performs input according to the shape, spatial position, posture, and movement of the operator's hand,
A finger shape input means for estimating the shape of the entire finger in a bent state (by the first, second, and third joints) by detecting the posture of the tip of the fingertip with respect to the back of the operator's hand is provided. Operation input device.

Appendix (2)
In an operation input device that performs input according to the shape, spatial position, posture, and movement of the operator's hand,
By detecting the posture of the tip of each fingertip with respect to the back of the operator's hand, it is provided with hand shape input means for estimating the finger bending state and estimating the hand shape from the relative relationship of the respective estimated finger shapes. An operation input device characterized by that.

  Further, according to the inventions of the supplementary notes (1) and (2), since the movement performed by the person using the hand and the finger is operated with the fingertip tip as an action point, the posture of the fingertip tip with respect to the back of the hand is changed. By detecting, it becomes possible to detect many operation shapes by hand, so that it is possible to estimate the finger shape and further the hand shape with very few measurement points, and the configuration of the device is also simplified. It becomes possible.

Appendix (3)
In an operation input device that performs input according to the shape, spatial position, posture, and movement of the operator's hand,
Back detection means for detecting the spatial movement and posture of the back of the operator's hand;
Spatial coordinate calculation means for obtaining a position and orientation in a three-dimensional space from information on the back detection means;
Finger posture detection means for detecting the bending action and posture of the tip of the first joint of each finger;
From the information detected by the spatial coordinate calculation means and the finger posture detection means, a finger shape estimation means for estimating the shape of the entire finger by estimating the bending angle of the finger tip with respect to the back of the hand;
Hand shape estimating means for estimating the shape of the hand based on the information of the shape of each finger as a whole from the finger shape estimating means;
An operation input device comprising operation input analysis means for analyzing operation input information from information of the space coordinate calculation means and the hand shape estimation means and determining a command for input.

  According to the invention of the supplementary note (3), it is possible to provide an operation input device that performs input based on the shape of the hand, the spatial position / posture, and the movement. Without forcing the burden of learning, daily operation actions and behaviors with hands and fingers can be used as operation commands as they are.

Appendix (4)
The finger posture detection means includes
A thumb first posture detecting means for detecting a bending action or posture of the first and second joints of the thumb in the bending direction;
The operation input device according to appendix (3), characterized in that the operation input device comprises a second thumb posture detecting means for detecting a bending action or posture of the thumb from the back of the hand to the palm side by third indirect.

  Further, according to the invention of the above supplementary note (4), it is possible to detect the movement of the thumb with a simple configuration without adding any new means by mounting two same movements as the configuration of the finger posture detecting means. It becomes possible to do.

Appendix (5)
The operation input device according to appendix (3), wherein the finger posture detecting means is constituted by an angular velocity sensor as the detecting means.

Appendix (6)
Note that spatial translational motion and rotational posture of the back of the hand are detected by three acceleration sensors respectively arranged on three orthogonal axes and three angular velocity sensors as detection means in the back detection means. (3) The operation input device.

  And according to invention of such additional remarks (5) and (6), size reduction of an apparatus can be achieved by utilizing small elements, such as an angular velocity sensor and an acceleration sensor.

Appendix (7)
Note that the finger posture detecting means is attached to only one finger other than the thumb, the thumb and the other one finger, the thumb and the other two fingers, or the thumb and the other plural fingers. (3) The operation input device of (4).

  And according to invention of such additional remark (7), many combinations are possible according to the finger to be used, and operation input device can be changed only by changing the mounting position or changing the number of mounting configurations. It is possible to easily change the configuration.

Appendix (8)
It is characterized by comprising joint angle estimating means for estimating the relative bending angle state of each of the first, second, and third joints of the finger only by detecting the bending angle of the finger tip with respect to the back of the hand. The operation input device according to appendix (4).

  And according to invention of such additional remark (8), each finger joint with respect to a finger-tip tip angle can be estimated by a simple process using a conversion table for converting each finger joint angle and table interpolation.

Appendix (9)
The operation input device according to appendices (3) and (4), wherein the finger posture detecting means is attached to an upper surface of a finger-like or ring-like ring attached to a fingertip ahead of the first joint.

Appendix (10)
The operation input device according to any one of appendices (3) and (4), wherein the finger posture detection means is attached to a fingernail so that it can be adhered.

Appendix (11)
The operation input device according to appendices (3) and (4), wherein the back detection means is attached to the back of a glove without a fingertip.

  And according to invention of such additional notes (9), (10), (11), since the sensor for detecting the posture information of a fingertip and the various sensors of the back detection means can be separately mounted. In addition, it can be used even if the individual's hand size is different, and since it does not cover the fingertip etc. like a glove, it does not hinder delicate work and operation using the fingertip So you can always wear it.

Appendix (12)
The operation input analysis means includes
The operation input device according to appendices (3) and (4), further comprising: a collision detection unit that analyzes and determines a collision operation pattern with respect to a desk, knee, fingers, or the like on a finger equipped with the finger posture detection unit.

Appendix (13)
The operation input analysis means includes
The operation input device according to additional notes (3) and (4), further comprising all collision detection means for analyzing and determining a collision operation pattern for a finger, a hand, and the like at the same time against a desk, floor, or knee.

  Further, according to the inventions of the supplementary notes (12) and (13), it is not necessary to newly attach an element such as a switch, and a virtual switch that can be used at any place using a desk, a knee or the like is configured, and a switch input The operation can be performed.

Appendix (14)
When initial hand shape information of a state where the hand is fully opened (par) and a state where the hand is held (goo) is stored in advance, and the detection by the full collision detection means is performed,
The operation input device according to appendix (13), further comprising initial reset means for forcibly changing the angle information to an initial hand shape close to the current hand shape.

Appendix (15)
Some initial hand shape information is stored in advance, and when detection by the collision detection means is performed,
The operation input device according to appendix (14), further comprising adaptive reset means for changing the current hand shape information to the nearest hand shape.

Appendix (16)
The initial hand shape information of the hand fully opened (par) state and the hand grasped (goo) state is stored in advance, and when these two hand shape data are detected in succession, the current hand shape (13) The operation input device according to (13), further comprising initial reset means for forcibly changing angle information to an initial hand shape close to.

  According to the inventions of the supplementary notes (14), (15), and (16), it is possible to easily correct the error data of the hand shape due to drift or the like, and special correction for correcting the drift of the angular velocity sensor. It is possible to use without installing a device such as a simple sensor, and there is no need to perform special operations such as calibration operation and initialization operation at the time of wearing to correspond to the shape of an individual hand. Anyone can immediately initialize and start using it, and it is possible to correct internal information by the operator's consciousness or unconscious action. On the other hand, the correction operation can be performed with a larger number of timeminders.

Appendix (17)
In the hand movement of pseudo two-dimensional mouse operation in space,
By setting a certain height and its range,
Within this height range,
Update the information according to the movement speed of the hand before and after, or the amount of movement as mouse movement information of one axis,
In addition, the information corresponding to the left and right movement speed or movement amount is updated as the mouse movement information of the other axis,
Outside the height range,
Pseudo 2D mouse operation means that does not update the mouse movement information for hand movements and that can perform two-dimensional mouse operation in a pseudo manner by using the detection information by the collision detection means of the index finger and middle finger as two button information. The operation input device according to any one of additions (1) to (4), comprising:

Appendix (18)
In the hand movement of pseudo joystick operation in space,
Left and right tilt by the wrist, or information according to the tilt amount as joystick left and right operation information,
Information based on the wrist tilt forward or backward or the amount of tilt is used as the joystick forward / backward operation information.
Rotation information on the left and right by the wrist, or information according to the amount of rotation is joystick rotation operation information,
The operation input device according to any one of appendices (1) to (4), comprising pseudo joystick operation means for performing pseudo joystick operation.

  According to the inventions of the supplementary notes (17) and (18), since a conventional application operated by a device such as a 2D mouse or a joystick can be used as it is, when operating the application It can be used without having to reconnect or change the device, and it is always limited to operations on a desk, such as a conventional mouse or joystick. It can be used without selecting an operating environment.

Appendix (19)
In the operation input analysis means, some hand shape information and its hand shape code are stored in advance, and when the same hand shape as the stored hand shape information is detected,
The operation input device according to appendices (3) and (4), comprising hand shape code processing means for transferring the hand shape code information.

Appendix (20)
In the operation input praying means, the finger identification code information and the link part code information are transferred as contact-related information between each finger indicating whether or not each finger is in contact with the other finger. The operation input device according to appendices (3) and (4), characterized by comprising code processing.

  Further, according to the inventions of the supplementary notes (19) and (20), angle information such as a finger joint as in the hand shape input device is essentially unnecessary data as the operation input device, and the daily information by the hand or finger is used. Can be used as an operation command as it is, and the processing load on the host PC side is very light, and the operation side is also easy. It becomes possible to operate the PC with a simple hand shape and gesture.

FIG. 1 is a block diagram showing a configuration of a main part of an operation input device according to an embodiment of the present invention. FIG. 2 is a diagram showing a coordinate system and the like defined for each finger, joint, and the like when viewed from the back of the hand with the right hand extended forward. FIG. 3 is a diagram showing a hand skeleton model assumed in the present invention. FIG. 4 is a diagram illustrating a hand-shaped model in the case of the three-fingered operation input device according to the present embodiment. FIG. 5 shows the tip angle of the fingertip as the thumb Y-axis rotation angle (θTy), thumb X-axis rotation angle (θTx), index finger Y-axis rotation angle (θI), and middle finger Y-axis rotation angle (θM) with respect to the back of the hand, It is a figure which shows that each joint information is defined by the relative angle information between adjacent joints when a hand is seen from the side. FIG. 6 shows that the angle information, which is posture information of the fingertip, is arranged so that the angular velocity sensor detects the circumference of the fingertip coordinate system XYZ around the Y axis (Pitch direction), and further, the position of the back of the hand (Xb, Yb, Zb), It is a figure which shows the image arrange | positioned so that inclination (Pitch, Roll) may be detected by the acceleration sensor 12 and the angular velocity sensor 14. FIG. FIG. 7 is a diagram illustrating the positional relationship between the fingertips when viewed from the tip of the hand toward the wrist. FIG. 8 is a block diagram showing detailed circuit configurations of the finger posture detecting means 1 and the back detecting means 5 of FIG. FIG. 9 shows the gravitational acceleration (g) of the earth when the acceleration sensor 12x for detecting the X-axis direction attached to the back of the operator's hand advances in the X-axis direction with an inclination θ and a motion acceleration (e). The acceleration component by is a = g−sin θ ((a) in FIG. 9), and the motion acceleration component is b = e−cos θ ((b) in FIG. 9), and the acceleration sensor 12x synthesizes an acceleration signal of a + b. FIG. FIG. 10 is a diagram illustrating a combination of sensors to be handled and a flow thereof according to the back state and the thumb state in the hand shape processing, and the wrist movement and finger movement at that time. FIG. 11 is a diagram illustrating the trajectories of the respective joint points with respect to the operation points (IP, Tp) of the index finger and the thumb in the grasping operation of the hand. FIG. 12 shows relative angles (θI1.θI2, θI3) of the first, second, and third joints of the index finger with respect to the angle between the operation point (Ip) of the index finger and the back of the hand (finger posture angle information: θI). It is a figure shown as a table. FIG. 13 shows an example of hand shape and transfer status. FIG. 13 (a) shows a state where the tip link portion of the thumb and the middle finger is in contact, and FIG. 13 (b) shows a state of the thumb and index finger. It is a figure which shows the state which the front-end | tip link part is contacting and the middle finger is contacting the 3rd link part of thumb. FIG. 14 is a diagram showing a hand shape and a state of the operation in the pseudo joystick mode in which the joystick stick operation and the button operation are simulated by the back information of the back of the hand and the finger operation. FIG. 15 is a diagram showing a transfer data format for each data processing mode. FIG. 16 is a diagram illustrating a process flow of the collision detection unit. FIG. 17 shows an operation in which a finger is lifted and immediately dropped, and an angular velocity signal (ω) and an angular acceleration signal (when the angular velocity becomes zero at the next moment after the angular velocity that is faster than the normal angular velocity at the time of dropping is expressed. It is a figure which shows the graph with (omega '). FIG. 18 shows a state in which the angular velocity sensor 7 of the finger posture detecting means 1 is attached to the tip of each finger. FIG. 18 (a) shows a state seen from the front side in a state attached to a finger other than the thumb. FIG. 18 (b) shows a state seen from the back side, and FIG. 18 (c) and FIG. 18 (d) show two types of attachment states on the thumb. FIG. 19 shows a state in which the fixtures of the various sensors of the back detection means 5 and the angular velocity sensors are attached to the tips of the respective fingers, in order to show the overall state of the operation input device attached. 19 (a) shows a state in which various sensors 21 of the back detection means 5 and other control means 22 are fixed to the back of the hand of the fingerless glove 20, and FIG. 19 (b) shows another type of fixing. As a tool, various sensors 21 of the back detection means 5 are fixed to the back of the back of the back fixing part 23, and other control means 22 are fixed to the wristband part 24 connected to the back fixing part 23, and the wrist. It is a figure which shows a mode that it is also fixed to.

Explanation of symbols

1 ... finger posture detection means,
2 ... finger shape estimation means,
3 ... hand shape estimation means,
4 ... operation input analysis means,
5 ... Back detection means,
6 ... spatial coordinate calculation means,
7 ... Angular velocity sensor,
8: Analog arithmetic circuit,
9: Analog / digital (A / D) converter,
10 ... Memory,
11 ... CPU,
12 (12x, 12y, 12z) ... acceleration sensor,
13 (13x, 13y, 13z) ... counter circuit,
14 (14x, 14y, 14z) ... angular velocity sensor,
15 (15x, 15y, 15z) ... analog arithmetic circuit,
16 (16x, 16y, 16z) ... A / D converter,
17: Interface section.

Claims (1)

Back detection means for detecting movement and posture of the back of the operator with respect to the gravitational acceleration direction by means of an acceleration sensor and an angular velocity sensor that are mounted on the back of the operator's hand and arranged on three axes that are orthogonal to each other;
A finger posture detecting unit that is attached to a plurality of fingers including the operator's thumb and detects the posture of the operator's finger, and an angular velocity sensor that detects a bending operation and posture of the first joint of the thumb. First finger posture detecting means, second finger posture detecting means comprising an angular velocity sensor for detecting a bending action or posture from the back of the hand to the palm side by the third joint of the thumb, and the first joint of the other finger that is not the thumb A finger posture detecting means including a third finger posture detecting means comprising an angular velocity sensor for detecting a bending operation and posture in a bending direction of
Have
Analyzing the hand gesture by determining the relative position and posture of the back of the hand from the detection information obtained from the back detection means and determining the angular position of the fingertip relative to the back of the hand based on the detection information obtained from the finger posture detection means A hand posture motion detecting device characterized in that it can be converted into operation input command information data.

Images