Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/20—Movements or behaviour, e.g. gesture recognition
  • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/107—Static hand or arm
  • G06V40/113—Recognition of static hand signs
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30196—Human being; Person
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

• the present invention relates to a method for the recognition of gestures of a human body and a recognition device for the detection of gestures of a human body.
• gestures of human bodies can be detected.
• systems on the market that are able to determine the positions of individual body parts or individual limbs relative to each other. From this relative position, e.g. Forearm to upper arm, gestures and thus a gesture control can be derived.
• Known methods are used, for example, to perform the control of computer games or television sets.
• a point cloud is usually generated by the depth camera from which the current position of the respective body parts and thus the correlation of the body parts to each other can be calculated by means of calculation algorithms. For all points in time, the entire point cloud must be processed according to this evaluation method.
• a disadvantage of known methods is that a relatively high computation effort is necessary at each point in time of the method. Thus, a movement of the body after each movement, a complete point cloud must be re-recorded and re-evaluated. This requires in particular in the distinction of small body parts to individual limb immense computing costs, which is usually not available. Accordingly, known methods are limited to recognizing relatively coarse gestures, e.g. the
• Fine movements such. B. different gestures of a hand, especially gestures by different finger positions are only with
• CONFIRMATION COPY Disproportionately complex computing use by known methods solvable. This drives the costs of implementing such methods at heights that are not economically viable. In addition, in such a case, very fine-resolution depth cameras are necessary in order to map the individual limbs at the necessary speed in the cloud of points from one another. This also greatly increases the cost required to carry out a corresponding method.
• a method according to the invention serves to detect gestures of a human body by means of a depth camera device, comprising the following steps: a) generating a cloud of points through the depth camera device into a
• a method according to the invention serves to recognize also fine gestures, in particular of individual limbs, such as the fingers of a hand of a human body. Basically, however, the method is applicable to the human body as a whole, that is, to any limb.
• limbs can be defined in particular as individual movable bone elements of the human body. These can be z. B. by the
• a method begins with an initialization.
• Depth camera device is preferably equipped with at least one depth camera and can generate in this way a three-dimensional point cloud.
• this point cloud thus arises as an initial image.
• the evaluation of this initial image takes place with respect to a recognition of
• the entire cloud of points or only partial areas of this cloud of points can be evaluated in detail.
• the evaluation is carried out only in the region of the body parts, which includes the body members necessary for the gestures. If, for example, a human body is detected and a gesture of the fingers is sought, the detailed evaluation of the initial image takes place only in the area of the hand in order to carry out the recognition of the individual phalanxia as limbs of the body.
• the setting of the pivot point is done with reference to the respectively recognized
• the individual fingers of the hand are one
• Human body defined by individual limbs in the form of phalanges. Between the individual limbs human joints are provided which have one or more rotational degrees of freedom.
• hinge points with exactly one defined degree of freedom. If the real joint between two limb members on the human body is an embodiment with two or more rotational degrees of freedom, it is of course also possible to set two or more joint points, each with a defined degree of freedom. Thus, even complex joints of a body, which have two or more rotational degrees of freedom, can be simulated according to the invention.
• By setting the hinge point results in an initial rotation angle, which in a defined manner, the positioning of the two adjacent
• each articulation point is determined in the respective coordinate system, belonging to the respective articulation point.
• Hinge point which is set in the method according to the invention, has its own coordinate system. Because of that, individual limbs
• a plurality of hinge points is preferably used and set. This also results in a large number of rotation angles for this multiplicity of articulation points.
• These can be used for a better overview, e.g. be specified or stored in a single-column and multi-cell vector. This single-columned and multicellular vector thus reproduces the relative position of the individual limbs in a defined and above all unambiguous manner.
• each recognized limb and a hinge point must be set.
• a recognition of all body limbs of a body take place, wherein only for the two hands or only for one hand the joint points are set for the further process steps.
• a selection is made from all recognized limbs. This selection can be a subset or also include all recognized limbs. At least, however, a single hinge point is performed on the at least one recognized limb.
• Rotation angle specification is also formed, for example, as a single-column, multi-cell vector.
• a line-by-line comparison can be made as to whether there is a match or a substantial match between or a sufficient, in particular predefined, proximity to the determined rotation angles and this rotation angle specification. If this is the case, the real movement position of the respective limbs of the human body corresponds to the gesture correlated with this rotation angle specification.
• the rotation angle specification can have both specific and one-time values, as well as value ranges.
• a particularly narrow or broad training of the respective gesture is to be carried out, accordingly, a particularly narrow or broad training of
• Rotation angle specification can be provided as a rotation angle range.
• a plurality of different rotation angle specifications are stored in a gesture-specific manner.
• the steps of comparing and recognizing the rotation angle and the gesture, respectively are performed for all the gesture-specific storage data of the rotation angle specifications, eg, sequentially or in parallel.
• the comparison is performed until a sufficient correlation in the form of coincidence or substantially coincidence between the determined rotation angle and the rotation angle specification is recognized. This can be done an assignment of the specific rotation angle to the specific for this rotation angle specification gesture.
• the recognition task is reduced completely to the comparison of rotation angle with rotation angle specification, which can be designed particularly cost-effective and simple in terms of the necessary computational effort.
• the one-line comparison of a multicell single-column vector with a corresponding rotation angle specification is a very simple arithmetic operation, which requires neither a complex arithmetic unit nor a particularly large amount of time.
• a further advantage of the method according to the invention is achieved in that a reduction of the actual human body from the point cloud to a corresponding model of the human body with regard to points of articulation and limbs could take place. This requires for the comparison between
• An inventive method is used in particular in medical technology, eg for gesture control of medical apparatus.
• medical technology eg for gesture control of medical apparatus.
• there it is advantageous since now a large number of commands can be controlled by a wide variety of finger gestures.
• Gesture control does not affect the sterility of the user, especially his hand.
• the explained and described advantages are accordingly particularly advantageously achievable in the medical field in medical apparatuses for their control.
• the method of the present invention may be used for a classical one
• Gesture recognition when controlling a machine or even a vehicle can be performed by a method according to the invention by means of gesture control.
• operator actions in a vehicle can be performed by a method according to the invention by means of gesture control.
• control of actions of technical devices such as televisions, computers, mobile phones or tablet PCs can also be an inventive method for
• Gesture recognition be used. Furthermore, in the medical environment in this way, a very accurate position detection of the individual limbs can enable a deployment in the field of teleoperation. Even a basic interaction between man and machine or human and robot is in the context of the present invention, a possible application.
• a method according to the invention can be further developed in such a way that steps d) to h) are carried out in a repetitive manner, wherein the following picture of the preceding pass will be set as an initial picture for the following pass.
• a tracking or a follow-up procedure is provided, so to speak, which makes it possible to carry out essentially continuous step-by-step monitoring with regard to a change in the gestures.
• the rotation angle specification comprises a predetermined rotation angle range, wherein it is compared whether the specific rotation angle is within the rotation angle range.
• the rotation angle specification can be a single-column multicellular vector. For each individual line can be a specific and a one-to-one
• Rotation angle be used as rotation angle specification. However, it is preferred if here a rotation angle range is indicated in each line, which is e.g. between 10 ° and 25 ° is specifically designed for a gesture.
• the width of the respective rotation angle range is preferably designed to be adjustable and in particular likewise star specific. So can by particularly close
• Rotation angle ranges a clean and defined demarcation of very similar finger gestures from each other. If only a small number of gestures can be distinguished from one another in a method according to the invention, a particularly wide range of rotational angles can also be used for greater freedom in the actual detection. The degree of misrecognition or the delimitation of similar gestures can accordingly be represented with particular preference over the rotation angle range and its width.
• the specificity is given by the sum of all the rotation angle specifications in such a multicellular vector. Depending on how wide the rotation angle range is executed, even poorly executed gestures can be detected. It is also possible to train gestures here. For this purpose, so-called training sets can be included, which are subsequently classified. On the basis of this training data, the redness angle specifications can be defined implicitly, so to speak.
• steps a) and b) are carried out with a defined gesture of the relevant limb, in particular at least twice in succession with different gestures. It is, so to speak, around . a defined initialization of the present method.
• a defined gesture for the Initialization step can provide.
• Even a defined gesture sequence, such as spreading all fingers and closing them to a fist can provide a double initialization step as two consecutive different gestures.
• the inventive method works even without the use of defined gestures for the initialization.
• these initialization defined gestures may improve the initial setting of the fulcrums for accuracy.
• the possibility of initialization described here can be used both to start a method according to the invention and in between. This close the
• the two loops can be as often as you like
• the first loop will pass twice before the procedure enters the second , loop. Since the second loop describes the recognition of the gesture and thus preferably continuous monitoring, this second loop is preferably repeated without a fixed final value. Also, a maximum repetition number of the second loop may trigger an automatic calibration through the first loop, for example after every 1000 passes through the second loop.
• this is carried out for at least two points of articulation, in particular for a plurality of articulation points, wherein the articulation points jointly
• inventive method are based. This makes it possible to use, for example, robotics rules in the reverse manner.
• robotics rules e.g. known transformation rules between the individual translational and / or rotationally movable coordinate systems of
• all points of the point cloud associated with the at least one pivot point are recognized during the evaluation of the subsequent image and the center of gravity of these points is set as the new pivot point.
• the actual positioning of the hinge point depends, among other things, on the resolution of the depth camera device. With relatively coarse depth camera devices, it is not possible to assign a single specific point to the respective hinge point. Thus, all points which are recognized as belonging to the respective pivot point are defined for this and the focus of these points as new
• Pivot point set This helps to allow an explicit and as accurate as possible positioning of the new pivot point even at lower-cost and less fine resolutions of the depth camera.
• the human hand offers a very large number of different gestures due to the large number of limbs and the large number of actually existing finger joints.
• the human hand forms a particularly simple and above all very variable usable medium to recognize different gestures.
• a method according to the invention in accordance with the preceding paragraph can be further developed such that the same number of articulation points and limbs forms a hand model for all fingers of the hand.
• This hand model is therefore in this case the limb model, as has already been explained.
• the thumb takes from a medical point of view a special position on the hand. That's it, the proximal joint of the thumb is not an actual finger joint in the medical sense, but represents a mobility of the thumb. To illustrate this mobility in hand model according to the invention, one or more points of joint can also be set here. However, if the gestural variants of this mobility of the thumb are not needed, then the corresponding
• blind joint point are set. This preserves the agreement with the number of hinge points for all fingers. However, the computational effort is reduced, at least for gesture recognition on the thumb. In other words, the relative movement and / or the position change for this pivot point is set to zero.
• Mobility of the joints are taken into account.
• a determination of the orientation of the recognized hand in “left” or “right” and in “palm view” or “back of hand” is made in addition to a sequence of passes of a method according to the invention.
• Another advantage may be, if in a method according to the invention for the back of the hand, the carpal and / or the arm stump or the
• Palm three pivot points are set. Since, as has already been explained, for example, by a plurality of points of the point cloud over the center of gravity, a definition of the location of the respective hinge point can be done in certain manual positions and at a single pivot point for the
• Back of the hand may be mispositioned. In other words, for the back of the hand would be less and / or closer to each other
• Points of the point cloud to be closed on the associated center of gravity.
• the points cloud points would contract around the center of gravity of the back of the hand, providing a worse geometric mean for that center of gravity.
• the position of the joint point thus set would be inaccurate, so that at certain
• three pivot points for the back of the hand are now preferably set.
• a relatively good result for the positioning of the back of the hand can still be achieved in this way.
• the back of the hand is pulled into the stump of the arm.
• the entire back of the hand is clamped by the three pivot points on the back of the hand and / or by the further pivot point of this embodiment, so that an undesirable mispositioning or contraction of the back of the hand to a single pivot point can be avoided.
• the length of the limb between the two articulation points has a predetermined value. The individual limbs are thus reproduced in the representation of the hand model or the limb model by their length, so to speak as a framework.
• the individual hinge points are determined by the length of the respective rod
• the length may also be adjustable, so that e.g.
• the joint between the metacarpal bone and the proximal finger bone is a joint in the human body with two rotational degrees of freedom.
• the rotation angles of at least two articulation points are stored in a single-column vector and compared with the rotation angle specification, in the form of a single-column vector, line by line.
• the rotation angle default vector is tag specific. Accordingly, for each desired gesture to be detected, a rotation angle specification and thus a gesture-specific rotation angle default vector are provided.
• a further advantage can be achieved if, in a method according to the invention, the rotation angle of the initial image for the subsequent image is adopted in the case of impossible detection of a limb and / or a pivot point in a subsequent image.
• the method can be performed further in the same way and with only minor errors. This is another advantage which makes the big difference too
• a compensation can e.g. by a corresponding broadening in the width of rotation angle ranges in the rotation angle specification.
• a recognition device for the recognition of gestures of a human body comprising a depth camera device and a control unit.
• Recognition device is characterized in that the control unit is designed for carrying out a method according to the invention.
• a recognition device brings the same advantages as described in detail with reference to FIG. 1
• FIG. 3 shows the hand of FIG. 2 with a limb model arranged therein
• FIG. 5 shows three body members in a first gesture position
• FIG. 6 shows the body members of FIG. 5 in a second gesture position
• Fig. 7 shows the timing of several embodiments of a
• Fig. 9 shows an embodiment of an inventive
• the point cloud 20 is shown here for the sake of clarity only for the outermost distal finger joint as a body member 12. In the same way, the recognition of all body members 12 and preferably also of the
• the individual articulation points 14 can be set. These correlate with the respective actual joint between two limbs 12. The distance between two
• adjacent hinge points 14 is preferably given as a length 13 of the respective body member 12 and specific body member. As can also be seen from FIG. 3, an equal number of all fingers 18 has been
• FIGS. 5 and 6 show schematically how the gesture recognition can take place.
• a separate coordinate system is defined for each hinge point 14, so that a corresponding rotation angle ⁇ for each hinge point 14 can be specifically recognized for this body joint 12. If there is a movement, e.g. by curving the finger, as is done from Fig. 5 to Fig. 6, so change
• FIG. 8 also shows a possible comparison with a rotation angle specification RV, which is likewise designed here as a vector with rotation angle specification ranges. In this embodiment the two vectors agree that the gesture can be recognized as being present.
• the rotation angle default RV is
• FIG. 7 it can be seen that at the beginning of the method at the first time t1, the initialization, that is to say the implementation, as has been described from FIG. 1 to FIG. 2, takes place. Subsequently, a comparison with the initial image IB can take place in a subsequent image FB at a second time t2. For the subsequent steps, the subsequent image FB from the first pass is set as the initial image IB of the second pass and, accordingly, the method can be expanded as desired.
• FIG 9 shows schematically a recognition device 100 according to the invention.
• a depth camera device 1 0 with at least one
• This depth camera device 1 10 is in
• the human body 10 in this case the hand 16, is located in the detection range of the depth camera apparatus 1 10.
• the above explanation of the embodiment describes the present invention solely by way of example. Of course, individual features of the embodiments, if technically feasible, can be combined freely with one another, without departing from the scope of the present invention.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Health & Medical Sciences (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Health & Medical Sciences (AREA)
• Psychiatry (AREA)
• Social Psychology (AREA)
• User Interface Of Digital Computer (AREA)
• Image Analysis (AREA)
• Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=51753180

Family Applications (1)

Country Status (5)

Families Citing this family (11)

Family Cites Families (10)

• 2013
  • 2013-10-19 DE DE201310017425 patent/DE102013017425A1/de not_active Ceased
• 2014
  • 2014-10-17 US US15/030,153 patent/US20160247016A1/en not_active Abandoned
  • 2014-10-17 EP EP14786612.3A patent/EP3058506A1/de not_active Withdrawn
  • 2014-10-17 CN CN201480057420.1A patent/CN105637531A/zh active Pending
  • 2014-10-17 WO PCT/EP2014/002811 patent/WO2015055320A1/de active Application Filing

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events