TWM617136U - Gesture control device - Google Patents

Abstract

A gesture control device includes an image processing module that is electrically connected to a photographing unit, and the image processing module is used to receive a two-dimensional image captured by the photographing unit for subsequent processing , Including capturing a two-dimensional image of the hand and performing gray-scale processing on the two-dimensional image of the hand and calculating the feature points of the two-dimensional image. The image processing module is also electrically connected to an image analysis module, and the image analysis The module is used to receive the calculated two-dimensional image information for subsequent analysis, including marking the calculated two-dimensional image, drawing the marked feature position or calculating other nodes, and finally judged by the gesture The unit performs gesture interpretation, and finally outputs the result of the gesture interpretation.

Description

Gesture control device

This creation is related to a control device, especially a gesture control device with man-machine interaction.

The traditional human-computer interaction interface is mouse, keyboard and joystick. With the rapid development of technology, in order to make human-computer interaction more convenient, somatosensory control provides a new input method. The most common somatosensory control change is through Gesture recognition for a new human-computer interaction.

Because gestures are a common way of communication between people in daily life, it is a very intuitive and convenient method. Therefore, under the guidance of technology, gesture recognition is gradually applied in one of many fields, including human-machine interface design, medical rehabilitation. Health, virtual reality and game design, etc.

There are two main ways to identify gesture information, one is dynamic gesture, the other is static gesture; dynamic gesture information includes hand movement trajectory, position information, and timing relationship, while static gesture information is mainly hand shape changes. Analyze gesture information and achieve the function of human-computer interaction according to different gestures.

The current common gesture recognition technology is to use depth cameras to obtain three-dimensional images. Each three-dimensional image must be pre-processed, such as image binarization and image background removal, noise elimination, etc., and then capture from a series of images Obtain and analyze relevant information such as the user's hand position and gestures. Then use the image coordinate value of the hand position to control the movement of the cursor of the display; then, it takes a long time to pre-process the three-dimensional image, which makes the speed and accuracy of the moving cursor difficult to match. Compared with mice, and the cost of devices such as depth cameras is too high, it is difficult to reduce the cost threshold of current gesture recognition devices and technologies.

In view of the above-mentioned deficiencies, the main purpose of this creation is to provide a gesture control device and its control method. The combination of a camera that captures two-dimensional images and an algorithm can determine the change of gestures and achieve the purpose of human-computer interaction. And thereby reduce the cost of the gesture control device.

In order to achieve the above-mentioned purpose, the author mainly provides a gesture control device, which includes an image processing module, the image processing module is electrically connected to a photographing unit, and the image processing module is used to receive data from the photographing unit. The two-dimensional image captured by the unit is subjected to subsequent processing. The image processing module further includes an image processing unit and a feature calculation unit. The image processing unit performs post-processing after receiving the captured two-dimensional image. Processing, including grabbing a two-dimensional image of the hand and performing gray-scale processing on the two-dimensional image of the hand. The to-be-processed two-dimensional image is input to the feature calculation unit, and the feature points of the two-dimensional image are calculated. The image processing module is also electrically connected to an image analysis module, the image analysis module is used to receive the calculated two-dimensional image information for subsequent analysis, wherein the image analysis module further includes an image analysis unit and A gesture judgment unit, wherein the image analysis unit is used to receive the two-dimensional image information for analysis, mark the calculated two-dimensional image, and draw the marked feature position or calculate other nodes, and finally The gesture judgment unit performs gesture judgment, and finally outputs the result of the gesture judgment.

In order to make the above and other purposes, features and advantages of this creation more obvious and easy to understand, the following is a detailed description of preferred embodiments in conjunction with the accompanying drawings.

1: Image processing module

11: Image processing unit

12: Feature calculation unit

2: Photography unit

3: Image analysis module

31: Image analysis unit

32: Gesture Judgment Unit

4: Gesture training module

41: Gesture Recording Unit

42: Gesture Training Unit

The first picture is the system block diagram of this creation.

The second picture is the flow chart of this creation.

The third figure is the flow chart of feature calculation for this creation.

The fourth picture is an example picture (1) of this creation.

The fifth picture is an example picture (2) of this creation.

The sixth picture is an example picture of this creation (3).

The seventh picture is an example picture of this creation (4).

First clarify that the "feature points" in this embodiment refer to several points obtained through image detection and analysis, which can be joint positions, skin lines at joints, nails, palm prints, or around the wrist Appearance and so on. Compare the acquired feature points with the joint points set during the previous training, and then complete the image key points at the marked joints. Use the pattern formed by the connection between the key points to compare the shape of the previously preset gesture to activate the corresponding function.

Please refer to the first picture, which is the system block diagram of this creation. The gesture control device of this creation mainly includes an image processing module 1, which is electrically connected to a photographing unit 2, and the image processing module 1 is used to receive the image captured by the photographing unit 2. The two-dimensional image is subjected to subsequent processing, and the photographing unit 2 is a camera with a function of capturing two-dimensional images; the image processing module 1 further includes an image processing unit 11 and a feature calculation unit 12, the image The processing unit 11 is electrically connected to the feature calculation unit 12. The image processing unit 11 performs post-processing after receiving the captured two-dimensional image, including capturing a two-dimensional image of the hand and the two-dimensional image of the hand. The image is gray-scaled, and the to-be-processed two-dimensional image is input to the feature calculation unit 12, and the feature point (scale-invariant feature transform descriptor) of the two-dimensional image is calculated, and the feature point calculation method is For each keypoint (keypoint), the pixel size of 16×16 is captured, and then averaged into 4×4 cells. Take the gradient and angle value of each grid into 8 bins histograms (Histogram), a total of 16 grids will get 16 histograms (8bins), which can be combined into 16×8=128 dimension data , And finally normalize the data (L2-Normalizing) to obtain the feature representing the key point; that is, the position of the feature point can be two-dimensional coordinates (X, Y). After the key point is converted by calculation, Its position coordinates are three-dimensional coordinates (X, Y, Z). The calculation conversion method can be to calculate the Z value of each key point based on the ratio of the size of the reference point to each key point by comparing the reference points in the image (for example, the face or the object behind the back). When obtaining the reference point, the Z value of each key point can also be obtained or corrected by hand movement.

Continue to refer to the first figure. The image processing module 1 is also electrically connected to an image analysis module 3. The image analysis module 3 is used to receive the calculated two-dimensional image information for subsequent analysis, wherein the image analysis module 3 further includes an image An analysis unit 31 and a gesture determination unit 32. The image analysis unit 31 is connected to the gesture determination unit 32 with electrical information, and the image analysis unit 31 is used to receive the two-dimensional image information for analysis, and to calculate the two-dimensional image Marking is performed, and the marked feature position is drawn or other nodes are calculated. Finally, the gesture judgment unit 32 performs gesture judgment, and finally outputs the result of the gesture judgment.

Continue to refer to the first figure. In addition, the image analysis module 3 is electrically connected to a gesture training module 4. The gesture training module 4 further includes a gesture recording unit 41 and a gesture training unit 42. The gesture recording unit 41 is electrically connected to the gesture The training unit 42, wherein the gesture recording unit 41 is used to record gesture images, and load the gesture images into the gesture training unit 42 for gesture training and generate gesture training files, which are then transferred to the gesture judgment unit 32 , As a basis for gesture judgment.

Please refer to the second figure, which is the flow chart of the gesture control method for this creation. The steps of the control method include capturing a two-dimensional hand image (S1), the two-dimensional hand image is operated by a camera with a function of capturing a two-dimensional image; and then the characteristics of the two-dimensional hand image are calculated Point (S2), pack Including gray-scale processing and feature point calculation on the two-dimensional image of the hand, where the steps of the calculation further include capturing a pixel size of 16×16 for each keypoint (S21), as shown in the third figure. As shown in the feature calculation flow chart, each pixel is averaged into 4×4 cells (S22), and the gradient and angle values of each cell are counted as 8 bins. Figure (Histogram) (S23), a total of 16 grids will get 16 histograms (8bins), and finally merge into 16×8=128 dimension data (S24), and finally normalize the dimension data (L2-Normalizing ) (S25), the characteristic of the key point can be obtained.

Continue to refer to the third figure. The step of the control method further includes marking the characteristic points of the two-dimensional hand image (S3). In this step, the calculated two-dimensional image is marked, and the marked characteristic positions are drawn or calculated. Node; Finally, the gestures (S4) of the two-dimensional hand images are judged for the marked images, and the gesture files stored in the system are interpreted to output the final interpretation results, as shown in Figures 4 to 7 In the illustrated embodiment diagram, according to different gestures, when the system is interpreted, each key point will generate a corresponding movement or a corresponding gesture establishment.

However, the implementations described above are preferred implementation examples. When the scope of implementation of this creation cannot be limited by this, any equivalent changes or modifications made in accordance with the scope of the patent application for this creation and the contents of the specification should all belong to the creation below The scope of patent coverage.

1: Image processing module

2: Photography unit

3: Image analysis module

4: Gesture training module

11: Image processing unit

12: Feature calculation unit

31: Image analysis unit

32: Gesture Judgment Unit

41: Gesture Recording Unit

42: Gesture Training Unit

Claims (2)

A gesture control device includes: an image processing module for receiving captured images for subsequent processing, and further including: an image processing unit in the image processing module After receiving the captured two-dimensional image, post-processing is performed, including capturing the two-dimensional image of the hand and performing gray-scale processing on the two-dimensional image of the hand; a feature calculation unit, the feature calculation unit and the image processing The unit is electrically connected, and the two-dimensional image to be processed is input into the feature calculation unit, and the feature points of the two-dimensional image are calculated; a photographing unit is electrically connected to the image processing module, and the photographing unit is It has the function of capturing two-dimensional images; an image analysis module is electrically connected to the image processing module to receive the calculated two-dimensional image information for subsequent analysis. The image analysis module further includes: an image analysis unit , Is used to receive the two-dimensional image information for analysis, mark the calculated two-dimensional image, and draw the node of the marked feature position or calculate other nodes; a gesture judgment unit is electrically connected to the image analysis Unit, the gesture determination unit is used for interpreting the gesture image, and then outputting the result of the interpretation of the gesture image. Such as the gesture control device described in claim 1, wherein the device further includes a gesture training module electrically connected to the image analysis module, and the gesture training module further includes a gesture recording unit and a gesture training Unit, the gesture recording unit is electrically connected to the gesture training unit, the gesture recording unit is used to record gesture images, and load the gesture images into the gesture training unit for gesture training and generate gesture training files, the files Then it is passed to the gesture judgment unit as a basis for gesture judgment.

Images