TWI435280B - Gesture recognition interaction system - Google Patents

Description

Gesture recognition interactive system

The invention relates to an interactive system, in particular to a gesture recognition interactive system.

The current hand-controlled interactive system needs to input data through various hardware peripheral input devices, such as gloves, keyboard, mouse, touch screen, etc. The disadvantage is that the use of the device will damage the peripheral device and increase the hardware. Cost, and in terms of interaction, it feels less in line with the natural principles of human habits.

Accordingly, it is an object of the present invention to provide a gesture recognition interactive system that avoids such deficits and adds natural sensation during interaction.

The gesture recognition interaction system comprises: a database storing a plurality of virtual object images and including a gesture correspondence table, wherein the gesture correspondence table is configured to store a control instruction corresponding to each gesture action; and an image capture device, Performing continuous shooting on the gesture of a user to obtain a dynamic image sequence having a plurality of captured images; and a processor for performing image separation on each captured image of the image sequence to obtain the image sequence Each hand region image is further characterized by each feature region image of the image sequence to obtain a feature information, and the feature information is compared from the gesture correspondence table to find a corresponding control command; And a screen for displaying an image of each hand area and displaying a virtual object image on the image of the hand area; wherein the processor further operates the virtual object image on the screen according to the control instruction.

The above and other technical contents, features and advantages of the present invention will be apparent from the following detailed description of the preferred embodiments.

As shown in FIG. 1 , a preferred embodiment of the gesture recognition interaction system of the present invention is adapted to recognize a gesture of a user to interact as an input signal, and includes: an image capture device 2 (eg, network digits) A camera or a camera), a database 5, a processor 4, and a screen 3.

The processor 4 is electrically connected to the image capture device 2, the data library 5, and the screen 3.

The database 5 stores a plurality of three-dimensional virtual object images, an animation corresponding to each object image, and includes a gesture correspondence table for storing a control instruction corresponding to each gesture action.

The gesture recognition interaction system performs a gesture recognition interaction method according to a training mode and an application mode, and the gesture recognition interaction method includes the following steps when performing the training mode, as shown in FIG. 2:

<training mode>

Step 11: An image capture device 2 photographs a predetermined range of a region in which a hand is located to obtain a captured image having a hand region and a background region.

Step 12: The processor 4 performs image separation on the captured image to obtain a hand region image.

In this embodiment, the detailed method for performing image separation is: the processor 4 obtains the hand region image according to a boundary value related to the grayscale value, but is not limited thereto, and may also use other Image separation technology.

Step 13: The screen 3 is controlled by the processor 4 to display the image of the hand area, and at least one virtual object image is displayed on the image of the hand area.

Step 14: On the screen 3, the user is informed that the virtual object image is operated by a dynamic gesture.

Among them, the operation includes selecting, holding, moving, zooming, rotating, and starting related animations.

Step 15: The image capturing device 2 continuously captures the gesture motion to obtain a dynamic image sequence with the plurality of captured images, and stores the image sequence in the database 5.

Step 16: The processor 4 performs image separation on each captured image of the image sequence to obtain an image of each hand region of the image sequence, and displays each hand region image on the screen in real time.

Step 17: The processor 4 performs feature extraction on the image sequence to obtain a feature information, where the feature information includes a finger movement track and a palm movement track.

The detailed method of performing feature extraction is as follows: the processor 4 performs edge calculation on each hand region image to obtain an image with a hand contour, and then presents a narrow outline according to the finger and a clustered outline according to the palm to distinguish the finger. The position and the palm position are further changed from the finger position and the palm position in the image sequence to obtain a movement trajectory, but the present invention is not limited thereto, and the present invention can also be implemented using other feature extraction techniques.

Step 18: The processor 4 establishes a control instruction according to the feature information, and stores the control instruction and the feature information in a gesture correspondence table.

<application mode>

As shown in FIG. 3, when the manual interaction method performs the application mode, the difference from the training mode is the same as steps 11 to 17, but there is no step 18, and the following steps are further included:

Step 21: The processor 4 compares the feature information from the gesture correspondence table to find a corresponding control instruction.

Step 22: The processor 4 operates the virtual object image on the screen 3 according to the control instruction.

In summary, the preferred embodiment of the present invention has the following advantages: under the system of the present invention, by recognizing the gesture action instead of the use of the peripheral input device, the interface of the human-computer interaction can be simplified to increase the natural feeling effect. And avoid the damage in the process of use to achieve the purpose of reducing the cost of hardware.

The above is only the preferred embodiment of the present invention, and the scope of the invention is not limited thereto, that is, the simple equivalent changes and modifications made by the scope of the invention and the description of the invention are All remain within the scope of the invention patent.

2. . . Image capture device

3. . . Screen

4. . . processor

5. . . database

11. . . Steps to take photography

12. . . Steps of image separation

13. . . Steps to display images

14. . . Steps to inform the operation

15. . . Dynamic photography steps

16. . . Steps of image separation

17. . . Feature extraction step

18. . . Steps to establish control instructions

twenty one. . . Steps to find control instructions

twenty two. . . Steps to proceed

Figure 1 is a schematic view of a preferred embodiment of the present invention;

2 is a flow chart of the preferred embodiment performing a training mode; and

3 is a flow chart of the execution mode of the preferred embodiment.

2. . . Image capture device

3. . . Screen

4. . . processor

5. . . database

Claims (5)

A gesture recognition interaction system includes: a database storing a plurality of three-dimensional virtual object images, an animation corresponding to each object image, and a gesture correspondence table for storing each gesture action Corresponding control command; an image capture device for continuously capturing the gesture of a user to obtain a dynamic image sequence having a plurality of captured images; a processor for each of the image sequences An image is captured for image separation to obtain an image of each hand region of the image sequence, and a feature image is captured for each hand region image of the image sequence to obtain a feature information, and the feature information is corresponding to the gesture information. The table is compared to find a corresponding control command; and a screen is used for instantly displaying each hand region image and presenting a three-dimensional virtual object image on the hand region image; wherein the processor is further configured according to the The control command operates the three-dimensional virtual object image on the screen. The gesture recognition interaction system according to claim 1, wherein the processor first executes a training mode, wherein the processor establishes the control instruction according to the feature information, and the The control command and the feature information are stored in the gesture correspondence table. According to the gesture recognition interactive system described in claim 1, wherein the three-dimensional virtual object image stored in the database is used as opposed to The animation should be displayed on the screen when the control command belongs to the startup related animation. The gesture recognition interactive system of claim 1, wherein the operations include selecting, holding, moving, zooming, and rotating. The gesture recognition interaction system according to claim 1, wherein the processor first executes a training mode in which the screen is controlled by the processor to display at least one virtual object image, and The user is informed to operate the three-dimensional virtual object image in a dynamic gesture.