TWI444908B - Hand gesture image recognition method and system using image orientation alignment - Google Patents

Description

Gesture image recognition method and system using image direction alignment processing

The present invention relates to a gesture image recognition method and system thereof using image direction alignment processing; in particular, an image orientation calculation method is used to calculate an angle value of a gesture image, and then a decomposition and recombination method is used to eliminate the non-skin color of the gesture image. Gesture image recognition method and system for non-skin color gap.

A related application device for recognizing gesture image recognition, for example, the Republic of China Patent Publication No. M382675, a new type of surveillance camera control device based on gesture recognition, which discloses that a camera lens can be turned up and down, left and right, and zoomed in. And remote control and other action command control devices. Using the gesture image of the user captured by the gesture camera to recognize the movement of the user's gestures up, down, left, right, front, and back, and to send the camera up, down, left, and right to the surveillance camera. To zoom in and out, without having to manipulate the mouse or joystick to release the above control message.

No. M382675 first uses a gesture camera to capture the user's hand image, and then uses the gesture displacement detection module to detect and recognize the user's hand position and trend, thereby calculating the direction of the camera to be controlled, and then transmitting the signal. Control the camera's lens to perform actions of going up, down, left, right, zooming in and out. The gesture displacement detecting module first calculates the position and area of the hand in the gesture camera, and serves as the origin of the gesture recognition. This part can also preset a gesture recognition on the screen of the gesture camera. The origin is used as a displacement detection reference point for subsequent gesture recognition, and defines a threshold value of a minimum area of a gesture, a threshold value of a minimum moving distance of a gesture, and a threshold value of a minimum area change of a gesture.

No. M382675, after the user's gesture area size is greater than a preset threshold minimum area threshold, the device starts to activate the gesture recognition function, and uses the gesture displacement detection module to detect the current gesture position and the displacement of the gesture origin. . When the displacement of the gesture movement is greater than a preset threshold value, the gesture movement is considered to occur. Conversely, when the displacement amount of the gesture movement is less than the threshold value, no gesture movement occurs; and the gesture displacement detection is performed. The module also detects the current gesture position and the direction of displacement of the gesture origin. If the gesture moves up, down, left or right, a control signal is triggered to drive the camera lens to move up, down, left or right. In addition, the gesture displacement detection module uses image recognition technology to determine the movement of the gesture in the range of motion; when the gesture moves forward, the area thereof is larger than the area threshold, and a control signal is triggered to drive the surveillance camera lens to zoom in. When the gesture moves backwards, its area will be smaller than the area threshold, triggering a control signal to drive the camera lens to zoom out.

Another application device for recognizing gesture image recognition, for example, the gesture recognition system of the Republic of China Patent No. I298461 and its method and invention patent, which discloses a gesture recognition system and a method thereof applied to an image capture device Notebook computer. The user can directly align the image capture device with a preset gesture, and the notebook computer will execute the corresponding application or function option of the gesture action to increase the user execution application or function options. The convenience of time.

Another application device for recognizing gesture image recognition, for example, the gesture recognition system of the Republic of China Patent Publication No. 201030630 and the method thereof, and a method for identifying a gesture recognition system comprising: a photographing device for obtaining a possible natural gesture An image is used to find the skin contour of the skin color portion from the image, and then classify the skin contour into a plurality of contour fragments of different angles; an arithmetic engine has a plurality of parallel computing units and a plurality of different angles; a gesture template library of the category, wherein the plurality of parallel operation units respectively find a gesture template that is closest to the contour fragments in the gesture template library of different angle categories; an optimal template selection means is freely found by the plurality of parallel operation units A plurality of approximate gesture templates are selected to select an optimal gesture template; and a display terminal is used to display an image of the best gesture template; thereby achieving the purpose of instantly recognizing the gesture without using any marker [marker less].

In addition, the related application techniques of conventional gesture image recognition have been disclosed in some U.S. patents, for example, U.S. Patent No. 7,702,130, User interface apparatus using hand gesture recognition and method thereof, No. 7,680,295, Hand-gesture based interface apparatus〞 No. 6,215,890, Hand gesture recognizing device〞, No. 6,002,808, Hand gesture control system〞, and No. 5,594,469, Hand gesture machine control system〞. The foregoing Japanese patents and U.S. patents are only for the purpose of the present invention and are not intended to limit the scope of the present invention.

Although the aforementioned patents have disclosed related gesture image recognition techniques, they do not provide a technique for how to align the gesture image orientation. In fact, in terms of gesture image recognition technology, it is necessary to properly align the gesture image, otherwise it affects the reliability of gesture image recognition. Therefore, the conventional gesture image recognition technology necessarily has the need to further align the direction of the gesture image.

In view of the above, in order to meet the above requirements, the present invention provides a gesture image recognition method and system thereof using image direction alignment processing, which utilizes an image direction calculation method to process a gesture image to accurately identify a gesture image and simplify gestures. Image and the purpose of reducing its amount of data.

The main object of the present invention is to provide a gesture image recognition method and system thereof using image direction alignment processing, which uses an image direction calculation method to calculate an angle value of a gesture image to correct the direction of the gesture image to achieve an accuracy. Identify the purpose of the gesture image.

Another object of the present invention is to provide a method and system for recognizing a gesture image using image direction alignment, which uses a decomposition and recombination method to eliminate non-skinning gaps of the gesture image, thereby reducing the size of the gesture recognition area and accelerating recognition, thereby simplifying Gesture image recognition and the purpose of reducing its amount of data.

In order to achieve the above object, the method for recognizing a gesture image using the image direction alignment method comprises: inputting a skin color gesture image; calculating a global character heart in the skin color gesture image; and selecting a region of interest in the skin color gesture image; Selecting a sub-region by using the region of interest; calculating a region property core in the sub-region; and calculating a pair of positive angles by using the global property heart and the region property heart.

The global nature of the preferred embodiment of the present invention is calculated from the skin tone pixels of the skin tone gesture image.

The preferred embodiment of the present invention utilizes the global property to calculate the region of interest.

The sub-area of the preferred embodiment of the invention is located below the global nature of the heart.

In the preferred embodiment of the present invention, the gesture image recognition method using the image direction alignment processing further includes: performing image decomposition and recombination processing on the skin color gesture image after the angle alignment is performed.

In the preferred embodiment of the present invention, a pixel scanning method is employed in the decomposition and recombination process.

The pixel scanning method of the preferred embodiment of the present invention is a pixel horizontal and vertical scanning method.

In the preferred embodiment of the present invention, the gesture image recognition method using the image direction alignment processing further includes: decomposing an original gesture image by using a singular value decomposition method to obtain a gain gesture image; and removing the dark background from the gain gesture image. Obtaining at least one type of skin pixel block; and performing skin color detection on the skin pixel block by using a skin color detection method to remove the remaining background of the skin pixel block to form the skin color gesture image .

The gesture image recognition system of the present invention includes: an input unit for inputting a skin color gesture image; and an arithmetic unit connected to the input unit, the calculation unit calculates a skin color gesture image a global nature heart; selecting a region of interest for the skin color gesture image; selecting a sub-region using the region of interest; calculating a region property center in the sub-region; calculating a pair of positive values using the global property heart and the region property heart An angle; and an output unit coupled to the calculation unit, the pair of positive angles being output from the output unit.

In a preferred embodiment of the present invention, singular value decomposition is performed in an RGB gamut space when the original gesture image is decomposed.

In a preferred embodiment of the present invention, singular value decomposition is performed in each color gamut channel when the singular value decomposition is performed in the RGB gamut space.

In a preferred embodiment of the present invention, a skin color detection method is used to remove the remaining background when the remaining background of the skin pixel block is removed.

The skin color detecting method according to the preferred embodiment of the present invention is a color skin color detecting and cutting method.

The chromaticity skin color detecting and cutting method of the preferred embodiment of the present invention adopts a Y _CbCr model.

In order to fully understand the present invention, the preferred embodiments of the present invention are described in detail below and are not intended to limit the invention.

The gesture image recognition method and system thereof using the image direction alignment processing in the preferred embodiment of the present invention are applicable to various gesture recognition devices and related application devices, such as various types of computer systems, medical care systems, or security systems, but they are not It is intended to define the scope of the invention.

FIG. 1 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof in an unprocessed original gesture image group according to a preferred embodiment of the present invention. Referring to FIG. 1 , a gesture image recognition method and system for image direction alignment processing according to a preferred embodiment of the present invention are used to process eight basic gesture images, such as the eight images shown in FIG. 1 . For example, eight images include a palm gesture, a fist gesture, a V-shaped gesture, etc., but are not intended to limit the scope of the invention.

Referring to FIG. 1 again, the gesture image recognition system using the image direction alignment processing according to the preferred embodiment of the present invention includes an input unit, a calculation unit and an output unit, wherein the input unit is connected to the calculation unit. And the output unit is connected to the calculation unit. The method for image recognition using the image direction alignment of the present invention is selected to be executed on the input unit [eg, various photographic devices, various image detecting devices, various image storage devices, or other similar image access devices] for inputting at least An original gesture image, such as the eight images shown in Figure 1.

FIG. 2 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for decomposing an original gesture image group by singular value decomposition according to a preferred embodiment of the present invention, and obtaining a schematic diagram of a gain gesture image set [enhanced image set], It corresponds to the original gesture image shown in Figure 1 [ie, the gesture image has not been processed]. Referring to FIG. 2, the gesture image recognition method using the image direction alignment processing of the present invention is selected to be executed in the calculation unit [eg, a computer or the like], which may first select and process the complex background by using singular value decomposition. Or, you can ignore complex backgrounds using singular value decomposition. The calculation unit decomposes the original gesture image by a singular value decomposition method to obtain a gain gesture image, such as the gain processed gesture image shown in FIG. 2 .

FIG. 3 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for obtaining a skin-like pixel block after removing a dark background from a gain gesture image group according to a preferred embodiment of the present invention, which corresponds to a Figure 2 shows the gain gesture image. Referring to FIG. 3, the gesture image recognition method using the image direction alignment processing of the present invention is executed in the calculation unit, and the dark background or the dark background is removed from the gain gesture image. At least one type of dermal pixel block is obtained, as shown in Figure 3, with the dark background image removed.

In the preferred embodiment of the present invention, the gesture image recognition method using image direction alignment processing performs singular value decomposition in an RGB gamut space when decomposing the original gesture image, and performs singular value in each color gamut channel [color channel]. Decomposition, which uses a singular value decomposition matrix,

Where U _A and V _A are orthogonal square matrixes, and Σ _A is a matrix containing diagonal singular values [intensity messages].

The Σ _A matrix is replaced by another singular value matrix obtained from a synthetic normalized image. The synthetic normalized image is a probability density function (PDF) having a Gaussian distribution with a mean value of 0.5 and a variation value of 1. Then, the ratio of the maximum singular value of the normalized matrix of the normalized image is

Where Σ _{G (μ=0.5, σ=1)} is the singular value matrix of the synthesized normalized image, τ is the weighting constant [weighting constant], and the weight constant is set to 6. The singular value ratio ξ is used to generate a new singular value matrix, which is an equalized matrix [equalized matrix], which generates a matrix,

among them Is an equalized image of the A color gamut channel [equalized image].

In order to further remove the remaining background from the dermatomorphic block, the gesture image recognition method and the system using the image direction alignment processing in the preferred embodiment of the present invention utilize a skin color detection method for the skin pixel block. Skin color detection to remove the remaining background of the skin pixel block [ie, the background portion of the skin-like pixels].

FIG. 4 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for obtaining a skin color gesture image group after removing the remaining background from the skin pixel panel according to a preferred embodiment of the present invention, which corresponds to the third The dark background image has been removed as shown. Referring to FIG. 4, the gesture image recognition method using the image direction alignment processing of the present invention is performed on the calculation unit, and then performing skin color detection on the skin pixel block to remove the skin image. The remaining background of the prime block [residual background], as shown in Figure 4, has removed the remaining background image.

In the preferred embodiment of the present invention, a gesture detection method using image direction alignment processing may select a skin color detection method, which is preferably a chrominance-based SCD segmentation method. Further, the skin color detection using a method of cutting _{the CbCr} model Y, where Y is the luminance component [luminance component], C _b and C _r is the value of the chrominance component [chrominance component].

FIG. 5 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for removing noise from a skin color gesture image group according to a preferred embodiment of the present invention, which corresponds to the removed residual shown in FIG. 4 Background image. Referring to FIG. 5, the gesture image recognition method using image direction alignment processing of the present invention is executed in the calculation unit, and then removing the noise [noise] from the skin color gesture image to improve the overall gesture image recognition. Reliability [reliability], as shown in Figure 5, has removed the noise of the skin tone gesture image.

In the preferred embodiment of the present invention, the gesture image recognition method using the image direction alignment processing is selected to be executed in the output unit [eg, an image display unit or other image output device], and the image of the skin color gesture image with the noise removed is selected. Output to the outside of the system or for subsequent gesture image processing.

FIG. 6 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for directly inputting another skin color gesture image group according to a preferred embodiment of the present invention. Referring to FIG. 6 , the gesture image recognition method using the image direction alignment processing of the present invention is selected and executed on the input unit, and the image input gesture unit may be directly input, that is, the first to fifth images are ignored. The singular value decomposition is used to deal with complex background processing. At this time, the calculation unit calculates a global centroid for the skin color gesture image in an appropriate manner.

The global nature of the heart calculates the two-dimensional coordinate position according to the skin color pixel of the gesture geometric image of the skin color gesture image. For example, the equation of the present invention adopts the equation for calculating the two-dimensional coordinate position of the global nature heart.

Where ( x _G , y _G ) is the two-dimensional coordinate position of the global heart, and N _p is the total number of skin pixels.

FIG. 7 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for generating a region of interest (ROI) in a skin color gesture image according to a preferred embodiment of the present invention, which corresponds to FIG. The skin tone gesture image shown. Referring to FIG. 7 , the gesture image recognition method using the image direction alignment processing of the present invention is selected and executed in the calculation unit, and may previously select to use the singular value decomposition to process the complex background; or, the singular value may be previously ignored. Decompose to deal with complex backgrounds. At this time, the calculation unit searches for the pixels in the skin color gesture image in an appropriate manner to obtain a region of interest.

In addition, the gesture image recognition method using the image direction alignment processing of the present invention may select the two-dimensional coordinate position ( x _G , y _G ) of the global nature heart to search for the region of interest in the skin color gesture image. The block, and the size of the block of interest is 100 pixels x 100 pixels, but it is not intended to limit the scope of the invention.

FIG. 8 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for generating a sub-region in a region of interest according to a preferred embodiment of the present invention, which corresponds to FIG. 7 Block of interest. Referring to FIG. 8 , the gesture image recognition method using the image direction alignment processing of the present invention is selected and executed in the calculation unit, and the calculation unit searches for a sub-area in the interest block in an appropriate manner, such as Figure 8 shows the frame.

In addition, the gesture image recognition method using the image direction alignment processing of the present invention may select the sub-region below the global property center, such as the frame position of each skin color gesture image shown in FIG. 8 , and the sub-region The size specification is 30 pixels x 80 pixels, but it is not intended to limit the scope of the invention.

FIG. 9 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for generating a global domain and a local centroid on a skin color gesture image group according to a preferred embodiment of the present invention, which corresponds to a The skin color gesture image shown in Fig. 6 and the sub-region shown in Fig. 8. Referring to FIG. 9 , the gesture image recognition method using image direction alignment processing of the present invention is selected and executed in the calculation unit, and the calculation unit generates the global domain heart ( x _G , y _G ) in the skin color gesture image. , as shown in Figure 9, the arrow above the image of each skin tone gesture. In contrast, the present invention calculates the two-dimensional coordinate position of a region's nature center based on the skin color pixels of the sub-region. The present invention also employs the aforementioned equation for calculating the two-dimensional coordinate position of the global nature of the heart, and obtains the two-dimensional coordinate position ( x _L , y _L ) of the nature of the region, as shown in FIG. 9 below each skin color gesture image. arrow.

FIG. 10 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for generating a global domain and a regional property on another skin color gesture image group according to a preferred embodiment of the present invention; FIG. 11 is a view showing the present invention; In the preferred embodiment, the gesture image recognition method using the image direction alignment processing and the system thereof complete the image alignment of the skin color gesture image group, which corresponds to the unaligned image shown in FIG.

Referring to Figures 10 and 11, the pair of positive angles [pan/tiltθ] are calculated using the global domain ( x _G , y _G ) and the regional nature ( x _L , y _L ), for example, the present invention uses calculations The equation for the positive angle is

In the preferred embodiment of the present invention, the gesture image recognition method using the image direction alignment processing and the system thereof perform rotation adjustment of the skin color gesture image according to the alignment angle, for example, rotation adjustment to a vertical position [upright position]. Next, the present invention can select to output the pair of positive angles from the output unit.

FIG. 12 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for inputting three gesture image groups of six sample individuals according to a preferred embodiment of the present invention. Referring to FIG. 12, the three gesture image groups include a palm gesture [digital five gesture, as shown in FIG. 12a], a V-shaped gesture [digital two gesture, as shown in FIG. 12b], and a three-finger gesture [ The number three gesture, as shown in Figure 12c].

FIG. 13 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for decomposing and reconstructing three gesture image groups of six sample individuals according to a preferred embodiment of the present invention, which corresponds to FIG. 12 The recombination gesture image has not been decomposed. Please refer to Figure 13, the three decomposition and reorganization gesture image groups include a palm gesture [digital five gestures, as shown in Figure 13a], a V-shaped gesture [number two gestures, as shown in Figure 13b] and three fingers Gesture [number three gestures, as shown in Figure 13c].

The gesture image recognition method using the image direction alignment processing of the present invention is selected to be executed in the input unit, and the image input gesture unit can be directly input, that is, the image direction alignment processing shown in FIGS. 1 to 11 is ignored. At this time, the calculation unit performs decomposition and recombination processing on the skin color gesture image in an appropriate manner.

Referring to FIG. 13 , the calculation unit uses a decomposition and recombination method to eliminate the non-skinning gap of the gesture image during the image decomposition and reorganization process, so as to reduce the size of the gesture recognition area and accelerate the recognition, so as to achieve simplified gesture image recognition and Reduce the amount of data. In the decomposition and recombination process, a scanned pixel-by-pixel method is used, and the pixel scanning method is a pixel horizontal and vertical scanning method. Finally, the present invention may select from the output unit to output the skin tone gesture image that has been decomposed and recombined.

The foregoing preferred embodiments are merely illustrative of the invention and the technical features thereof, and the techniques of the embodiments can be carried out with various substantial equivalent modifications and/or alternatives; therefore, the scope of the invention is subject to the appended claims. The scope defined by the scope shall prevail.

[No component symbol]

FIG. 1 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system thereof in an unprocessed original gesture image group according to a preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of a gesture image recognition method and a system for correcting image processing by using image direction alignment processing according to a preferred embodiment of the present invention. After the original gesture image group is subjected to singular value decomposition processing, a schematic diagram of the gain gesture image group is obtained.

FIG. 3 is a schematic diagram of a gesture image recognition method using image direction alignment processing according to a preferred embodiment of the present invention and a system for obtaining a skin-like pixel block after removing a dark background from a gain gesture image group.

FIG. 4 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system for removing a residual background from a skin texel block according to a preferred embodiment of the present invention.

FIG. 5 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system for removing noise from a skin color gesture image group according to a preferred embodiment of the present invention.

FIG. 6 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system for directly inputting another skin color gesture image group according to a preferred embodiment of the present invention.

FIG. 7 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system thereof for generating a region of interest in a skin color gesture image according to a preferred embodiment of the present invention.

FIG. 8 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system for generating a sub-region in a region of interest according to a preferred embodiment of the present invention.

FIG. 9 is a schematic diagram showing a gesture image recognition method using image direction alignment processing and a system thereof for generating a global nature heart and a region property heart on a skin color gesture image group according to a preferred embodiment of the present invention.

FIG. 10 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system thereof for generating a global nature heart and a regional property heart on another skin color gesture image group according to a preferred embodiment of the present invention.

11 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system thereof for performing image alignment on a skin color gesture image group according to a preferred embodiment of the present invention.

FIG. 12 is a schematic diagram of a gesture image recognition method using image direction alignment processing and a system for inputting three gesture image groups of six sample individuals according to a preferred embodiment of the present invention.

Figure 13 is a schematic diagram of a gesture image recognition method and system for correcting image orientation using a preferred embodiment of the present invention. The three gesture image groups of six sample individuals are decomposed and reconstructed.

[No component symbol]

Claims (10)

A gesture image recognition method using image direction alignment processing, comprising: inputting a skin color gesture image; calculating a global property heart in the skin color gesture image; selecting a region of interest in the skin color gesture image; and utilizing the region of interest The block selects a sub-region; calculates a region property core in the sub-region; and calculates a pair of positive angles using the global property heart and the region property heart. The gesture image recognition method using the image direction alignment processing according to the first aspect of the patent application scope, wherein the global property heart is calculated by the skin color pixel of the skin color gesture image. According to the first aspect of the patent application, the gesture image recognition method using image direction alignment processing is used, wherein the region of interest is calculated by using the global property. According to the method for identifying a gesture image that is processed by the image direction according to the first aspect of the patent application, the method further comprises: performing image decomposition and recombination processing after the color gesture image is aligned at an angle. According to the fourth aspect of the patent application, the gesture image recognition method using the image direction alignment processing, wherein a pixel scanning method is used in the decomposition and recombination processing. A gesture image recognition system using image direction alignment processing, comprising: an input unit for inputting a skin color gesture image; and an arithmetic unit for calculating a global character heart of the skin color gesture image; the calculation unit is The skin color gesture image selects a region of interest, and uses the region of interest to select a sub-region; the calculation unit calculates a region property center in the sub-region, and calculates a pair of positive angles by using the global property heart and the region property heart; An output unit that outputs the aligned angle. According to the sixth aspect of the patent application, the gesture image recognition system adopts image direction alignment processing, wherein the global property is calculated by the skin color pixel of the skin color gesture image. The gesture image recognition system adopting image direction alignment processing according to item 6 of the patent application scope, wherein the region of interest is calculated by using the global property heart. According to the sixth aspect of the patent application, the gesture image recognition system that uses the image direction alignment processing further includes: performing image decomposition and recombination processing on the skin color gesture image after the angle alignment is performed. According to the ninth application of the patent application, the gesture image recognition system adopts the image direction alignment processing, wherein a pixel scanning method is adopted in the decomposition and recombination processing.