KR101269107B1 - Method for recognizing hand gesture using camera and thereof apparatus - Google Patents

Abstract

The present invention relates to a hand gesture recognition method and apparatus using a camera. According to the present invention, detecting a reference point of the hand taken by the camera, and if the movement is detected from the hand detected the reference point, storing the trajectory information of the hand corresponding to the detected movement, and the movement Computing a difference value between the detected current position of the hand and the reference point and comparing it with a preset first threshold value, and when the difference value is greater than or equal to the first threshold value, hand trace information corresponding to the detected movement. And a step of comparing a trajectory DB for each hand gesture stored in advance, and determining a hand gesture that matches the trajectory information of the hand from the trajectory DB.
According to the method and apparatus for recognizing a hand gesture using the camera, the degree of hand movement from a reference point may be primarily determined, and then the recognition rate of the hand gesture may be increased by comparing and verifying the trace information of the hand movement and the previously stored trajectory DB. There is an advantage to that.

Description

Method for recognizing hand gesture using camera and apparatus

The present invention relates to a method and apparatus for recognizing a hand gesture using a camera, and more particularly, to a method and apparatus for recognizing a hand gesture using a camera that can determine a hand gesture by tracking a human hand using the camera.

Conventionally, various methods of recognizing a user's movement and processing the same have been disclosed. As an example, Korean Laid-Open Patent Publication No. 10-2007-0082773 photographs a user's motion while a sensor is attached to a user's hand or finger, processes the photographed image based on the position of the sensor, and then stores the image. It is a configuration that recognizes a user's command compared to data.

According to the conventional invention, as a technology for recognizing an operation command by comparing an image acquired during operation with an existing stored pattern, a lot of samples and data are required for each hand movement because a pattern for a predetermined operation must be acquired and stored in advance. . In addition, since a separate sensor must be installed in the hand, it is cumbersome to use and has the disadvantage of analyzing the position of the sensor.

An object of the present invention is to provide a method and apparatus for recognizing a hand motion using a camera that can easily recognize a hand motion by using a hand far from a reference point and a hand motion trajectory information.

The present invention provides a method for detecting a reference point of a hand photographed by a camera, and when a motion is detected from a hand from which the reference point is detected, storing the hand trace information corresponding to the detected motion, Calculating a difference value between the current position of the detected hand and the reference point and comparing the difference with a preset first threshold value; when the difference value is greater than or equal to the first threshold value, track information of the hand corresponding to the detected movement; A method of recognizing a hand gesture using a camera, the method comprising: comparing a previously stored trajectory DB for each hand gesture, and determining a hand gesture matching the hand information of the hand from the trajectory DB.

The detecting of the reference point of the hand may include comparing the change amount of the position of the hand of the current time point with respect to a previous time point with a preset second threshold value, and if the change amount is less than the second threshold value, Comparing the variance value of the position information of the hand with a preset third threshold, determining that the hand is in a stopped state when the variance value is less than the third threshold value, and the time of the stopped state The method may include determining, by the reference point, a final position of a hand corresponding to a time point passed by a predetermined time.

Here, the hand gesture recognition method using the camera may further include outputting a result of detecting the reference point as a sound or a screen.

In addition, when the hand is detected, an icon corresponding to the captured image of the hand may be provided on the screen to move along the movement of the hand.

In addition, when the reference point is detected, the icon may be displayed in a different form than before the reference point is detected.

The comparing of the trajectories may include: when the difference is greater than or equal to the first threshold, selecting at least one hand gesture candidate group similar to a moving direction of a hand exceeding the first threshold from a trajectory list for each hand gesture in the trajectory DB. The extracted traces of the hand gesture candidate group and the extracted trace information of the hand may be compared with each other.

In addition, the present invention is a reference point detection unit for detecting a reference point of the hand photographed by the camera, and if the movement is detected from the hand detected the reference point, a storage unit for storing the information on the hand trajectory corresponding to the detected movement; A position comparison unit configured to calculate a difference value between the current position of the hand where the movement is detected and the reference point and compare it with a preset first threshold value, and when the difference value is greater than or equal to the first threshold value, The present invention provides a hand gesture recognition apparatus including a trajectory comparator for comparing hand trajectory information with a prestored trajectory DB for each hand gesture, and a motion discrimination unit for determining a hand gesture consistent with the hand trajectory information from the trajectory DB.

According to the method and apparatus for recognizing a hand gesture using a camera according to the present invention, after determining a hand's distance from a reference point, the recognition rate of the hand gesture is again verified by comparing the hand movement information and the previously stored hand movement trace DB again. There is an advantage to increase.

1 is a block diagram of a hand gesture recognition apparatus using a camera according to an embodiment of the present invention.
2 is a flowchart of a hand gesture recognition method using FIG. 1.
FIG. 3 is an exemplary diagram illustrating a type of hand gesture recognition according to FIG. 1.
4 is a reference diagram for describing operations S230 to S250 of FIG. 2.
FIG. 5 is a detailed flowchart for describing operation S210 of FIG. 1.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

1 is a block diagram of a hand gesture recognition apparatus using a camera according to an embodiment of the present invention. The gesture recognition apparatus 100 may include a reference point detector 110, a storage unit 120, a position comparison unit 130, a trajectory comparison unit 140, a trace DB 150, an operation determination unit 160, and a display unit ( 170).

2 is a flowchart of a hand gesture recognition method using FIG. 1. Hereinafter, a hand gesture recognition method using the camera will be described in detail with reference to FIGS. 1 and 2.

First, the reference point detector 110 detects a reference point of a hand photographed through a camera (not shown) (S210). Here, the camera may be built in the device 100 or wired or wirelessly connected. The reference point may correspond to the center of the palm, but the present invention is not necessarily limited thereto. A specific embodiment of the reference point detection according to the step S110 will be described later.

Then, when a movement is detected from the hand from which the reference point is detected, the storage unit 120 stores the trajectory information of the hand corresponding to the detected movement (S220).

Thereafter, a difference value between the current position of the hand where the movement is detected and the reference point is calculated and compared with a preset first threshold value (S230). This step S230 is performed by the position comparison unit 130.

Here, the difference value may correspond to a distance difference from the reference point to the current position of the hand, and may also include displacement or movement direction information of the hand moved from the reference point. In addition, the difference value may be calculated for each (x, y, z) coordinates or in combination.

If the distance difference is less than the first threshold, it is assumed that the hand gesture is not performed. On the contrary, when the difference is greater than or equal to the first threshold, the hand trajectory information corresponding to the detected movement is compared with the information stored in the hand trajectory DB 150 for each hand movement (S240). This step S240 is performed by the trajectory comparison unit 140.

Here, the case where the difference value is greater than or equal to the first threshold includes all cases where the distance difference is larger than the first threshold value, or when the difference is larger and smaller, and it is determined that all of these cases are started and finished.

The trajectory DB 150 maps and stores hand trajectory information corresponding to the various hand gestures. Therefore, according to the step S230, the hand movement information stored in the storage unit 120 and the hand movement trajectory information in the previously stored trajectory DB 150 according to the current hand movement are compared with each other, and the current movement corresponds to any hand movement. Make sure it is correct.

That is, the motion determination unit 160 determines a hand motion that matches the hand information of the hand from the trajectory DB 150 (S250).

For example, when the hand trajectory information according to the current sensed movement coincides with the first trajectory information within an arbitrary error range among the various trajectory information included in the trajectory DB 150 (or the first trajectory information). If the best hand motion), it can be determined that the current hand movement is the first hand motion corresponding to the first trajectory information. Here, comparing the trajectory information with each other determines that the corresponding hand gesture corresponding to the trajectory information having the best trajectory among the various trajectory information is the current hand gesture.

As described above, the present invention recognizes the hand motion in consideration of both the degree away from the reference point and the hand movement information.

FIG. 3 is an exemplary diagram illustrating a type of hand gesture recognition according to FIG. 1. 3 illustrates an operation of moving a hand to the right (1), left (2), up (3), down (4), and camera (5) direction. Of course, FIG. 3 illustrates only five simple operations for convenience of description, and the hand gesture to which the present invention is applied is not necessarily limited thereto.

4 is a reference diagram for describing operations S230 to S250 of FIG. 2. 4 illustrates a case where a difference value between a current position of a hand where a movement is detected and a reference point is out of a first threshold in a positive direction of the x-axis. Through this, it may be assumed that the current hand gesture is the first hand gesture.

Thereafter, the first trajectory information in the trajectory DB 150 defining the first hand gesture is compared with the trajectory information of the hand according to the current movement. If the hand trajectory information according to the current movement matches the first trajectory information among previously stored trajectory information, it may be finally determined that the first hand gesture previously assumed is correct for the current hand gesture.

The reason why the first threshold value and the trajectory information should be compared together is as follows.

For example, if the hand movement is almost unchanged in the Z-axis direction, proceeds diagonally with respect to the X and Y directions, then proceeds in the X-axis direction again and exceeds the first threshold, in the present situation, the first It may be mistaken that it is a hand motion (movement only in the X-axis direction). However, by comparing with the trajectory DB 150 thereafter, it is not the first hand gesture that is moved only in the x-axis direction, but another hand gesture (ex, second hand gesture) in which the diagonal direction and the x-axis direction are combined. Can be accurately determined.

In order to apply this process in detail, the trajectory comparison unit 140 moves the hand that exceeds the first threshold value in the trajectory list for each hand movement in the trajectory DB 150 when the difference value is greater than or equal to the first threshold value. At least one hand gesture candidate group (eg, a first hand gesture and a second hand gesture) similar to a direction is extracted. Here, the movement direction of the hand can be easily obtained from the position information of the current hand moved from the reference point, which can be obtained together when acquiring the difference value.

In addition, the trajectory DB 150 includes trajectory information for each hand gesture, and if the trajectory information is known, the direction information (eg, direction information from the start point to the end point of the hand gesture) to which the corresponding hand gesture moves is also known. The hand gesture candidate group similar to the moving direction may be extracted from the trajectory DB 150. Here, when determining the degree of similarity, information about the direction of movement of the hand with respect to the X, Y, and Z axes, or the degree of inclination with respect to the X, Y, and Z axes may be used. In addition to the above-described example, the similarity comparison determination may use various known methods.

Thereafter, the trajectory comparison unit 140 compares the trajectory of the extracted hand gesture candidate group with the trajectory information of the detected hand. Accordingly, the gesture determination unit 160 may accurately determine that the second hand gesture is the actual hand gesture among the extracted hand gesture candidate groups ex, the first hand gesture, and the second hand gesture.

According to the configuration of the present invention, by first extracting the hand gesture candidate group in the corresponding direction exceeding the threshold value first, and finally extracts the hand gesture to match the trajectory information secondly, it is possible to increase the reliability of hand gesture recognition, Computation time can be reduced by using only the hand gesture candidate as the target of the trajectory comparison.

In order to determine the operation, it may be determined that the operation is performed when the ratio is within an appropriate range in consideration of the moving ratio in the X, Y, and Z directions for each of the trajectory sections. Examples of such operation determination may exist in various embodiments.

FIG. 5 is a detailed flowchart for describing operation S210 of FIG. 1. A reference point detection method of the hand according to the reference point detector 110 is as follows.

First, the position of the hand to be photographed is stored in the real time queue. Then, the amount of change in the position of the hand photographed at the present point in time with respect to the position of the hand photographed at the previous point in time is compared with a preset second threshold (S510). In the calculation of the position change amount, various known methods for detecting the position change or the movement of the object from the captured image may be used.

If the amount of change is greater than or equal to the second threshold, it is determined that the hand moves and the process returns to step S510. This indicates that the reference point has not been detected, since the reference point of the hand cannot be accurately detected when the hand moves beyond the limit range. When the reduction of the S510 is performed by resetting the queue.

When the change amount is less than the second threshold value, the dispersion value of the position information of the hand accumulated so far is compared with the preset third threshold value (S520). In this case, when the dispersion value is greater than or equal to the third threshold value, the reference point of the hand cannot be accurately detected as described above. Therefore, the queue is reset and the operation S510 is performed again.

When the dispersion value is less than the third threshold, it is determined that the hand is in a stopped state, that is, in a state in which there is no movement (S530). Thereafter, the time in the stopped state is compared with a preset threshold time (S540). If the stopped state is greater than or equal to the threshold time, it is determined that the situation where the reference point can be detected is reached. That is, accordingly, the final position of the hand corresponding to the point in time at which the stationary state has passed the predetermined time is detected as the reference point (S550). At this time, the center of the palm of the hand can be set as a reference point among the final position of the hand, but is not necessarily limited to the present invention.

If the time in the stopped state is less than the threshold time, the reference point may not be detected correctly, so the queue is reset and the step S510 is executed again.

Here, the result of detecting the real time position of the hand and the reference point is output on the screen of the display unit 170, so that the user can quickly and easily recognize the detection time of the reference point. The display unit 170 may correspond to a TV, a monitor, or the like. In this case, the display unit 170 may notify that the reference point is detected by an image, a phrase, or the like on the screen, and may be provided by a separate speaker and output by sound.

In addition, when the hand is detected, the display unit 170 may provide an icon (eg, a hand icon) corresponding to the image of the photographed hand on the screen to move in accordance with the movement of the hand. . That is, before and after the reference point of the hand is detected, an icon synchronized with the movement of the hand is displayed on the display unit 170 so that the recognized hand movement can be conveniently displayed. According to this, it is possible to easily check whether the desired hand gestures are performed properly by visually checking the hand gestures currently being taken.

In addition, when the reference point is detected, the display unit 170 may display the icon in a form different from that before the reference point is detected. For example, the display unit 170 may display an icon corresponding to the image of the hand even during the reference point detection process. While the reference point is detected, the shape of the icon is displayed in a dim or dotted line, and as soon as the reference point is detected, the icon is displayed in a clear and dark form so that the detection result of the reference point can be easily confirmed. In addition, the color of the icon may be expressed differently before and after the reference point detection.

According to the present invention as described above, the operation is recognized using an intuitive and simple algorithm rather than a probabilistic method. That is, according to the method and apparatus for recognizing a hand motion using a camera according to the present invention, the hand motion is first determined by first determining the degree of the hand's distance from the reference point, and then comparing and verifying the hand movement information and the previously stored hand motion DB according to the hand motion. There is an advantage to increase the recognition rate.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: gesture recognition device using a camera
110: reference point detection unit 120: storage unit
130: position comparison unit 140: trajectory comparison unit
150: trajectory DB 160: motion determination unit
170:

Claims (12)

Detecting a reference point of a hand photographed through a camera;
Storing movement information of a hand corresponding to the detected movement when a movement is detected from the detected hand;
Calculating a difference value between the current position of the hand on which the movement is detected and the reference point and comparing the difference with a preset first threshold value;
Comparing the hand trajectory information of the hand corresponding to the detected movement with a pre-stored hand trajectory DB when the difference is greater than or equal to the first threshold value; And
Determining a hand gesture matching the hand information of the hand from the trajectory DB,
Detecting the reference point of the hand,
Comparing the amount of change in the position of the hand of the current time point with respect to the previous time point with a preset second threshold value;
If the amount of change is less than the second threshold, comparing a dispersion value of the position information of the hand accumulated so far with a preset third threshold;
Determining that the hand is stationary when the dispersion value is less than the third threshold; And
And detecting the final position of the hand corresponding to the point in time at which the stationary state has passed a predetermined time as the reference point. delete The method according to claim 1,
And outputting a result of detecting the reference point as a sound or a screen. The method according to claim 3,
When the hand is detected, hand gesture recognition method using a camera provided on the screen to move the icon corresponding to the image of the hand photographed according to the movement of the hand. The method of claim 4,
When the reference point is detected, hand gesture recognition method using a camera to display the icon in a different form than before the reference point is detected. The method according to claim 1,
Comparing the trajectory,
When the difference value is greater than or equal to the first threshold value, at least one hand gesture candidate group similar to a moving direction of a hand exceeding the first threshold value is extracted from the hand gesture trajectory list in the trajectory DB, and the trace of the extracted hand gesture candidate group is determined. Comparing the hand trajectory information of the hand,
The hand movement direction is a hand gesture recognition method using a camera that means the movement direction of the hand with respect to the x, y, z axis. A reference point detector for detecting a reference point of a hand photographed through a camera;
A storage unit for storing the hand trajectory information corresponding to the detected movement when a movement is detected from the hand from which the reference point is detected;
A position comparison unit which calculates a difference value between the current position of the hand where the movement is detected and the reference point and compares the first threshold with a preset first threshold value;
A trajectory comparison unit comparing the trajectory information of the hand corresponding to the detected movement with a pre-stored trajectory DB for each hand motion when the difference is greater than or equal to the first threshold value; And
A motion determination unit for determining a hand motion that matches the hand information of the hand from the locus DB,
Wherein the reference point detection unit comprises:
Compare the amount of change in the position of the hand of the current time point with respect to the previous time point with a preset second threshold;
When the amount of change is less than the second threshold, the variance value of the position information of the hands accumulated so far is compared with a preset third threshold,
If the variance is less than the third threshold, determine that the hand is at rest;
And a camera for detecting a final position of a hand corresponding to a time point in which the stationary state has passed a predetermined time as the reference point. delete The method of claim 7,
And a display unit for outputting a result of detecting the reference point as a sound or a screen. The method according to claim 9,
The display unit includes:
When the hand is detected, the hand gesture recognition device using a camera provided on the screen to move the icon corresponding to the image of the hand photographed according to the movement of the hand. The method of claim 10,
The display unit includes:
When the reference point is detected, hand gesture recognition apparatus using a camera to display the icon in a different form than before the reference point is detected. The method of claim 7,
The trajectory comparison unit,
When the difference value is greater than or equal to the first threshold value, at least one hand gesture candidate group similar to a moving direction of a hand exceeding the first threshold value is extracted from the hand gesture trajectory list in the trajectory DB, and the trace of the extracted hand gesture candidate group is determined. Comparing the hand trajectory information of the hand,
The hand movement direction is a hand gesture recognition device using a camera that means the direction of the hand movement relative to the x, y, z axis.

Images