WO2016085122A1

WO2016085122A1 - Gesture recognition correction apparatus based on user pattern, and method therefor

Info

Publication number: WO2016085122A1
Application number: PCT/KR2015/011022
Authority: WO
Inventors: 성연식; 김필영; 김지원; 손준혁
Original assignee: 계명대학교 산학협력단
Priority date: 2014-11-28
Filing date: 2015-10-19
Publication date: 2016-06-02
Also published as: KR101596600B1

Abstract

The present invention relates to a gesture recognition correction apparatus based on a user pattern, and a method therefor. A gesture recognition correction method using a gesture recognition correction apparatus based on a user pattern comprises the steps of: storing numerical values of various body gestures and probability values for the numerical values; recognizing at least one body gesture of a user using data sensed by a sensor; converting the body gesture recognized by the sensor into a numerical value; comparing a probability value for the numerical value to a first reference value and, if the probability value is less than the first reference value, determining the body gesture recognized by the sensor as an error; and if the body gesture is determined to be an error, recognizing the recognized body gesture by correcting the body gesture to a gesture pattern with the highest probability value among stored gesture patterns.

Description

User pattern based motion recognition correction device and method

The present invention relates to an apparatus and method for correcting motion recognition based on a user pattern, and more particularly, a motion recognition correction device capable of increasing a recognition rate of a physical motion by determining and correcting an error occurring when a body motion is recognized through a sensor. And to a method thereof.

Conventionally, a human-machine interface is generally achieved through a keyboard, a remote controller, a mouse, or a keypad provided in an electronic device, but as a function of an information communication device is developed, a finger or a pen is brought into contact with a touch panel using a touch sensor. Touch-based systems are being utilized. However, this has a problem in that the touch is not properly made when a glove, water, or dust is on the hands, and recently, convenience functions that can adjust the device with only a minute motion of the user have been developed to maximize user convenience. .

In particular, the motion-based interface that allows the user to interact naturally with the computer through the user's intuitive movement, such as the user's body movements and gestures are attracting attention. In addition to three-dimensional motion, various expressions are possible and convenient to use, and there are various applications such as game / entertainment, medical / special treatment, and education.

However, the motion-based interface has a problem in that it is expensive and body motion recognition is not smooth. Recently, the number of sensors tends to be minimized in order to lower the product price of the body motion recognition device. As a result, the recognition rate of the body motion is lowered and the error occurrence rate is increased. In particular, since the sensor in the device is located only underneath, the parts covered by other objects or other body parts have a high recognition error rate, which makes it difficult for a user to perform a desired operation and causes inconveniences.

The background technology of the present invention is disclosed in Korean Patent Publication No. 10-2014-0089858 (published on July 16, 2014).

An object of the present invention is to provide a motion recognition correction apparatus and method for determining the recognition and correction of errors occurring when the body motion is recognized through the sensor to increase the recognition rate of the body motion.

According to an embodiment of the present invention for achieving the above technical problem, in the user pattern-based motion recognition correction device, a motion pattern storage unit for storing the numerical values of the various body movements and the probability values thereof, the sensor sensed by the sensor A motion recognition unit for recognizing at least one body motion of the user using data, a numerical conversion unit for converting a body motion recognized by the sensor into a numerical value, and comparing a probability value of the numerical value with a first reference value If the value is less than the first reference value, the error determining unit determines the physical motion recognized by the sensor as an error, and if the determination result is the error, the physical motion having the highest probability value among the stored motion patterns. And a motion correction unit for correcting and recognizing.

The motion pattern storage unit may store probability values calculated by applying a Bayesian algorithm to numerical values of successive physical motions as in the following equation.

Represents the Bayesian probability of one recognized body

The value is the numerical value of the body as recognized at time t-1,

The value represents the numerical value of the body recognized at t-2 hours.

The numerical conversion unit may digitize and store at least one of the recognized arm direction, leg direction, head direction and the degree of arm bending, the degree of leg bending, and the degree of trunk bending of the body.

The numerical conversion unit may obtain a numerical value of the recognized physical motion as follows.

Where h _t represents the numerical value of the body as recognized at time t,

Is

The floor function that is less than or equal to and is represented by the largest integer, and y is a variable used to increase the probability.

When the probability value for the numerical value is greater than or equal to the first reference value, the physical motion recognized by the sensor may be determined as the physical motion of the user.

If the highest probability value among the previously stored motion patterns is higher than a second reference value, the motion correction unit converts the body motion recognized by the sensor into a body motion having the highest probability value, and recognizes the highest motion value. If the probability value is less than or equal to the second reference value, the probability value may be recognized as a body motion recognized by the sensor.

The motion corrector may generate a control signal corresponding to the recognized body motion and transmit the generated control signal to the paired electronic device.

A motion recognition correction method using a user pattern based motion recognition correction device, comprising: storing numerical values and probability values of various body motions, at least one physical motion of a user using data sensed by the sensor Recognizing a body, converting a body motion recognized by the sensor into a numerical value, comparing a probability value for the numerical value with a first reference value, and if the body is smaller than the first reference value, Determining an operation as an error, and correcting and recognizing a motion as a body motion having the highest probability value among the stored operation patterns when it is determined as an error.

According to the present invention, the motion recognition correction apparatus determines an error on a motion recognized by a sensor based on a motion pattern of a user, recognizes a motion with a high probability value among existing motion patterns, and lowers a motion error rate. To perform the desired operation.

1 is a block diagram illustrating a motion recognition correction device according to an embodiment of the present invention.

2 is a block diagram of a motion recognition correction device according to an embodiment of the present invention.

3 is a flowchart illustrating a method of correcting hand gestures in a body motion according to an exemplary embodiment of the present invention.

4 is an exemplary diagram for describing a sequential hand gesture in a recognized body motion according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

As shown in FIG. 1, the gesture recognition correcting apparatus 200 according to an exemplary embodiment of the present invention may be implemented by being connected to the gesture recognition sensor 100 and the electronic device 300.

The motion recognition sensor 100 recognizes a movement and a position of a measurement target. To recognize this, the motion recognition sensor 100 may use a geomagnetism, an acceleration, a gyro sensor, or the like. Movement may be recognized by wearing gloves using an acceleration sensor, a gyro sensor, and a bending sensor, or a sensor may be worn on a camera system, an infrared reflector, a wrist, or a body, and a band equipped with a sensor may be recognized.

In addition, by using the existing camera mounted on the electronic device so that it can be driven only by software without additional hardware, it recognizes the motion or installs two or more cameras and measures the distance using the difference between the images viewed from each camera. Can be recognized. In addition, the motion may be recognized using Kinect structure, time of flight (ToF), Point Cloud, Body Gesture, etc., but is not limited thereto.

The motion recognition correction apparatus 200 may be manufactured and used as a USB memory and mounted between the motion recognition sensor and the electronic device, and may be separated from the electronic device so that the motion recognition correction device 200 does not need to be separately installed and may be used in various ways.

The motion recognition correcting apparatus 200 recognizes a user's body motion using the information sensed by the motion recognition sensor 100 and determines a recognition error of the recognized body motion based on a Bayesian probability. If it is determined that the recognition error, the corrected body motion is corrected, and a control signal corresponding to the corrected body motion or correctly recognized body motion is transmitted to the paired electronic device 300 to control the operation of the electronic device 300. .

The electronic device 300 is a device controlled according to the motion detected by the motion recognition sensor 100, and may be a human-machine interface such as a digital TV, a set-top box, a cellular phone, a tablet, a PC monitor, and the like. The device is not limited and can be applied. In addition, the electronic device 300 may receive a control signal from the motion recognition correcting apparatus 200 according to the detected motion and perform a function according to the control signal.

Motion recognition correction apparatus 200 according to an embodiment of the present invention is a motion pattern storage unit 210, motion recognition unit 220, numerical conversion unit 230, error determination unit 240, motion correction unit 250 It includes.

First, the operation pattern storage unit 210 stores numerical values and Bayesian probability values of various operation patterns of the user. Since the conditions vary according to the use environment such as the frequency and type of operation patterns used for each user, the operation pattern storage unit 210 learns and stores data (value values and Bayesian probability values) for the operation patterns suitable for each user. Can be.

That is, by repeatedly storing various body motion patterns of the user and applying the stored patterns to the Bayesian algorithm, the motion pattern storage unit 210 stores probability values through learning about not only one body motion but also continuous body motions. Can be.

That is, the operation pattern storage 210 stores the probability calculated through the Bayesian algorithm as shown in Equation 1 below.

Represents the Bayesian probability of one recognized body

Value is the numerical value of the body recognized at t-1 hours

The value represents the numerical value of the body as recognized at t-2 hours.

More specifically, as shown in Figure 4 to be described later

Is the first action,

Corresponds to the second operation.

The motion recognition unit 220 receives the data sensed by the motion recognition sensor 100 to recognize a user's body motion. For example, a hand gesture among the body motions of the user may be recognized by sensing data of various hand gestures such as two fingers or a fist. In addition, the motion recognition unit 220 may recognize a body motion that changes continuously.

The numerical conversion unit 230 converts at least one or more body motions recognized by the motion recognition unit 220 into numerical values. The conversion of the numerical value of the body motion may be set in various ways according to the sensor that recognizes the motion and the electronic device using the motion.

For example, the direction of the arm, leg, head, and the degree of bending of the body, the degree of bending of the leg, the degree of trunk bending, etc., or the direction of the hand, the direction of the palm, the position, the direction of the fingers, the degree of bending of the knuckles. It converts the numerical values according to the information received by the motion recognition sensor that is actually used for the position pointed by the finger and the like.

The error determination unit 240 obtains a probability value for the numerical value converted by the numerical conversion unit 230 from the operation pattern storage unit 210, and operates when the probability value for the numerical value is smaller than the first reference value. The body motion recognized by the recognition unit 220 is determined to be an error.

When the error determiner 240 determines that the error is an error, the operation corrector 250 searches for the highest probability value among the operation patterns stored in the operation pattern storage 210, and the highest probability value is greater than the second reference value. If it is large, it is recognized by correcting the body motion with the highest probability value. In addition, the motion corrector 250 transmits a control signal corresponding to the body motion corrected or recognized as the first body motion to the electronic device 300.

Hereinafter, a method of correcting hand gesture recognition among hand motions based on a user pattern according to an exemplary embodiment of the present invention will be described with reference to FIGS. 3 and 4. For convenience of description, the present invention will be described according to hand gestures among various body movements, but is not limited thereto. The motion recognition correcting apparatus 200 according to the embodiment of the present invention does not necessarily recognize and correct only one body part. That is, the motion recognition correcting apparatus 200 may recognize and correct two or more different body parts together.

3 is a flowchart illustrating a method for correcting a hand gesture in a body motion according to an exemplary embodiment of the present invention, and FIG. 4 is an exemplary diagram for describing a continuous hand gesture in a recognized body motion according to an exemplary embodiment of the present invention.

Hereinafter, for convenience of description, when the third operation is performed by the user in a state where the first operation and the second operation are already performed as shown in FIG. 4, it is determined whether an error is detected in the recognized third operation and determined as an error. For example, the process of correcting and recognizing the case will be described.

In addition, it is assumed that after the first operation and the second operation are performed, as shown in FIG. 4, a probability of performing three third operations is determined in advance through learning and Bayesian probabilities and stored in the operation pattern storage 210.

That is, when the third motion is a subsequent motion after the first motion of the fist and the second motion of pointing only the black finger, if the third motion is a motion of extending the palm of the palm toward the sky (3-1 motion), the probability is 50%. It is assumed that there is a 30% probability in the case of extending and stopping only (3-2 movement) and 20% in the case of fist-taking movement (3-3 movement).

First, the gesture recognition correcting apparatus 200 recognizes the third hand gesture through the data transmitted through the gesture recognition sensor 100 (S310). That is, the motion recognition unit 220 recognizes which motion the third hand gesture corresponds to by using the sensing information recognized by the motion recognition sensor.

For convenience of explanation, it is assumed that the gesture recognition unit 220 recognizes the third gesture as a fist fist (3-3).

In operation S320, the motion recognition correcting apparatus 200 converts the recognized hand motion into a numerical value.

At this time, the numerical value is to digitize at least one of the direction of the palm, the shape of the finger, whether the left or right hand and the degree of bending. For example, with respect to the palm direction, 0 indicates the bottom, 1 indicates the upward direction, '-' indicates the left direction, and '+' indicates the right direction. The finger of the left hand or the right hand may be represented by 1 to 5.

In this way, the shape of the hand gesture can be numerically converted by combining a number or a sign indicating the direction of the palm, the shape of the finger, whether the left hand or the right hand is, and the degree of bending.

In particular, the orientation value of the palm (

) Can be obtained as shown in Equation 2 below.

Where h _t represents the numerical value of the hand recognized at time t,

Is

It is a floor function that is less than or equal to the largest integer and y is a variable used to increase the probability. The direction of the hand is-

From +

Since the probability of the other direction having the same value is represented by a small value. So, to increase the probability, divide and multiply the predefined variable y.

For example, if h _t is measured as "1,2,3,4,5,6,7,8,9,10", if y is 3, h ' _t Converts to 0,0,3,3,3,6,6,6,9,9. Since 1 and 2 are the same group, 3, 4, and 5 can be grouped into the same group, so the probability of getting the same number can be increased. In other words, y also binds the value of h ' _t in a certain unit.

As described above, when the numerical value of the third operation recognized through the Bayesian probability is calculated, the motion recognition correcting apparatus 200 compares whether the probability value of the numerical value is smaller than the first reference value (S330).

Here, the first reference value is a lower limit threshold and is a variable that can be changed in design according to the recognition accuracy required by the user.

If the probability value of the recognized third motion is equal to or greater than the first reference value, the motion recognition correcting apparatus 200 determines the third motion recognized by the motion recognition sensor 100 as a correctly recognized hand motion. In operation S340, the third operation may be performed without further correction.

For example, when the first reference value is 10%, the error determination unit 240 because the probability value of the recognized 3-3 motion (fisting motion) is 20% greater than the first reference value. Determines a hand gesture (operation 3-3) recognized by the gesture recognition unit 220 as a third operation.

Meanwhile, when the probability value of the recognized third operation is smaller than the first reference value, the motion recognition correcting apparatus 200 determines that the recognized third operation is incorrectly recognized.

That is, according to the exemplary embodiment of the present invention, even if the third motion is recognized by the motion recognition sensor 100, if it is determined that the occurrence probability is lower than the first reference value based on the Bayesian probability, the gesture recognition process may be performed. It is determined that an error has occurred.

For example, when the first reference value is 25%, since the recognized third operation (operation 3-3) is less than the Bayesian probability (20%) of the first reference value, the motion recognition correcting apparatus 200 is recognized. The third operation (operation 3-3) is determined to be a recognition error.

If it is determined in step S340 that the recognition error, the motion recognition correction apparatus 200 after searching for the hand gesture having the highest probability value among the pre-stored operation patterns for the third operation after the first operation, the second operation, and then It is compared whether the high probability value is greater than the second reference value (S350).

Here, the second reference value is an upper limit and is a variable that can be changed in design according to the recognition accuracy required by the user, similarly to the first reference value. In addition, it is obvious that the second reference value is higher than the first reference value.

If the probability value of the hand gesture having the highest probability value is smaller than the second reference value, the gesture recognition correcting apparatus 200 may perform the hand gesture first recognized by the gesture recognition sensor 100 in operation S350 (operation 3-3). ) Is determined as the third operation in step S310.

That is, according to the embodiment of the present invention, when the probability value of the hand gesture having the highest probability value is the same as or smaller than the second reference value, it is determined that the reliability to be recognized as the hand gesture having the highest probability value is low. The motion recognized by the motion recognition sensor 100 is determined as a third motion.

More specifically through the above example, if the second reference value is 60%, since the probability of the 3-1 operation having the highest probability value is 50% and smaller than the second reference value, the motion recognition correcting apparatus ( 200 determines a hand motion (operation 3-3) recognized by the motion recognition sensor 100 as the third operation again as in S350.

On the other hand, when the probability value of the hand gesture having the highest probability value is greater than the second reference value, the motion recognition correcting apparatus 200 recognizes the hand gesture having the highest probability value as the third operation (S360).

That is, according to the embodiment of the present invention, when the probability value of the hand gesture having the highest probability value is larger than the second reference value, it is determined that the hand gesture having the highest probability value is high in recognition of the third gesture, The hand gesture having the highest probability value is determined as the third gesture.

For example, when the second reference value is 45%, since the probability of the 3-1 operation having the highest probability value is 50% and larger than the second reference value, the motion recognition correcting apparatus 200 may use the motion recognition sensor ( The 3-1 operation is determined as the third operation without being recognized as the hand gesture recognized by the operation 100).

As described above, according to the exemplary embodiment of the present invention, the motion recognition correcting apparatus 200 determines whether or not the motion of the hand gesture recognized by the motion recognition sensor 100 is substantially corrected through the first reference value and the second reference value. Will be decided.

In other words, if you want to make the hand gesture more concise and accurate, the first reference value will be set high and the second reference value will be set low. have.

Next, the motion recognition correcting apparatus 200 generates a control signal corresponding to the finally recognized motion and transmits it to the electronic device 300. That is, the motion recognition correcting apparatus 200 transmits a control signal corresponding to the motion first recognized by the motion recognition sensor 100 or the motion corrected by the hand motion having the highest probability value to the electronic device 300.

In the above example, the control signal corresponding to the combination of the first operation-second operation-finally recognized third operation is transmitted to the electronic device 300.

As described above, according to the exemplary embodiment of the present invention, the motion recognition correcting apparatus determines whether an error is detected in the motion recognized by the sensor based on the motion pattern of the user, and recognizes the motion as having a high probability value among the existing motion patterns. It lowers the operation error rate and allows the user to perform the desired operation.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

= Explanation of the sign =

100: motion recognition sensor 200: motion recognition correction device

300: electronic device 210: operation pattern storage unit

220: motion recognition unit 230: numerical conversion unit

240: error determination unit 250: motion correction unit

Claims

In the user pattern based motion recognition correction device,

An operation pattern storage unit which stores numerical values of various body movements and probability values thereof;

A motion recognition unit for recognizing at least one body motion of a user using data sensed by the sensor;

A numerical conversion unit for converting a physical motion recognized by the sensor into a numerical value;

An error determination unit that compares a probability value of the numerical value with a first reference value and determines that the physical motion recognized by the sensor is an error when the probability value of the numerical value is smaller than the first reference value; And

And a motion correcting unit correcting and recognizing a body motion having the highest probability value among the stored motion patterns when it is determined as an error.
The method of claim 1,

The operation pattern storage unit,

An apparatus for correcting motion recognition for storing probability values calculated by applying a Bayesian algorithm to numerical values of successive physical motions as follows:

Represents the Bayesian probability of one recognized body
The value is the body value recognized at t-1 hours
The value represents the numerical value of the body recognized at t-2 hours.
The method of claim 2,

The numerical conversion unit,

And at least one of the recognized arm direction, leg direction, head direction, and the degree of arm bending, the degree of leg bending, and the degree of trunk bending of the body.
The method of claim 3,

The numerical conversion unit,

A motion recognition correction device for calculating the numerical value of the recognized body motion as follows:

Where h t represents the numerical value of the body recognized at time t
Is
The floor function that is less than or equal to and represents the largest integer, and y is a variable used to increase the probability.
The method of claim 4, wherein

The error determining unit,

And a body motion recognized by the sensor as the body motion of the user when the probability value of the numerical value is greater than or equal to the first reference value.
The method of claim 5,

The motion correction unit,

If the highest probability value among the previously stored motion patterns is higher than a second reference value, the body motion recognized by the sensor is converted into a body motion having the highest probability value and recognized. 2 or less, the motion recognition correction device recognizing a body motion recognized by the sensor.
The method of claim 6,

The motion correction unit,

And generating a control signal corresponding to the recognized body motion and transmitting the generated control signal to the paired electronic device.
In the motion recognition correction method using a user pattern-based motion recognition correction device,

Storing numerical values of various body movements and probability values thereof;

Recognizing at least one body motion of a user using data sensed by the sensor,

Converting a body motion recognized by the sensor into a numerical value;

Comparing the probability value for the numerical value with a first reference value, and determining the physical motion recognized by the sensor as an error if it is smaller than the first reference value; And

And correcting and recognizing a body motion having the highest probability value among the stored motion patterns if it is determined to be an error.
The method of claim 8,

The storing of the numerical values of the physical motions and the probability values thereof may include:

A method of correcting motion recognition for storing probability values calculated by applying a Bayesian algorithm to numerical values of successive physical motions as follows:

Represents the Bayesian probability of one recognized body,
The value is the numerical value of the body as recognized at time t-1,
The value represents the numerical value of the body recognized at t-2 hours.
The method of claim 9,

The step of converting the body motion into a numerical value,

And recognizing and storing at least one of the recognized arm direction, leg direction, head direction, and arm bending degree, leg bending degree, and body bending degree of the recognized body.
The method of claim 10,

The step of converting the body motion into a numerical value,

A motion recognition correction method for calculating the numerical value of the recognized body motion as follows:

Where h t represents the numerical value of the body as recognized at time t,
Is
It is a floor function that is less than or equal to and is represented by the largest integer, and y is a variable used to increase the probability.
The method of claim 11,

The determining of the recognized physical motion as an error may include:

And if the probability value for the numerical value is greater than or equal to the first reference value, determining the body motion recognized by the sensor as the body motion of the user.
The method of claim 12,

Recognizing and correcting the physical motion having the highest probability value may include:

If the highest probability value among the previously stored motion patterns is higher than a second reference value, the body motion recognized by the sensor is converted into a body motion having the highest probability value and recognized. 2 or less, the motion recognition correction method for recognizing a body motion recognized by the sensor.
The method of claim 13,

Recognizing and correcting the physical motion having the highest probability value may include:

And generating and transmitting a control signal corresponding to the recognized body motion to the paired electronic device.