KR101234095B1 - Gesture cognitive device and method for recognizing gesture thereof - Google Patents

Abstract

A gesture recognition apparatus is provided. The motion recognition apparatus includes an input device including a light source and an inertial sensor, and a motion recognition unit that detects a change in the position of the light source during the user's motion section determined from the sensing signal of the inertial sensor and extracts a motion trajectory of the user.

Inertial sensor, light source, motion recognition

Description

Gesture cognitive device and method for recognizing gesture

The present embodiment relates to a technology for recognizing a user's motion, and more particularly, to a motion recognition device in which a light source and an inertial sensor are combined, and a motion recognition method using the same.

The present invention is derived from a study conducted as part of the IT source technology development project of the Ministry of Knowledge Economy [Task Management Number: 2008-F-039-01, Task name: Development of human-robot interaction mediated technology].

The camera-based motion input method is a method of recognizing motion by capturing motion images of a user as continuous images to recognize an object in the image and capturing the motion of the object in the continuous image. This method is a limitation of image recognition technology and shows a clear limit in terms of freedom of movement and recognition rate. In particular, it is difficult to apply due to the reduction in recognition rate due to lighting conditions and background complexity.

Recently, a technique for detecting a motion trajectory and recognizing motion based on an infrared light source tracking method using an infrared camera has been introduced. This method has a simple low cost device configuration and can measure motion trajectories relatively accurately, which is advantageous for motion input. In particular, unlike the inertial data, the difference between users is not great because it directly tracks the motion drawn by the motion.

In the conventional light source tracking method, a user attaches or grabs a light source to a body and moves the light source, the camera tracks the movement of the light source to measure the trajectory, and any data processing system recognizes the movement pattern shown in the trajectory. Recognize the user's actions.

However, the conventional light source tracking method is difficult to distinguish between the user's operation and the operation. For example, when the light source continuously emits light, the user's trajectory is measured as a continuous line without interruption, so it is difficult to automatically find at which point of the trajectory the operation is finished.

In the prior art, in order to solve the above problem, the trajectory is artificially broken so that the user can control the light source only during operation by using a short circuit means such as a button to distinguish the user's operation. However, this method can distinguish the user's motions, but it does not allow the user to concentrate on only the motions, thereby impairing the convenience of motion input. In particular, under a situation in which the operation input is frequent, the degree of convenience is increased, and thus the accuracy of the operation input may be lowered.

An object of the present invention is to provide a gesture recognition apparatus that can increase the accuracy of user gesture recognition.

Another object of the present invention is to provide a user gesture recognition method using a gesture recognition apparatus.

According to an aspect of the present invention, there is provided a gesture recognition apparatus including a light source and an inertial sensor, which detects a change in the position of a light source during a user operation period determined from a sensing signal of an input device and an inertial sensor. It includes a motion recognition unit for extracting the motion trajectory of the.

According to another aspect of the present invention, there is provided a method for recognizing a motion, which includes determining a user's operation section from a sensing signal output from an inertial sensor and changing a position of a light source according to a determination result of the user's operation section. Detecting and extracting a motion trajectory of the user.

The motion recognition apparatus and the motion recognition method using the same according to the present invention are configured to include a light source and a sensor in an input device worn or possessed by a user, and the user motion trajectory according to a change in the position of the light source or the light source according to the sensing signal of the sensor. By automatically controlling the operation of the motion recognition unit for outputting the user, the user motion trajectory can be easily separated, thereby improving the inaccuracy of the user motion recognition.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the embodiments of the present invention, reference should be made to the accompanying drawings, which illustrate embodiments of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

FIG. 1 is a schematic block diagram of an apparatus for recognizing a motion according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a signal waveform and a user motion determination according to a sensing result of the sensor shown in FIG. 1.

Referring to FIG. 1, the gesture recognition apparatus 100 according to the present exemplary embodiment may include an input device 110 and a gesture recognition unit 120.

The input device 110 includes a light source 111 for generating a visual signal, for example, light LS, and a sensor 113 for sensing a user's movement, and may be worn or possessed by the user.

The sensor 113 may be an inertial sensor capable of sensing a change in inertia force according to a user's movement and outputting a sensing signal SS. For example, an acceleration sensor, an angular velocity sensor, an inclination sensor, a gravity sensor, a geomagnetic sensor Or a gyro sensor.

The motion recognition unit 120 may include a motion determination unit 121, a light source tracking unit 123, and a trajectory processing unit 125.

The operation determination unit 121 may determine the operation state of the user based on the sensing signal SS output from the sensor 113 of the input device 110, and may determine the user operation section accordingly.

The operation determining unit 121 may output a control signal for controlling at least one operation among the light source 111, the light source tracking unit 123, and the trajectory processing unit 125 according to the determination result. The control signal may be an enable / disable signal (ES / DS).

According to an embodiment, the operation determination unit 121 outputs the enable / disable signal ES / DS according to a result of the operation state determination of the user to the light source 111, and the light source 111 is an enable / disable signal. It may be turned on or off in response to (ES / DS).

According to another exemplary embodiment, the operation determination unit 121 outputs the enable / disable signal ES / DS to the light source tracking unit 123 according to the user's operation state determination result, and the light source tracking unit 123 is in It may or may not be operated in response to the enable / disable signal ES / DS. The light source tracking unit 123 may track and detect the position of the light source 111 in response to the enable signal ES of the operation determination unit 121.

According to another exemplary embodiment, the operation determiner 121 outputs the enable / disable signal ES / DS to the trajectory processing unit 125 according to a result of determining the operation state of the user, and the trajectory processing unit 125 enables the trajectory processing unit 125. It may or may not be operated in response to the / disable signal ES / DS. The trajectory processing unit 125 extracts and outputs the user's operation trajectory TS from the light source position signal PS output from the light source tracking unit 123 in response to the enable signal ES of the operation determination unit 121. Can be. The motion recognition apparatus 100 may recognize a user's motion based on the motion trajectory TS output from the trajectory processing unit 125.

In the present exemplary embodiment, an example in which the operation determining unit 121 is included in the operation recognizing unit 120 is illustrated, but the present invention is not limited thereto. For example, according to various embodiments of the gesture recognition apparatus 100, the gesture determination unit 121 may be included in the input device 110 or may be configured independently. When the operation determiner 121 is included in the input device 110, it may be preferable that the turn-on or turn-off of the light source 111 is controlled according to the determination result of the operation determiner 121.

1 and 2, the operation determiner 121 analyzes the sensing signal SS output from the input device 110, and determines the operation state of the user and the user's operation section according to the analysis result. can do.

According to an exemplary embodiment, the sensing signal SS may be an acceleration signal that changes according to a user's motion, and thus the sensor 113 of the input device 110 may be a three-axis acceleration sensor. The 3-axis acceleration sensor may output an sensing signal SS by sensing an acceleration signal that is changed in x, y, and z axes according to a user's motion.

The operation determiner 121 may receive the sensing signal SS from the sensor 113 during the time t0-t5 of the time axis T.

The operation determiner 121 may convert the sensing signal SS into an energy change amount EV. For example, the operation determiner 121 calculates a standard deviation of the sensing signal SS of each of the plurality of time intervals L0 to L4 to calculate an operating energy value of the sensing signal SS, and calculates each calculated operation. By calculating the standard deviation of the energy values, the energy change amount EV for the sensing signal SS of each of the plurality of sections L0 to L4 may be calculated.

The calculated energy change amount EV may be compared with each of the plurality of reference levels REF1 and REF2, and the operation state of the user and the user's operation section may be determined according to the comparison result.

For example, as in the time t0-t1 and the time t4-t5 of the time axis T, if the calculated energy change amount EV is smaller than the first reference level Ref1, the operation determination unit 121 may operate the user. Can be determined as the end.

Further, as in the time t2-t3 of the time axis T, if the calculated energy change amount EV is greater than the first reference level Ref1 or the same within an error range and smaller than the second reference level Ref2, The operation determination unit 121 may determine the operation state of the user as the standby.

In this case, the operation determination unit 121 may perform a counting operation during the time t2-t3 of the time axis T. The operation determiner 121 may compare the counting value CS according to the counting operation with a third reference level, for example, the reference counting level Cref. According to the comparison result, the operation determination unit 121 may determine the operation state of the user as the end of the standby. To this end, the operation determination unit 121 may further include a counter (not shown).

As in the time t1-t2 and the time t3-t4 of the time axis T, if the calculated energy change amount EV is greater than the second reference level Ref2 or the same within an error range, the operation determination unit 121 The operation state of the user may be determined as a start.

The operation determining unit 121 may enable / disable signals (ES) capable of controlling at least one operation among the light source 111, the light source tracking unit 123, or the trajectory processing unit 125 according to a result of determining the operation state of the user. / DS) can be printed.

Referring back to FIG. 1, the light source tracking unit 123 may change a position of the light source 111, for example, a position change of the light LS output from the light source 111, according to the determination result of the operation determination unit 121. It can track and detect, and output the light source position signal PS according to a detection result.

The trajectory processing unit 125 may extract the user motion trajectory TS based on the light source position signal PS output from the light source tracking unit 123, and the user's motion based on the extracted user motion trajectory TS. Can be recognized.

FIG. 3 is a flowchart illustrating a motion recognition according to an operation of the motion recognition apparatus shown in FIG. 1. In this embodiment, S 'and S are the user's operating state, EV is the energy change amount, Ref1 is the first reference level, Ref2 is the second reference level, Cref is the reference counting level, CS is the counting value.

1 to 3, when the user performs a predetermined movement while wearing or holding the input device 110, the sensor 113 of the input device 110 may sense the sensing signal SS according to the movement of the user. Outputs The sensing signal SS is input to the operation determining unit 121 (S10).

The operation determination unit 121 calculates an energy change amount EV from the sensing signal SS (S20). For example, the operation determiner 121 obtains an operating energy value from the sensing signal SS of each of the plurality of sections L0 to L4, and calculates a standard deviation of each operating energy value, thereby calculating the energy change amount EV for each section. Can be calculated.

Subsequently, the previously defined operation state S 'of the user may be determined (S31, S33, and S35), and the calculated energy change amount EV may be compared with at least one reference level Ref1 or Ref2 (S41). , S43).

If the previously defined user operation state S 'is the operation end (S' = 0) (S31), the operation determination unit 121 may compare the energy change amount EV and the second reference level Ref2 ( S41).

As a result of the comparison, when the energy change amount EV is greater than the second reference level Ref2 or the same within an error range, the operation determination unit 121 determines the current user operation state S as the start of operation (S = 1). It may be (S51).

However, when the energy change amount EV is smaller than the second reference level Ref2, the operation determiner 121 may compare the energy change amount EV with the first reference level Ref1 (S43).

As a result of the comparison, when the energy change amount EV is greater than the first reference level Ref1 or the same within an error range, the operation determination unit 121 determines the current user operation state S as operation standby (S = 2). It may be (S53).

However, when the energy change amount EV is smaller than the first reference level Ref1, the operation determination unit 121 may determine the current user operation state S as the operation termination (S = 0) (S55).

When the current operation state S of the user is determined, the operation determining unit 121 controls at least one operation among the light source 111, the light source tracking unit 123, or the trajectory processing unit 125 according to the determination result, and the sensor The sensing signal SS is input again from 113 (S10).

If the previously defined user operation state S 'is not an operation end (S' = 0) but an operation start (S '= 1) (S33), the operation determination unit 121 may determine the energy change amount EV described above. In operation S41, the second reference level Ref2 may be compared.

If the user operation state S 'previously defined is operation standby (S' = 2) rather than operation termination (S '= 0) and operation start (S' = 1) (S35), the operation determination unit 121 Initializes the counting value CS to 0 (S37) and performs a counting operation during the operation waiting time. Next, the counting value CS may be compared with the reference counting level Cref (S45).

As a result of the comparison, when the counting value CS is greater than the reference counting level Cref or the same within the error range, the operation determining unit 121 determines that the current user operation state S is the end of the operation (S = 0). It may be (S55). When the current operation state S of the user is determined, step S10 of receiving the sensing signal SS from the sensor 113 described above may be performed.

However, when the counting value CS is smaller than the reference counting level Cref, the counting value CS may be compared with the reference counting level Cref while increasing the counting value CS sequentially (S60). According to the comparison result, the current operation state S of the user may be determined (S55) or the counting value CS may be increased again to compare with the reference counting level Cref (S45).

Meanwhile, if the user operation state S 'previously defined is not an operation end (S' = 0), an operation start (S '= 1), and an operation wait (S' = 2), the gesture recognition apparatus 100 An error message may be output (S57).

FIG. 4 is a diagram illustrating an embodiment of user gesture recognition of the gesture recognition apparatus illustrated in FIG. 1.

1 and 4, the operation determiner 121 calculates an energy change amount EV for each of the plurality of sections L1 to L5, and calculates the calculated energy change amount EV by a plurality of reference levels. By comparing with each of (Ref1, Ref2), it is possible to determine the operation state of the user.

In this case, the operation determining unit 121 may adjust the operation section determination criteria of the user by adjusting the threshold values of each of the plurality of reference levels Ref1 and Ref2.

As shown in FIG. 4, when a user performs a write operation on the number 5 in a three-dimensional space, the operation determiner 121 according to the magnitude of each of the threshold values of the plurality of reference levels Ref1 and Ref2. The operation state of the user may be determined by dividing the stroke unit or may be determined by dividing the operation unit.

Here, the motion congestion section between strokes in stroke unit divisions is shorter than the motion congestion section between motions in motion unit divisions. Therefore, in order to classify the operation state of the user in the unit of stroke, the operation determination unit 121 may adjust the threshold value of each of the plurality of reference levels Ref1 and Ref2 to be smaller than in the case of classifying the operation unit.

The operation determination unit 121 may determine the operation state of the user as the start of the operation during the first period L1 to the fourth period L4 of the sensing signal SS, and the light source tracking unit 123 may move the user. The light source position signal PS may be detected. The trajectory processing unit 125 may extract the user motion trajectory TS based on the light source position signal PS.

The gesture recognition apparatus 100 may recognize a user's motion, for example, a writing operation on the number 5, based on the user motion trajectory TS output from the trajectory processing unit 125.

On the other hand, the operation determiner 121 may determine the user's operation state as the operation standby during the fifth section L5 in the fourth section L4 of the sensing signal SS. In this case, the operation determining unit 121 may compare the counting value CS counting the operation waiting time with the reference counting level Cref, and determine the operation state of the user as the end of the operation according to the comparison result.

If it is determined by the operation determining unit 121 that the user's operation state is standby or termination, at least one of the light source 111, the light source tracking unit 123, or the trajectory processing unit 125 is output from the operation determination unit 121. It may be disabled by the disable signal DS.

The operation determination unit 121 may determine the operation state of the user again from the sensing signal SS output from the sensor 113 according to a later user operation.

Although the content of the present invention has been described with reference to one embodiment illustrated in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. will be. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS A brief description of each drawing is provided to more fully understand the drawings recited in the description of the invention.

1 is a schematic block diagram of a gesture recognition apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating signal waveforms according to sensing results of the sensor illustrated in FIG. 1 and determining a user's motion accordingly.

FIG. 3 is a flowchart illustrating a motion recognition according to an operation of the motion recognition apparatus shown in FIG. 1.

FIG. 4 is a diagram illustrating an embodiment of user gesture recognition of the gesture recognition apparatus illustrated in FIG. 1.

Claims (16)

delete An input device including a light source and an inertial sensor; A motion recognition unit for detecting a change in the position of the light source and extracting a motion trajectory of the user during the user motion section determined from the sensing signal of the inertial sensor; And And an operation determination unit configured to calculate an energy change amount from the sensing signal and determine the user operation section according to an operation state of the user by comparing the calculated energy change amount with at least one reference level. The method of claim 2, wherein the at least one reference level comprises a first reference level and a second reference level, The operation determining unit, If the amount of energy change is greater than or equal to the second reference level, the operation state of the user is determined to be the start of operation, and if the amount of energy change is less than the second reference level, the operation state of the user is one of waiting for operation or ending operation. Motion recognition device for determining. The method of claim 3, wherein the operation determination unit, If the amount of energy change is greater than or equal to the first reference level and is less than the second reference level, the operation state of the user is determined as the operation standby, and if the energy change amount is less than the first reference level, the operation state of the user Gesture recognition unit for determining that the operation is terminated. The method of claim 3, wherein the operation determination unit, A counter for counting operation during the operation waiting time after the operation state of the user is determined to be the operation waiting, further comprising: And the operation determining unit determines the operation state of the user to be the end of the operation in the operation standby when the counting value of the counter is greater than or equal to a reference counting level. The method according to claim 2, And the operation determining unit determines the operation state of the user by the stroke unit or the operation unit by adjusting the threshold value of the at least one reference level. The method of claim 6, The operation determining unit may adjust the threshold value when the operation state of the user is divided into the stroke units to have a size smaller than the threshold value when the operation state of the user is divided into the operation units. . An input device including a light source and an inertial sensor; And And a motion recognition unit configured to extract a user's motion trajectory by detecting a change in the position of the light source during the user's motion determined from the sensing signal of the inertial sensor. The input device, An operation determination unit configured to calculate an energy change amount from the sensing signal and determine the user operation section according to the calculated energy change amount; The light source is controlled to be turned on or turned off according to the determination result of the operation determining unit, and the motion recognition unit detects a change in the position of the light source when the light source is turned on to extract the operation trajectory of the user. Motion recognition device. An input device including a light source and an inertial sensor; And And a motion recognition unit configured to extract a user's motion trajectory by detecting a change in the position of the light source during the user's motion determined from the sensing signal of the inertial sensor. The motion recognition unit, An operation determination unit calculating an energy change amount from the sensing signal and determining the user operation section according to the calculated energy change amount; A light source tracking unit detecting a change in position of the light source according to the determination result; And And a trajectory extraction unit configured to extract the operation trajectory of the user according to the detection result. One of the light source tracker and the trajectory extractor is enabled or disabled according to a determination result of the operation determiner. delete Determining a user operation section from the sensing signal output from the inertial sensor; And Detecting a change in the position of the light source according to the determination result and extracting a user's motion trajectory; Determining the user operation section, Calculating an amount of energy change from the sensing signal; And And comparing the calculated energy change with at least one reference level, and determining an operation state of the user according to a comparison result. The method of claim 11, wherein the at least one reference level comprises a first reference level and a second reference level, The determining of the operation state of the user according to the comparison result, If the amount of energy change is greater than or equal to the second reference level, the operation state of the user is determined to be the start of operation, and if the amount of energy change is less than the second reference level, the operation state of the user is one of waiting for operation or ending operation. Determination of motion recognition. The method of claim 12, wherein determining the operation state of the user according to the comparison result, If the amount of energy change is greater than or equal to the first reference level and is less than the second reference level, the operation state of the user is determined as the operation standby, and if the energy change amount is less than the first reference level, the operation state of the user And determining that the operation has ended. The method of claim 12, wherein determining the operation state of the user according to the comparison result, Counting an operation waiting time after the operation state of the user is determined to be the operation waiting: Comparing the counting value with a reference counting level; And If the counting value is greater than or equal to the reference counting level, determining the operation state of the user as the end of the operation in the operation standby. The method of claim 11, The determining of the user operation section may further include adjusting a threshold of the at least one reference level. The determining of the operation state of the user based on the comparison result may include determining the operation state of the user by a stroke unit or an operation unit according to the at least one reference level at which the threshold value is adjusted. 16. The method of claim 15, The adjusting of the threshold value of the at least one reference level may include: the threshold value when the operation state of the user is divided into the stroke units and the threshold value when the operation state of the user is divided into the operation units. Motion recognition method to adjust to have a smaller size.

Images