KR20140144560A - Method for operating a gesture recognition apparatus - Google Patents

Abstract

According to an embodiment of the present invention, the operating method of a gesture recognition apparatus comprises: a step of recognizing a gesture of a user; a step of comparing the recognized gesture with a database and determining a language corresponding to the recognized gesture; a step of converting the determined language into a speech signal; and a step of outputting the speech signal. As a result, the present invention can recognize a gesture of a user, convert the gesture into a speech, and output the speech and, in addition, can convert the gesture into a speech of another language and output the speech, thereby improving convenience of use for the user.

Description

[0001] The present invention relates to a gesture recognition apparatus,

The present invention relates to a gesture recognition apparatus and an operation method thereof, and more particularly, to a gesture recognition apparatus and a gesture recognition method that can recognize a gesture of a user, And a method of operating the same.

The functions and services of electronic devices such as portable terminals and TVs are becoming increasingly diversified.

However, in using such electronic devices, the user has to input complicated operations using a terminal or a remote control device, so that the user can use the desired functions.

In order to improve the user interface, there is an increasing research on techniques for recognizing a user's gesture. In addition, there is an increasing research on techniques that can improve user convenience by using gesture recognition technology in addition to command input.

An object of the present invention is to provide a gesture recognition device and a method for operating the gesture recognition device capable of recognizing a user's gesture, converting the output of the gesture into voice, and converting and outputting the same to another language.

According to another aspect of the present invention, there is provided a method of operating a gesture recognition device, comprising: recognizing a gesture of a user; comparing the recognized gesture with a database to determine a language corresponding to the recognized gesture; Converting the identified language into a voice signal, and outputting the voice signal.

According to the present invention, the gesture of the user can be recognized and converted into voice, output to other languages, and user convenience can be improved.

1 is a flowchart illustrating an operation method of a gesture recognition apparatus according to an exemplary embodiment of the present invention.
2 is a diagram for explaining a gesture recognition process according to an embodiment of the present invention.
3 is a flowchart illustrating an operation method of a gesture recognition apparatus according to an exemplary embodiment of the present invention.
4 and 5 are diagrams for explaining a method of operating a gesture recognition apparatus according to an embodiment of the present invention.
6 is an internal block diagram of a gesture recognition apparatus according to an embodiment of the present invention.
7 is an internal block diagram according to an embodiment of the recognition unit of FIG.
8A to 8D are diagrams for explaining a method of operating a gesture recognition apparatus according to an embodiment of the present invention.
9 is a flowchart illustrating an operation method of a gesture recognition apparatus according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

If a person who speaks a language is unable to speak, it is difficult to communicate and he / she knows a foreign language.

Also, if you want to talk to someone who can not speak, you have to learn sign language.

The present invention can provide a gesture recognition device and a method of operating the same that can help smooth conversation with another party having difficulty in conversation using the gesture recognition technology.

FIG. 1 is a flowchart illustrating an operation method of a gesture recognition apparatus according to an exemplary embodiment of the present invention. FIG. 2 is a flowchart illustrating a gesture recognition process according to an exemplary embodiment of the present invention. Referring to FIG.

1, an operation method of a gesture recognition apparatus according to an embodiment of the present invention first recognizes a gesture of a user (S110)

According to an embodiment of the present invention, at least one of the movements of the user's hands or fingers may be recognized through the electromyogram sensor 210, as shown in FIG.

Alternatively, the gesture recognition step (S110) may recognize at least one of a movement of the user's hand or fingers through a six-axis sensor. FIG. 2 (b) shows an example in which a six-axis sensor is provided in an arm band type gesture recognition device. However, the gesture recognition device according to the embodiment of the present invention is not limited to this form. For example, the recognition unit having a sensor for gesture recognition may be various types such as a clock type, an arm band type, and a chip type. In addition, the electromyogram sensor 210 may be of various types such as a clock type, an arm band type, a chip type, and the like.

According to an embodiment of the present invention, when a finger is moved or hydrated as if touching a keyboard, a motion of a user's hand and / or a finger can be detected through an electromyographic sensor or a six-axis sensor.

Meanwhile, the gesture recognition step (S110) may include recognizing an electromyogram of the user's fingers through the electromyogram sensor 210, removing noise of the recognized electromyogram data, determining, based on the recognized electromyogram data, And combining the movements of the fingers. Meanwhile, a specific embodiment using the electromyogram sensor 210 will be described later with reference to Figs. 8A to 9.

Meanwhile, the gesture recognition step (S110) may include collecting operation data of the user through various sensing means, and estimating the gesture of the user by analyzing the collected data.

Meanwhile, the gesture recognition device according to the present invention may compare the recognized gesture with a database to determine a language corresponding to the recognized gesture (S120), and convert the identified language into a voice signal. S130) Here, the database may be stored and managed in the form of a mapping table of a language and a word corresponding to a gesture, and may be provided in a gesture recognition device or an external server.

Meanwhile, the speech signal conversion step (S130) may convert the speech signal into a speech signal corresponding to a different language of the same or similar meaning to the identified language.

On the other hand, the language corresponding to the recognized gesture may be changed or a plurality of languages may be used according to the setting of the user. For example, the user inputs a gesture corresponding to the mother tongue, but the gesture recognition device can determine the mother tongue of the user and switch the language to a foreign language of the same or similar word as the word determined in the mother tongue.

Then, the voice signal is output (S140). In the case where the gesture recognition device has a speaker, voice can be transmitted through the speaker directly to the conversation. Alternatively, the voice signal may be transmitted to an output means such as an external speaker, a portable terminal, or the like so that the corresponding device can actually output the voice.

FIG. 3 is a flowchart illustrating an operation method of a gesture recognition apparatus according to an embodiment of the present invention. FIGS. 4 and 5 are views for explaining an operation method of a gesture recognition apparatus according to an embodiment of the present invention.

Referring to the drawings, a gesture recognition device can capture movements of a finger and a wrist using an electromyogram, a six-axis sensor, or the like (S311). For example, Lt; / RTI >

In addition, the movement of the finger can be captured using the EMG (S312). For example, it is possible to detect the operation of hitting the user's virtual keyboard by using an EMG sensor or the like.

Meanwhile, the gesture recognition device collects data such as X, Y, and Z axis movements (angles) and EMG data (S320), and analyzes the data to estimate movements of the finger and / or wrist (S330)

Thereafter, the gesture recognition device may convert the gesture into the identified language by referring to the motion-language data stored in the database (DB) (S340). The database (DB) may also include data related to the sign language gesture .

Meanwhile, if necessary, for example, if conversation with another foreigner is required, the conversion may further include a step of converting into another foreign language (S350)

After the language conversion, the gesture recognition device can control the voice signal to be actually outputted through the speaker or the external output means (S360)

For example, when a user who can not talk talks with another person, the gesture recognition device inputs the gesture 411 corresponding to sign language or the virtual keyboard 420, So that the voice 520 corresponding to the gesture of the user can be actually output. In addition, the conversation partner can smoothly proceed with the conversation while listening to the output voice 520 instead of the user's gesture and in response (530).

Alternatively, when the user speaks with a person who can not speak the user or when the foreigner can not understand the pronunciation of the user, the gesture 411, 420 is input to the speaker 510, . In addition, the conversation partner can smoothly proceed with the conversation while listening to the output voice 520 instead of the user's gesture and in response (530).

FIG. 6 is an internal block diagram of a gesture recognition apparatus according to an embodiment of the present invention, and FIG. 7 is an internal block diagram according to an embodiment of the recognition unit of FIG. 8A to 9 are views for explaining a method of operating a gesture recognition apparatus according to an embodiment of the present invention.

6, the gesture recognition apparatus according to an embodiment of the present invention may include a gesture recognition unit 610, a control unit 620, a storage unit 630, an interface unit 640, and a speaker 650 have.

The gesture recognition unit 610 may include various sensors for recognizing the gesture of the user. For example, the gesture recognition unit 610 may include at least one of sensor means such as a six-axis sensor, an acceleration sensor, and an EMG sensor.

The control unit 620 can control the overall operation in the gesture recognition device.

The storage unit 630 may store a program for each signal processing and control in the control unit 620, and may store signal processed video, audio, or data signals.

Meanwhile, the storage unit 630 may include a database used for discriminating a language corresponding to a gesture motion of the user. It may also store at least one country's sign language gestures and language tables.

The interface unit 640 may provide an interface for connecting to a wired / wireless network including an Internet network, and may communicate with a physically separated sensor or the like or may communicate with an external device.

The control unit 620 may process the input data through the gesture recognition unit 610 and the interface unit 640 to generate and output a signal for video or audio output.

The video signal and audio signal processed by the control unit 620 can be outputted through the speaker 650. [ Alternatively, the video signal and the audio signal processed by the control unit 620 can be transmitted to an external device such as a portable terminal through the interface unit 640.

FIG. 7 shows an example of a specific configuration of a gesture recognition unit when recognizing a gesture using an electromyogram. FIGS. 8A to 8D and 9 are diagrams for explaining a specific example of a gesture recognition process using EMG data to be.

Referring to the drawings, an electromyogram sensor and an amplifier (EMG Amplification Device) 710 are connected to a specific position of a user's arm, for example, channel 1 (Ch.1), channel 2 (Ch.2) ) Are connected to measure the EMG data of each muscle.

In one channel, the EMG data is measured in waveform as shown in FIG. 8B, and the data in the A / D converter 720 can be converted into digital data. On the other hand, FIG. 8C illustrates data measured on the three channels.

The signal processing unit 730 may include a time sample extraction unit 731, a feature extraction unit 732, and a transformation unit 730 so that the collected EMG signal can be accurately extracted After the noise is removed, data at a predetermined point in time can be extracted as shown in FIG. 8D.

The classifier 740 can then estimate movement of the hand by combining the movements of the fingers. On the other hand, according to the embodiment, the controller 620 can estimate the motion instead of the classifier 740. [

For example, it is determined from the data of channel 1 (Ch. 1) measuring the electromyogram of the second finger that the second finger is comfortably sagged, and from the data of channel 2 (ch.2) measuring the electromyogram of the third finger It is judged that the third finger is laid comfortably, and from the data of the channel 3 (Ch.3) measuring the electromyogram of the thumb, the thumb can be grasped and judged as a force.

Thus, it is possible to judge that the hand motion of the user is holding something thin.

9, it is determined that the second finger is grasped from the data of the channel 1 (Ch.1) that senses the movement of the second finger, and the channel 2 Ch. 2), and judges that the third finger is grasped from the data of the channel 3 (Ch.3) in which the movement of the thumb is sensed. You can judge it as a form holding something like an elliptical shape.

According to the present invention, the gesture of the user can be recognized and converted into voice, output to other languages, and user convenience can be improved.

The gesture recognition apparatus and the operation method thereof according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments can be applied to all or some of the embodiments May be selectively combined.

The method of operating the gesture recognition apparatus of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by a processor included in the gesture recognition apparatus. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

610:
620:
630:
640:
650: Speaker

Claims (6)

Recognizing a user's gesture;
Comparing the recognized gesture with a database to determine a language corresponding to the recognized gesture;
Converting the identified language into a voice signal; And
And outputting the voice signal. The method according to claim 1,
The gesture recognition step may include recognizing an electromyogram of the user's fingers through an electromyogram sensor, removing noise of the recognized electromyogram data, and combining the motions of the fingers based on the recognized electromyogram data The gesture recognition apparatus comprising: The method according to claim 1,
Wherein the gesture recognition step recognizes at least one of movement of the user's hand or fingers through an electromyogram sensor. The method according to claim 1,
Wherein the gesture recognition step recognizes at least one of movement of the user's hand or fingers through a six-axis sensor. The method according to claim 1,
Wherein the gesture recognition step includes collecting operation data of the user and estimating a gesture of the user by analyzing the collected data. The method according to claim 1,
Wherein the voice signal conversion step converts the voice signal into a voice signal corresponding to a different language of the same or similar meaning to the identified language.

Images