KR20160107421A - Remote Controller Using Finger Gesture - Google Patents

Abstract

The present invention relates to a remote control apparatus which recognizes a finger in an image and generates a control signal using information respecting the recognized finger. The remote control apparatus according to one embodiment of the present invention includes: a camera for capturing an image; an image input unit for receiving the image captured by the camera; an image analyzing unit for extracting a hand region from the image inputted to the image input unit, and recognizing a finger from the extracted hand region by using an ellipse detection algorithm; and a control signal generating unit for generating a control signal by using information respecting the finger recognized by the image analyzing unit.

Description

[0001] The present invention relates to a remote control using finger gesture,

The present invention relates to a remote control device, and more particularly, to a remote control device that recognizes a finger in an image and generates a control signal using information on the recognized finger.

Recently, motion recognition technology and smart TV have been developed and many devices have been introduced to provide a simple and convenient interface to users.

One of these interfaces is a technique of recognizing a finger or a finger to input a command.

One conventional method of recognizing a finger is to use a circle recognition algorithm as shown in Fig. This circle recognition algorithm is an algorithm for recognizing a finger by recognizing a circle in the hand area after recognizing the hand area. As shown in the figure, there is a problem that the finger segment is mistakenly recognized as a finger in addition to the finger in the hand area.

2 is a flowchart of an image processing method for recognizing a finger according to the prior art. According to this, the image of the hand region is first acquired from the image photographed by the camera. Then, the center of gravity is obtained from the image of the hand region acquired as shown in FIG. 3, and a circle having a certain size is drawn based on the center of gravity, and the number of fingers is recognized by the amount of change in the pixel value of the edge portion of the circle. FIG. 4 is an exemplary result obtained by applying this method. FIG. 4 is a view of a hand recognition image, and FIG. 4 is a finger recognition result image obtained by applying the image processing method of FIG. As shown in FIG. 4, this method has a problem in that it can recognize the number of fingers but can not know the exact position information of the fingertip.

On the other hand, an apparatus for remotely controlling an electronic device by recognizing a finger is a device called a lip motion as shown in Fig. Lip motion is a device that recognizes a user's hand gesture and performs a command corresponding to the gesture. It can recognize the fingertip of a user, but it is difficult to use it in a place such as a place where the distance is far from a TV watching or a moving device , The unit price is high, and there is a problem that the distance can not be overcome due to the usb cable.

SUMMARY OF THE INVENTION In order to solve the above problems, an object of the present invention is to provide a remote control device capable of accurately recognizing information on a user's finger and remotely controlling the electronic device without adding a special device have.

To solve the above object, a remote control apparatus according to the present invention comprises:

A camera for capturing an image; A video input unit receiving an image photographed by the camera; An image analyzer for extracting a hand region from an image input to the image input unit and recognizing a finger using an ellipse detection algorithm in the extracted hand region; And a control signal generator for generating a control signal using information about the finger recognized by the image analysis unit.

According to an aspect of the present invention, the image analyzing unit includes a filtering module for filtering the input image with a predetermined color; A binarization module for binarizing the image filtered by the filtering module; A hand region extraction module for recognizing and extracting a hand region from an image binarized by the binarization module; And a finger recognition module for recognizing information about a finger using an ellipse detection algorithm for detecting an ellipse region in the image extracted by the hand region extraction module.

According to an aspect of the present invention, the filtering module may convert the input image into a YCbCr image, and set a filter value such that the Cb value of the converted YCbCr image is 77 to 127 and the Cr value is 133 to 174 .

According to an aspect of the present invention, the hand region extraction module performs a viola-jones algorithm or a convolution neural network algorithm.

According to one aspect of the present invention, the viola-jones algorithm is a method of generating an integral image composed of sub windows and measuring feature values from the generated integral image to recognize a hand area .

According to an aspect of the present invention, the control unit may further include a communication unit for transmitting the generated control signal to an external electronic device.

According to the remote control apparatus of the present invention, it is possible to remotely control the electronic device by accurately recognizing information on the user's finger. That is, when watching TV or using an air conditioner, it is possible to easily operate the electronic device by using the change of the number of fingers and the gesture of the finger instead of the remote control.

Further, since it can be implemented using a general camera, there is no need to add a special device, which can reduce the cost compared to existing devices (for example, lip motion) It is possible to overcome the problem and improve the user's convenience.

1 is a diagram illustrating a conventional finger recognition method.
2 is a flowchart showing a finger recognition method according to the related art.
Figs. 3 and 4 are diagrams for explaining the finger recognition method of Fig. 2. Fig.
FIG. 5 is a photograph showing the use state of the lip motion, which is one of devices for remotely controlling the electronic device by recognizing the finger according to the related art.
6 is a flowchart illustrating an operation of a remote controller according to an exemplary embodiment of the present invention.
7A is a block diagram illustrating a remote control apparatus according to an embodiment of the present invention.
7B is a block diagram illustrating a remote control apparatus according to another embodiment of the present invention.
8 is a block diagram illustrating an image analysis unit of a remote control apparatus according to an embodiment of the present invention.
FIG. 9 is a flowchart showing a method of detecting an object using a bottom shape feature.
10 is a diagram illustrating a Haar-like feature used in a Viola-Jones algorithm used to recognize a hand region in a remote control apparatus according to an exemplary embodiment of the present invention.
FIG. 11 is a diagram illustrating an experiment result of recognizing a finger using a remote control device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals as used in the appended drawings denote like elements, unless indicated otherwise. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather obvious or understandable to those skilled in the art.

FIG. 6 is a flowchart illustrating an operation procedure of a remote control apparatus according to an embodiment of the present invention. FIG. 7A is a block diagram illustrating a remote control apparatus according to an embodiment of the present invention. Meanwhile, FIG. 7B is a block diagram illustrating a remote control apparatus according to another embodiment of the present invention.

As shown in FIG. 6, the remote control apparatus according to an embodiment of the present invention inputs an image, filters the input image based on the skin color, recognizes the hand in the filtered image, After recognizing the finger with the ellipse, the controller controls the electronic device by generating a control signal corresponding to the number and / or direction of the recognized finger.

The remote control apparatus 100 according to an embodiment of the present invention includes a camera (not shown), an image input unit 110, an image analysis unit 120, a control signal generation unit 130, And may include a communication unit 140 that can perform wired / wireless communication with an external electronic device according to an embodiment.

The remote control apparatus 100 of FIG. 7A is a case where the remote control apparatus is outside the electronic apparatus D, and the remote control apparatus 100 of FIG. 7B is a case where the remote control apparatus is inside the electronic apparatus.

In the case of Fig. 7A, since the remote control device is provided outside the electronic device, the control signal generated by recognizing the finger in the remote control device is transmitted to the electronic device through wired / wireless communication. In the case of Fig. 7B, since the remote control device is mounted inside the electronic device, communication with the electronic device is unnecessary, and a communication section is not required.

7A, a remote control apparatus 100 according to an embodiment of the present invention includes a camera (not shown), an image input unit 110, an image analysis unit 120, (130), and a communication unit (140).

The user is photographed by the camera. That is, if the user exists in the viewing angle of the camera, the user is photographed with the image. At this time, the camera need not be a camera having a special function, and a generally used 2D camera suffices. For example, you can use one 2D camera. On the other hand, if it is difficult to obtain accurate images due to noise and illumination, an infrared camera may be used.

The image input unit 110 receives an image captured by the camera. For example, an RGB image captured by a 2D camera is input. The image input unit 110 receives an operation in which a user sets up a specific number of fingers or moves in a preset direction while a specific number of fingers is raised to remotely control the electronic device. The image input unit 110 receives an image of a user captured by the camera and transmits the image to the image analysis unit 120.

The image analysis unit 120 extracts the hand region from the image transmitted from the image input unit 110 and recognizes the finger using the ellipse detection algorithm in the extracted hand region.

The image analysis unit 120 includes a filtering module 121, a binarization module 122, a hand region extraction module 123, and a finger recognition module 124, as shown in FIG.

The filtering module 121 filters the image input from the image input unit 110 into a predetermined color. An image captured by a camera and input in RGB format is filtered based on a certain color. For example, when filtering is performed based on the skin color, only the skin color is left, and all the remaining images disappear. By removing all but the skin color, the recognition rate for the hand area can be increased. That is, the input image is filtered with leaving only the skin color, and the hand region is recognized based on the filtered skin color image. For example, the skin color filtering converts RGB images into YCbCr images, and then derives pixels whose chrominance components (Cb, Cr) are within a predetermined range. At this time, it is preferable to set the filter value to a Cb value of 77 to 127 and a Cr value of 133 to 174 in the converted YCbCr image. If the range is in the above range, the hand area recognition rate is high, but if it is in the other range, the hand area recognition rate is relatively low, and there is a high possibility that the malfunction occurs in the electronic device control by the finger gesture.

The binarization module 122 binarizes the image filtered by the filtering module 121. That is, the binarization operation is performed by setting the value within the set Cb value and Cr value range to 1 and setting the remaining values to 0. By the binarization, the result image as shown in the left picture of FIG. 4 is extracted. The filtered and binarized image is subjected to an AND operation with the original input image (RGB image) to extract an image remaining on the skin only from the input image.

The hand region extraction module 123 recognizes and extracts the hand region from the binarized image by the binarization module 122. Hand regions can be extracted using similar algorithms such as the viola-jones algorithm or the convolution neural network algorithm. Of course, the present invention is not limited to this, and other algorithms for detecting a specific object in an image or an image may be used.

The Viola-Jones algorithm is an algorithm for detecting a specific object in an image or an image. In the present invention, it is proposed to recognize a hand region using a Haar-like feature of the Viola-Jones algorithm.

FIG. 9 is a flowchart illustrating a method of detecting an object using a bottom shape feature, and FIG. 10 is a diagram illustrating a Haar-like feature used in the Viola-Jones algorithm. Although there are numerous sub-shape features, some of them are illustrated in Fig. The process of recognizing the hand region using the bottom shape feature is as follows.

First, the image of the object and other parts are collected at a constant ratio, and each image is normalized to a certain size (for example, 24x24) to collect and process the learning data.

Next, feature values of the specific shape features are calculated for the generated learning data.

Then, by using Adaboost, a strong classifier that can strongly detect an object among a number of weak classifiers is made, and a plurality of strong classifiers are constituted by a cascade structure.

Then, a new image is input to the cascade structure for classifying the positive image and the negative image, the negative image is discarded, and a desired object is detected by finally selecting both images.

In other words, the filtered and binarized image is scanned with a sub-window as illustrated in FIG. 10, and the sub-window of FIG. 9 (a) to (h) (integer image) generation method, and then inputs it into the cascade structure to measure the feature value to detect the object. As a result, the image of the hand region as shown in the left side of FIG. 4 can be extracted.

The finger recognition module 124 recognizes a finger using an ellipse detection algorithm for detecting an ellipse region in the image extracted by the hand region extraction module 123. [ The ellipse detection algorithm is an algorithm for extracting an ellipse in an image or an image using the center and shape of an ellipse, for example, a fuzzy C-means technique can be used. The fuzzy C-means method is a method for determining the number of candidate initial clusters and the initial cluster center from image segmentation, and performs image clustering to find the minimum enclosed rectangle of the elliptic candidate region required for elliptic region extraction For the minimum adjacent rectangle found in this way, initial ellipses corresponding to the area are set in the region, and the ellipse verification based on the fittness test is performed, and the region having high fitness is extracted as the ellipse region.

FIG. 11 is a diagram illustrating an experiment result of recognizing a finger using a remote control device according to an embodiment of the present invention. In the drawing, the left side is an example of recognizing the hand region in the original image. And the right side is an example of recognizing the finger in the hand area recognized in the left drawing.

The control signal generating unit 130 generates a control signal using the information on the finger recognized by the image analyzing unit 120. Here, the information about the finger is information about the number of fingers set by the user, the number of fingers set by the user, and the moving direction of the fingers.

A control signal corresponding to the number of fingers of the user and / or the direction of movement of the finger is set in the electronic device (D). Therefore, the user can input the control command to the electronic device through the remote control device implemented in the form of FIG. 7A or 7B without the physical remote control device such as the existing remote controller. In addition, since the finger and its moving direction can be recognized through the camera, it can be utilized as an input device such as a mouse.

Here, the electronic device D includes, for example, a TV, a refrigerator, a washing machine, a computer, and a mobile terminal. A remote control device according to the present invention is mounted on the conventional electronic device, So that various operations can be performed without a remote controller.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation in the spirit and scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the appended claims.

100: remote control device
110:
120: Image analysis section
130: control signal generation unit
140:

Claims (6)

A camera for capturing an image;
A video input unit receiving an image photographed by the camera;
An image analyzer for extracting a hand region from an image input to the image input unit and recognizing a finger using an ellipse detection algorithm in the extracted hand region;
And a control signal generation unit for generating a control signal using information about the finger recognized by the image analysis unit.
The image processing apparatus according to claim 1,
A filtering module for filtering the input image with a preset color;
A binarization module for binarizing the image filtered by the filtering module;
A hand region extraction module for recognizing and extracting a hand region from an image binarized by the binarization module;
And a finger recognition module for recognizing information about a finger using an ellipse detection algorithm for detecting an ellipse region in the image extracted by the hand region extraction module.
The filtering module according to claim 2, wherein the filtering module converts the input image into a YCbCr image, and sets the filter value to a Cb value of 77 to 127 and a Cr value of 133 to 174 in the converted YCbCr image Remote control device.
3. The remote control device of claim 2, wherein the hand region extraction module performs a viola-jones algorithm or a convolution neural network algorithm.
5. The viola-jones algorithm according to claim 4, wherein the viola-jones algorithm generates an integral image composed of sub windows and measures feature values from the generated integral image to recognize the hand area Remote control device.
The method according to any one of claims 1 to 5,
And a communication unit for transmitting the generated control signal to an external electronic device.

Images