KR101068281B1 - Portable information terminal and content control method using rear finger movement and gesture recognition - Google Patents

Portable information terminal and content control method using rear finger movement and gesture recognition Download PDF

Info

Publication number
KR101068281B1
KR101068281B1 KR1020100001293A KR20100001293A KR101068281B1 KR 101068281 B1 KR101068281 B1 KR 101068281B1 KR 1020100001293 A KR1020100001293 A KR 1020100001293A KR 20100001293 A KR20100001293 A KR 20100001293A KR 101068281 B1 KR101068281 B1 KR 101068281B1
Authority
KR
South Korea
Prior art keywords
finger
step
information terminal
portable information
movement
Prior art date
Application number
KR1020100001293A
Other languages
Korean (ko)
Other versions
KR20110080871A (en
Inventor
김동주
김정현
홍광석
Original Assignee
성균관대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 성균관대학교산학협력단 filed Critical 성균관대학교산학협력단
Priority to KR1020100001293A priority Critical patent/KR101068281B1/en
Publication of KR20110080871A publication Critical patent/KR20110080871A/en
Application granted granted Critical
Publication of KR101068281B1 publication Critical patent/KR101068281B1/en

Links

Images

Abstract

The present invention recognizes finger movements and finger gestures of a hand held by the portable information terminal by using an image acquisition device provided at the rear of the portable information terminal including an image recognition device and a memory. The present invention relates to a method of controlling available content, the method comprising: a first step of acquiring an image including a finger region of a hand held by the portable information terminal using the image acquisition device; Detecting a finger region from the acquired image and extracting features for finger movement and finger gesture recognition; A third step of recognizing a finger gesture using the feature extracted in the third step to determine whether a finger movement shape of the hand holding the portable information terminal is a finger gesture or a finger movement; A fourth step of detecting a position of a finger from the acquired image when it is determined that the finger movement is performed in the third step; A fifth step of detecting an area change of a finger object region from the acquired image when it is determined that the finger movement is performed in the third step; A sixth step of performing recognition on finger movement by using the feature extracted in the third step, the position of the finger detected in the fourth step, and the area change of the finger object region detected in the fifth step; And a seventh step of controlling the portable information terminal and the content by using the result of the recognition performed in the third step or the result of the recognition performed in the sixth step.

Description

TECHNICAL FOR CONTROLLING A PORTABLE INFORMATION TERMINAL AND CONTENTS THEREOF USING RECOGNIZING FINGER MOTIONS AND GESTURES ON THE BACK OF THE PORTABLE INFORMATION TERMINAL}

The present invention relates to a portable information terminal itself and to a method for controlling content that can be used as a portable information terminal. A method for recognizing and controlling content available to a portable information terminal and a portable information terminal.

In the past, in order to control the contents available by the portable information terminal itself and the portable information terminal, it was common to use a user interface such as a touch screen or a keypad provided in the front portion (front) of the portable information terminal. As portable information terminals are widely used today, the demand for technology development for a method of controlling the portable information terminals under various conditions of use is increasing. However, there is currently no prior art related to a method of controlling a content available to a portable information terminal or a portable information terminal by recognizing finger movements or finger gestures of a hand held by the portable information terminal.

In order to solve this problem, the present invention recognizes finger movements and finger gestures of a hand held by a portable information terminal using an image acquisition device provided at a rear portion of the portable information terminal, thereby enabling content to be used as a portable information terminal and a portable information terminal. It is an object to provide a method of controlling.

The present invention recognizes finger movements and finger gestures of a hand held by the portable information terminal by using an image acquisition device provided at the rear of the portable information terminal including an image recognition device and a memory. A method of controlling available content, the method comprising: a first step of acquiring an image including a finger region of a hand held by the portable information terminal using the image acquisition device; Detecting a finger region from the acquired image and extracting features for finger movement and finger gesture recognition; A third step of recognizing a finger gesture using the feature extracted in the third step to determine whether a finger movement shape of the hand holding the portable information terminal is a finger gesture or a finger movement; A fourth step of detecting a position of a finger from the acquired image when it is determined that the finger movement is performed in the third step; A fifth step of detecting an area change of a finger object region from the acquired image when it is determined that the finger movement is performed in the third step; A sixth step of performing recognition on finger movement by using the feature extracted in the third step, the position of the finger detected in the fourth step, and the area change of the finger object region detected in the fifth step; And a seventh step of controlling the portable information terminal and the content by using the result of the recognition performed in the third step or the result of the recognition performed in the sixth step. Here, the finger gesture refers to a specific movement shape of a finger previously defined among the movement shapes of the finger and stored in the memory of the portable information terminal, and the finger movement refers to the movement shape of the finger instead of the finger gesture among the movement shapes of the finger. Will be pointing.

Further, according to an embodiment of the present invention, the feature extracted in the third step includes the degree of finger bending of the hand holding the portable information terminal, and the position of the finger detected in the fourth step is a two-dimensional plane or a multi-dimensional space region. The change in the area of the finger object region detected in the fifth step may be a change in the area of the finger object region in the acquired image according to the change of the distance between the image capturing apparatus and the finger.

In addition, according to an embodiment of the present invention, the portable information terminal may further include a touch screen or a keypad, and when the portable information terminal and the content are controlled by the control method, the touch screen or the keypad may be a portable information terminal and the content. When the command for controlling the portable information terminal and the content is not controlled by the user, the touch screen or keypad may receive a command for controlling the portable information terminal and the content from the user. You can make it as possible.

According to the present invention, the user's convenience can be increased by controlling the portable information terminal itself and the contents available to the portable information terminal in a non-contact manner with a simple finger operation, and a conventional portable information terminal control method such as a touch screen or a keypad and the like. The function is provided as it is, but in addition, the user can operate and control the portable information terminal with only one hand holding the portable information terminal, thereby providing a more convenient and efficient user environment.

1 is a block diagram of a system for performing a portable information terminal and a content control method using finger movement and gesture recognition according to an embodiment of the present invention.
2 is a flowchart of a method for performing a portable information terminal and a content control method using finger movement and gesture recognition according to an embodiment of the present invention.
3 is a block diagram of a system for performing a portable information terminal and a content control method using finger movement and gesture recognition according to another embodiment of the present invention.
4 is a flowchart of a method for performing a portable information terminal and a content control method using finger movement and gesture recognition according to another embodiment of the present invention.
FIG. 5 is a diagram illustrating an embodiment for acquiring a finger gesture image when a portable information terminal is held by a right hand.
FIG. 6 is a diagram illustrating an embodiment of obtaining a finger gesture image when a portable information terminal is held by a left hand.
7 is a diagram illustrating a method of executing a command according to a movement of a center coordinate of a finger position when a portable information terminal is held by a right hand according to a preferred embodiment of the present invention.
8 is a diagram illustrating a method of executing a command according to a change in the area of a finger object area when a portable information terminal is held by a right hand according to a preferred embodiment of the present invention.
9 is a diagram illustrating a method of executing an instruction according to a movement of a center coordinate of a finger position when a portable information terminal is held by a left hand according to a preferred embodiment of the present invention.
FIG. 10 illustrates a method of executing a command according to a change in the area of a finger object area when a portable information terminal is held by a left hand according to a preferred embodiment of the present invention.

In the present specification, the finger gesture refers to a specific movement shape of a finger previously defined among the movement shapes of the finger and stored in the memory of the portable information terminal, and the finger movement refers to a finger other than the finger gesture among the movement shapes of the finger. Will point to the shape of the movement.

Further, in the present invention, the feature used for the movement or gesture recognition of a two-dimensional plane or a multi-dimensional finger includes a degree of bending of a finger and a shape of a symbolized finger within a range that can be photographed by an image photographing device provided in a portable information terminal. The location of the finger center coordinates in the two-dimensional plane or the multi-dimensional space area, the distance of the continuous finger movement, that is, the area change of the finger object area in the image according to the distance between the image capturing device and the finger. Finger movements or finger gestures recognized using such a feature perform a series of processes for operating and executing a pre-mapped portable information terminal control command or a control command for embedded related content.

A portable information terminal and a content control method using finger movement and gesture recognition will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system for performing a portable information terminal and a content control method using finger movement and gesture recognition according to an embodiment of the present invention, and FIG. 2 is a finger movement and gesture according to an embodiment of the present invention. A flowchart of a portable information terminal and content control method using recognition.

As shown in FIG. 1, the system 100 for performing a portable information terminal and a content control method according to the present invention includes a finger region image acquisition unit 101, a finger region detection and feature extraction unit 103, and a finger. The gesture and finger motion determination unit 105, the finger position detection unit 107, the finger object area area change detection unit 109, the finger motion recognition unit 111, and the portable information terminal and the content control unit 113 are included.

In addition, the portable information terminal and the content control method 200 according to the present invention, as shown in Figure 2, using the image acquisition device to obtain an image including the finger region of the hand holding the portable information terminal In a first step S201, a finger region is detected from the obtained image, a second step S203 for extracting features for finger movement and finger gesture recognition, and a feature extracted for the finger gesture using the features extracted in the third step. In the third step (S205) of determining whether the shape of the finger movement of the hand holding the portable information terminal is a finger gesture or a finger movement by performing the recognition, if it is determined that the finger movement in the third step, In step S207 of detecting a position of a finger from the captured image, when it is determined that the finger movement is performed in the third step, the acquired image Using the fifth step S209 of detecting the area change of the finger object area, the feature extracted in the third step, the position of the finger detected in the fourth step, and the area change of the finger object area detected in the fifth step A sixth step (S211) of performing a recognition on a finger movement, and a seventh step of controlling the portable information terminal and contents using the result of the recognition performed in the third step or the result of the recognition performed in the sixth step. It may include (S213).

The finger region image acquisition unit 101 performs a first step (S201) of acquiring an image including a finger region of a hand held by the portable information terminal using an image acquisition device provided in the portable information terminal. The image acquisition device included in the portable information terminal is generally a built-in type provided in the rear portion (back side), front portion (front side), or side portion (side) of the portable information terminal. The invention can be applied. In addition, the present invention can be applied regardless of whether the image capturing apparatus is a single camera, a multiple camera, or a stereo camera. In the embodiment of the present invention, it is assumed that the image capturing apparatus is provided on the rear portion (back side) of the portable information terminal. In relation to image acquisition, FIG. 5 illustrates an embodiment for acquiring a finger gesture image when a portable information terminal is held by a right hand, and FIG. 6 is a finger gesture image when a portable information terminal is held by a left hand. Figure is a diagram showing an embodiment for obtaining the.

The finger region detection and feature extractor 103 detects a finger region from the image acquired by the finger region image acquirer 101 and performs a second step S203 of extracting a feature for finger movement and finger gesture recognition. do. Here, the image includes not only a two-dimensional plane image, but also a three-dimensional image which can know the depth (distance) information on the two-dimensional plane. In the present invention, in order to recognize the image of the object to reflect the various characteristics of the object, from the acquired object image information using the Hada-like feature-based AdaBoost learning algorithm, Template Matching, etc. Detection and feature detection using a feature extraction algorithm that includes Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA), and 2-D Discrete Cosine Transform (2D-DCT). Extract In particular, in detecting the finger region, color information, contour features, and the like may be utilized, and as a feature related to the shape of a finger, a finger bending degree of a hand holding a portable information terminal may be used.

The finger gesture and the finger motion determination unit 105 recognize the finger gesture by using the feature extracted from the finger region detection and feature extraction unit 103 so that the shape of the finger movement of the hand holding the portable information terminal is changed. A third step (S205) is performed to determine whether it is a finger gesture or a finger movement. As described above, the finger gesture in the present specification refers to a specific movement shape of a finger which is previously defined among the movement shapes of the finger and stored in the memory of the portable information terminal. It may be mapped to a specific control command and stored in the memory of the portable terminal in advance. The purpose of symbolizing the finger gesture is to allow a user to directly execute a specific control command by taking a variety of finger gestures by using the index finger having a relatively high degree of freedom of movement when the user is simply holding the portable information terminal. In the third step, a pattern recognition process is performed on which finger gesture corresponds to the shape of the finger motion in the finger region image, and when a corresponding finger gesture exists, a specific control command is mapped to the shape of the user's finger motion. The method determines the symbolized finger gesture to control the portable information terminal and the content (S213). If the corresponding finger gesture does not exist, the user's finger movement shape is determined to be a continuous finger movement and additional features are determined. The process proceeds to the extraction step (S207, S209).

When it is determined that the finger movement is continuous in the third step S205, the finger position detector 107 performs a fourth step S207 of detecting the position of the finger from the acquired image. In the fourth step S207, as a feature for recognizing the movement of the finger in the finger movement recognition step S211, the position of the finger center coordinates in the two-dimensional plane or the multi-dimensional space area is provided.

When it is determined that the finger movement is continuous in the third step S205, the finger object region area change detection unit 109 performs a fifth step S209 of detecting an area change of the finger object area from the obtained image. . In the fifth step S209, as a feature for recognizing the movement of a finger in the finger movement recognizing step S211, the area change of the finger object region in the acquired image according to the distance between the image acquisition device and the finger is changed. to provide.

In the order in which the fourth step S207 and the fifth step S209 are performed, the fifth step S209 may be performed after the fourth step S207 is performed, and the fifth step S209 may be performed. Thereafter, the fourth step S207 may be performed.

The finger motion recognition unit 111 detects the feature extracted by the finger region detection and feature extraction unit 103, the position of the finger detected by the finger position detection unit 107, and the finger object region area change detection unit 109. A sixth step S211 of performing finger recognition using a change in the area of the finger object region is performed. This is a step of recognizing the degree of finger movement in the continuous movement of the finger. After detecting the corresponding finger region by using the finger movement image captured by the image capturing apparatus, the area of the detected finger object region is the maximum- The image is normalized to a finger object image having a specific size by using minimum information and a rotation angle of the image, and the center point coordinates of the corresponding object image are obtained using the normalized object image. Thereafter, the corresponding center point is obtained based on the detected finger region to track the degree of change of the position coordinate according to the change of the center point. Here, the change of the center point means the change of the position of the finger object, and the recognition result of the change of the position of the finger object is mapped with a control command such as up / down / left / right / selection (click), and the touch is performed. It can be presented to the user with visual effects such as cursor movement or mouse pointing in the screen panel.

In the finger gesture recognition step S205 or the finger motion recognition step S211, various objects are applied by applying pattern recognition algorithms such as K-NN, support vector machine (SVM), and embedded hidden markov model (EHMM) based on the extracted feature. An image recognition result is derived, and the present invention includes an overall technique of detecting a corresponding region using a variety of pattern recognition algorithms, and interpreting and recognizing the region in multiple dimensions. In addition, the present invention recognizes a hand shape that is robust to angle and self occlusion using multi-dimensional finger movement or gesture estimation, and recognizes finger gesture and finger movement by analyzing the motion shape of the finger. In general, in order to stabilize the recognition result, temporal continuity must be taken into consideration.Motion Energy Image (MEI), which is a binary image representing temporal position change of finger movement, and a MotionHistory Image (MHI) pattern representing temporal lapse information of the motion HMM recognition method that expresses continuous posture as a state change by using is used in the final process of recognition. In order to remove the noise of the input image, one erosion and morphology operations are performed, and a method of using a dilation morphology operation to prevent the finger region from shrinking is also used. Can be. In addition, in order to effectively express and recognize body poses and motions by extracting feature values from a visual input image, finger motion shape feature extraction methods include the Motion Energy Image (MEI) and Motion History Image (MHI) patterns, and Ross Cutler. It may include a method of (Ross Cutler), a cardboard model (cardboard model). In addition, according to the present invention, the features used for finger movement and finger gesture recognition include, as described above, the degree of bending of the finger of the hand holding the portable information terminal, the shape of a predefined finger, two-dimensional plane, or multi-dimensional. The location of the finger center coordinates in the space area or the area change of the finger object area in the acquired image according to the distance between the finger and the image capturing device may be included.

3 is a block diagram 300 of a system for performing a portable information terminal and a content control method using finger movement and gesture recognition according to another embodiment of the present invention, and FIG. 4 is a finger movement according to another embodiment of the present invention. And a method 400 for performing a portable information terminal and a content control method using gesture recognition.

3 and 4, according to an exemplary embodiment of the present invention, the finger gesture recognition may be integrated into the finger gesture and the finger motion recognition step S411 and performed by the finger gesture and the finger motion recognition unit 311. In this case, the final recognition of the finger gesture may not be performed by the finger gesture and the finger movement determining unit 305 for determining whether the motion shape of the finger is the finger gesture or the finger movement (S405).

The portable information terminal and the content control unit 113 control the portable information terminal and the content by using the result of the recognition performed in the finger gesture recognition step S205 or the result of the recognition performed in the finger motion recognition step S211. Step S213 is performed.

According to the present invention, when a finger movement shape is recognized as a finger gesture, a specific control command mapped to the recognized finger gesture is directly executed. On the other hand, when it is recognized as a finger movement, five categories of control commands, for example, up / down / left / right / selection (click), may be executed for controlling the portable information terminal and embedded related contents. In this case, the characteristics of the image capturing apparatus provided in the portable information terminal and the range of finger movement or finger gesture expression are considered.

7 is a diagram illustrating a method of executing a command according to a movement of a center coordinate of a finger position when a portable information terminal is held by a right hand according to a preferred embodiment of the present invention. A diagram illustrating a method of executing an instruction. FIG. 9 is a diagram illustrating a method of executing a command according to a movement of a center coordinate of a finger position when a portable information terminal is held by a left hand according to a preferred embodiment of the present invention, and FIG. A diagram illustrating a method of executing an instruction.

Hereinafter, the configuration and operation of the present invention shown in and described with reference to FIGS. 7 to 10 will be described in at least one embodiment. Control commands by finger movements and gestures can be further expanded and changed in order of command mapping. However, the technical spirit of the present invention and its core configuration and operation are not limited thereto.

7 to 10 illustrate preferred embodiments of keypad operation of a portable information terminal, and control commands such as up / down / left / right / selection (click) in operating a general dial keypad composed of 3 × 4 keys. The configuration method of the is disclosed.

As shown in FIG. 7 or 9, in generating a left / right movement command, a user photographs a finger region moved in a left / right direction by using an image acquisition device provided in a portable information terminal and captures a finger region from the captured image. After the feature region is detected, the coordinates of the center point of the object image are obtained using the normalized object image. Thereafter, the amount of change of the center coordinate is observed and recognized, and the movement of the corresponding finger is performed as a left / right movement command, thereby controlling or executing the portable information terminal and embedded related contents. In addition, after observing the change in the coordinates of the center point, the step of recognizing the movement or gesture of the finger as a left / right command when the duration of a certain time or a predetermined frame is shown using a time stamp or gesture spotting, etc. Include.

Reference is made to FIG. 8 or 10 with respect to generating an up / down movement command using the finger's movement feature in a similar manner. When the distance between the image capturing device and the corresponding finger is close, the area of the finger object area is very large, and when the distance is far, the area of the finger area is detected to be narrower, and thus the change in the area of the finger object area is detected. A degree can be classified into a new feature in addition to the center point coordinate feature. Therefore, in the step of generating the up / down control command, if the area is wide according to the area of the corresponding finger object image, that is, if the distance between the image capturing device and the corresponding finger is close, the motion of the finger is reduced. The "up" movement is defined as "up" movement, but the area of the object image corresponding to the intermediate stage between "up" and "down" in modeling the object feature using the area of the object image is defined as "up". Control commands corresponding to, "middle" and "bottom" can be generated independently. In addition, after observing the change in the area of the finger object image by using a time stamp or gesture spotting after observation, the movement or gesture of the corresponding finger may be recognized as a left / right command. Can be.

In addition, the control point for selecting or executing a portable information terminal and embedded related content, ie, using a center-based coordinate feature and an area of an object image in controlling a selection function similar to a window-based click or double-click function, may be used. The control command may be generated by using a sudden movement of a finger or a gesture. As an example, as shown in FIG. 7 or 9, the finger of the corresponding hand held by the portable information terminal is moved by moving the center coordinates of the finger in a state where the distance between the image acquisition device and the finger is far from the corresponding finger. The extent of the change in the area of the object image can be confirmed. In addition, since the change of the center coordinates can be observed by moving the finger to the left / right, the system recognizes the change in the area of the center coordinates or the object area and recognizes the portable information terminal and the related contents. It is used as a control command for selection or execution.

Image recognition technology for finger gestures or finger movements can be applied to an image-based object recognition technology including a two-dimensional or three-dimensional gesture image as described above, and according to the present invention, The image recognition technology may utilize all conventional image recognition technologies using a single / multi camera or a stereo camera provided in the portable information terminal. In addition, the present invention may include reconstructing the two-dimensional plane and the multi-dimensional finger model in order to track and analyze the multi-dimensional phase movement distance of the continuous finger by extracting the finger region from the entire image.

Furthermore, the system according to the present invention can acquire an image according to the present invention so that a user can execute a control command based on an image acquisition device, combined with a user interface represented by a touch screen and a keypad for inputting system control and operation commands. The apparatus may further include an input mode determiner configured to determine whether a control command is input through the device. In the input mode determination unit, when a control command using finger gesture recognition is input, a user cannot input a control command through a touch screen or a keypad, and a visual expression (mouse) on a touch screen panel through a control command according to the present invention. Pointing, etc.). In other words, the portable information terminal and the content control method using the finger movement and gesture recognition according to the present invention, when the portable information terminal and the content is controlled by the control method according to the present invention, the touch screen or keypad controls the portable information terminal and the content. In order to be able to receive a command for the user from the user, and when the portable information terminal and the content is not controlled using the control method according to the present invention, the touch screen or keypad is a user command for controlling the portable information terminal and content It may further include a function to be in a state that can be input from.

With respect to the present invention, it will be understood from the foregoing description that the foregoing and equivalents thereof may be embodied in various forms. Thus, the true scope of the present invention includes the following claims and any equivalents that may be associated with such person skilled in the art, and are not limited by the specific embodiments described herein.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Also, in this specification, the singular forms are intended to include the plural forms as well, unless the context clearly indicates that the plural is not.

101: finger region image acquisition unit
103: finger region detection and feature extraction unit
105: finger gesture and finger movement determination unit
107: finger position detection unit
109: finger object region area change detection unit
111: finger motion recognition unit
113: portable information terminal and content control unit
S201: finger area image acquisition step
S203: finger region detection and feature extraction step
S205: finger gesture and finger movement judgment step
S207: Finger Position Detection Step
S209: Finger object region area change detection step
S211: Finger Movement Recognition Step
S213: portable information terminal and content control step

Claims (6)

  1. Content that can be used as a portable information terminal and a portable information terminal by recognizing finger movements and finger gestures of a hand held by the portable information terminal by using an image acquisition device provided at a rear portion of the portable information terminal including an image recognition device and a memory. As a method of controlling
    A first step of acquiring an image including a finger region of a hand holding the portable information terminal using the image acquisition device;
    Detecting a finger region from the acquired image and extracting features for finger movement and finger gesture recognition;
    A third step of recognizing a finger gesture by using the feature extracted in the second step to determine whether a finger movement shape of the hand holding the portable information terminal is a finger gesture or a finger movement;
    A fourth step of detecting a position of a finger from the acquired image when it is determined that the finger movement is performed in the third step;
    A fifth step of detecting an area change of a finger object region from the acquired image when it is determined that the finger movement is performed in the third step;
    A sixth step of performing recognition on finger movement by using the feature extracted in the third step, the position of the finger detected in the fourth step, and the area change of the finger object region detected in the fifth step; And
    A seventh step of controlling the portable information terminal and the content by using the result of the recognition performed in the third step or the result of the recognition performed in the sixth step,
    The finger gesture refers to a specific movement shape of a finger previously defined among the movement shapes of the finger and stored in the memory of the portable information terminal, and the finger movement refers to the movement shape of the finger instead of the finger gesture among the movement shapes of the finger. A portable information terminal and content control method using finger movement and gesture recognition, characterized in that.
  2. The method of claim 1,
    The feature extracted in the third step includes a finger bending degree of the hand holding the portable information terminal, wherein the portable information terminal and the content control method using finger movement and gesture recognition.
  3. The method of claim 1,
    The position of the finger detected in the fourth step is the position of the finger center coordinates in the two-dimensional plane or the multi-dimensional space region.
  4. The method of claim 1,
    The change in the area of the finger object area detected in the fifth step is a change in the area of the finger object area in the acquired image according to the change of the distance between the image capturing device and the finger. And content control method.
  5. The method of claim 1,
    The feature extracted in the third step includes the degree of bending of the finger of the hand holding the portable information terminal,
    The position of the finger detected in the fourth step is the position of the finger center coordinates in the two-dimensional plane or the multi-dimensional space region,
    The change in the area of the finger object area detected in the fifth step is a change in the area of the finger object area in the acquired image according to the change of the distance between the image capturing device and the finger. And content control method.
  6. The method according to any one of claims 1 to 5,
    The portable information terminal further includes a touch screen or a keypad,
    A state in which the touch screen or keypad cannot receive a command for controlling the portable information terminal and contents from a user when controlling the portable information terminal and contents by the control method according to any one of claims 1 to 5. When the portable information terminal and the content are not controlled by the control method according to any one of claims 1 to 5, the touch screen or the keypad inputs commands for controlling the portable information terminal and the content from the user. A portable information terminal and content control method using finger movement and gesture recognition, characterized in that the state can be received.
KR1020100001293A 2010-01-07 2010-01-07 Portable information terminal and content control method using rear finger movement and gesture recognition KR101068281B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020100001293A KR101068281B1 (en) 2010-01-07 2010-01-07 Portable information terminal and content control method using rear finger movement and gesture recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020100001293A KR101068281B1 (en) 2010-01-07 2010-01-07 Portable information terminal and content control method using rear finger movement and gesture recognition

Publications (2)

Publication Number Publication Date
KR20110080871A KR20110080871A (en) 2011-07-13
KR101068281B1 true KR101068281B1 (en) 2011-09-28

Family

ID=44919708

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020100001293A KR101068281B1 (en) 2010-01-07 2010-01-07 Portable information terminal and content control method using rear finger movement and gesture recognition

Country Status (1)

Country Link
KR (1) KR101068281B1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101281461B1 (en) * 2012-04-24 2013-07-03 군산대학교산학협력단 Multi-touch input method and system using image analysis
WO2014200485A1 (en) * 2013-06-13 2014-12-18 Intel Corporation Techniques for user authentication on a computing device via pattern recognition

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040196400A1 (en) 2003-04-07 2004-10-07 Stavely Donald J. Digital camera user interface using hand gestures
KR100563737B1 (en) 2003-12-30 2006-03-28 주식회사 팬택앤큐리텔 Method and appratus for user operating interface using camera function of mobile communication unit
KR100783552B1 (en) 2006-10-11 2007-12-07 삼성전자주식회사 Input control method and device for mobile phone
KR100952603B1 (en) 2009-11-05 2010-04-15 팬터로그인터액티브 주식회사 Screen scroll method and device using camera

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040196400A1 (en) 2003-04-07 2004-10-07 Stavely Donald J. Digital camera user interface using hand gestures
KR100563737B1 (en) 2003-12-30 2006-03-28 주식회사 팬택앤큐리텔 Method and appratus for user operating interface using camera function of mobile communication unit
KR100783552B1 (en) 2006-10-11 2007-12-07 삼성전자주식회사 Input control method and device for mobile phone
KR100952603B1 (en) 2009-11-05 2010-04-15 팬터로그인터액티브 주식회사 Screen scroll method and device using camera

Also Published As

Publication number Publication date
KR20110080871A (en) 2011-07-13

Similar Documents

Publication Publication Date Title
Murthy et al. A review of vision based hand gestures recognition
JP4972201B2 (en) Man-machine interaction based on sign
TWI617953B (en) Multi-task switching method, system and electronic device for touching interface
CN101853071B (en) Gesture identification method and system based on visual sense
Argyros et al. Vision-based interpretation of hand gestures for remote control of a computer mouse
US20060209021A1 (en) Virtual mouse driving apparatus and method using two-handed gestures
CN102272773B (en) Method, apparatus and computer program product for providing hand segmentation for gesture analysis
TWI489317B (en) Method and system for operating electric apparatus
JPWO2011142317A1 (en) Gesture recognition apparatus, method, program, and computer-readable medium storing the program
EP2538305A2 (en) System and method for close-range movement tracking
US20110158476A1 (en) Robot and method for recognizing human faces and gestures thereof
US7340077B2 (en) Gesture recognition system using depth perceptive sensors
US10242255B2 (en) Gesture recognition system using depth perceptive sensors
US8837780B2 (en) Gesture based human interfaces
JP4323180B2 (en) Interface method, apparatus, and program using self-image display
EP2605844A2 (en) Method circuit and system for human to machine interfacing by hand gestures
US9916012B2 (en) Image processing apparatus, image processing method, and program
JP6360050B2 (en) Method and system for simultaneous human-computer gesture-based interaction using unique noteworthy points on the hand
KR20100138602A (en) Apparatus and method for a real-time extraction of target's multiple hands information
DE202008000261U1 (en) Multiple Berührungsdatenfussionssystem
JP2004280834A (en) Motion recognition system using virtual writing plane, and recognition method thereof
CN102591459A (en) Classification of posture states
CN104898879A (en) Method and apparatus for data input
US8994652B2 (en) Model-based multi-hypothesis target tracker
CN103294996B (en) A kind of 3D gesture identification method

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20140617

Year of fee payment: 4

FPAY Annual fee payment

Payment date: 20150703

Year of fee payment: 5

FPAY Annual fee payment

Payment date: 20170303

Year of fee payment: 6

FPAY Annual fee payment

Payment date: 20170703

Year of fee payment: 7

LAPS Lapse due to unpaid annual fee