KR20140140482A - Apparatus and method for processing an user input using movement of an object - Google Patents

Abstract

The present invention relates to an apparatus and a method for processing user input using movement of an object. According to an embodiment of the present invention, the user input processing apparatus determines whether to track the movement of the object by using an input image including the movement information of the object. The user input processing apparatus tracks the movement of the object.

Description

[0001] APPARATUS AND METHOD FOR PROCESSING AN USER INPUT USING MOVEMENT OF AN OBJECT [0002]

The following embodiments relate to a user input processing apparatus and method, and a technique for processing user input using an input image.

Traditional mouse input interfaces move the cursor position by tracking the movement of the mouse with ball rolling or laser scanning. In addition, the mouse input interface can perform a specific function corresponding to the cursor position in response to a click of the mouse button.

With recent advances in image processing techniques, these traditional mouse input interfaces can be replaced by user interface technology that recognizes objects in three-dimensional space.

A user input processing apparatus according to one side judges whether or not the input image corresponds to a first mode input requesting tracking of the motion using an input image providing at least one point at which motion of the object is sensed ; And a processor for tracking the motion when the input image corresponds to the first mode input.

In this case, the determination unit may compare the scattering of the at least one point with a threshold value, and may determine that the input image corresponds to the first mode input when the scattering is equal to or greater than the threshold value.

The processing unit may further include: a center of gravity calculation unit for calculating a center of gravity of the at least one point; A line detector for detecting a dominant line associated with the at least one point; And a tip detecting unit for detecting a tip of the object based on the center of gravity and the dominant line.

The determination unit may compare the scatter of the at least one point with a threshold value and determine that the input image corresponds to a second mode input indicating a selection input when the scattering is smaller than the threshold value.

Also, the determination unit may check whether the pattern of the at least one point corresponds to a predetermined pose, and if the pattern corresponds to the predetermined pose, the input image may be input to a third mode input It can be judged as corresponding.

A method of processing a user input according to another aspect includes: acquiring an event signal in which motion of an object is detected; Calculating a center of gravity of a plurality of pixels included in the event signal; Extracting a dominant line included in the plurality of pixels; And recognizing a tip of the object based on the center of gravity and the dominant line.

According to another aspect of the present invention, there is provided a method of processing a user input, comprising: obtaining an event signal in which motion of an object is detected; Dividing a plurality of pixels included in the event signal into a predetermined number of sections; Calculating orientation vectors of the plurality of pixels; Calculating feature vectors of the sections based on the orientation vectors; And recognizing a tip of the object based on the feature vectors.

1 is a block diagram illustrating a user input processing apparatus according to an embodiment;
FIGS. 2A and 2B are diagrams for explaining operation modes determined according to the scattering of a plurality of pixels included in an input image according to an exemplary embodiment; FIG.
3A to 3D are diagrams illustrating a technique of recognizing a tip of an object according to an exemplary embodiment.
4A to 4D illustrate a technique for recognizing a predetermined pause corresponding to the pause mode according to an embodiment.
5 is a flow chart illustrating a method of processing a user input for tracking movement of an object according to an embodiment;
6 is an operational flow diagram illustrating a user input processing method for recognizing a selection input according to an embodiment.
7A to 7D are diagrams for explaining a technique of recognizing a tip of an object according to another embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a user input processing apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a user input processing apparatus 100 according to an embodiment recognizes an input using an object in a predetermined space. An object is an object for performing an input to a user input processing device, and may include, for example, a user's hand or the like.

The user input processing apparatus 100 includes a determination unit 110 and a processing unit 120.

The determination unit 110 may determine whether the input image corresponds to a first mode input requesting tracking for motion using an input image providing at least one point at which motion of the object is sensed.

The input image may be an output image of a dynamic vision sensor that photographs the object. The dynamic vision sensor may include a passive image sensor using the principle of operation of a human retinal optic nerve. The dynamic vision sensor is an event-based image sensor, and can output the event signal in response to the movement of the object. Hereinafter, an input image is referred to as an event signal for convenience of explanation, and a point is referred to as a pixel.

The event signal may include a signal generated asynchronously in response to motion of the object. The event signal may include information such as the optic nerve signal transmitted from the human retina to the brain. For example, an event signal may not be generated for a stationary object, but only when a moving object is sensed. Of course, the event signal may include information related to moving objects.

The plurality of pixels included in the event signal may correspond to the parts of the object in which the motion is detected. For example, suppose a user moves the entire hand. Referring to FIG. 2A, the event signal may include a plurality of pixels corresponding to the entire hand.

Alternatively, it is assumed that the user moves only the index finger. Referring to FIG. 2B, the event signal may include only pixels corresponding to the index finger portion.

The determination unit 110 can determine the operation mode based on the scattering of the plurality of pixels. The determination unit 110 may calculate the scatter of the plurality of pixels included in the event signal. The determination unit 110 may calculate a scattered distribution of a plurality of pixels in a predetermined direction. For example, the determination unit 110 may calculate a scatter where a plurality of pixels are dispersed in the y-axis direction with respect to the center of gravity of a plurality of pixels.

The determination unit 110 may determine an operation mode corresponding to the scattering of the plurality of pixels.

The determination unit 110 may compare the scattering of a plurality of pixels included in the event signal with a threshold value and determine that the event signal corresponds to the first mode input when the scattering is equal to or greater than the threshold value.

Also, the determination unit 110 may determine that the event signal corresponds to the second mode input indicating the selection input when the scattering is smaller than the threshold value. Hereinafter, for convenience of explanation, the operation in the first mode is referred to as a pointing operation, and the operation in the second mode is referred to as a click operation.

Here, the threshold value can be predetermined as a value between a scatter value of a portion to be moved for a pointing operation and a scatter value of a portion to be moved for a click operation.

Referring to FIG. 2A, when an event signal is generated by the user moving the entire hand while holding the index finger, the determination unit 110 can determine that the user has input the pointing operation.

Referring to FIG. 2B, when an event signal is generated by bending the index finger while the user does not move the hand, the determination unit 110 may determine that the user has input the click operation.

Accordingly, the user input processing apparatus 100 according to one embodiment can provide a spatial mouse function using a single image sensor. The spatial mouse function is an interface provided by sensing input in a three-dimensional space.

Generally, in order to utilize the space mouse function, it is necessary to generate a three-dimensional image of an object and analyze the generated three-dimensional image. This is because, in order to determine whether or not the click operation is inputted, information about the depth change of the object part included in the image (for example, the index finger part performing the click operation) is required. In this case, at least two pieces of image data are required to generate a three-dimensional image.

On the other hand, the user input processing apparatus 100 according to one embodiment can provide a spatial mouse function using image data output from a single image sensor by using an event signal according to the movement of the object.

When the input image corresponds to the first mode input (for example, a pointing input), the processing unit 120 may track the motion of the object and output a signal for the pointing operation. Alternatively, the processing unit 120 may output a signal for a click operation when the input image corresponds to a second mode input (e.g., a click input).

The pointing operation is a concept inclusive of an operation for indicating a specific position on the display, for example, may include an operation of determining the position of the mouse cursor in the display, an operation of moving the position of the mouse cursor in the display, .

The click operation is a concept including, for example, a left-click operation, a double-click operation, and the like, comprehensively including an operation of activating at least one function related to a point being pointed.

The processing unit 120 is a signal for a pointing operation, and can output information about a position indicated by a tip of the object. Also, the processing unit 120 can output information indicating that the click operation is recognized as a signal for a click operation.

Of course, it is to be understood that a general mouse interface such as a drag operation, a drag-and-drop operation, a right-click operation, and a scroll operation can be implemented using various combinations and combinations of pointing operations and click operations, .

FIGS. 2A and 2B are views for explaining operation modes determined according to the scattering of a plurality of pixels included in an input image according to an exemplary embodiment.

Referring to FIG. 2A, an event signal according to an exemplary embodiment may include a plurality of pixels corresponding to movement of the entire hand.

For example, if the user moves his entire hand in a specific direction with his index finger held, the event signal may include a plurality of pixels corresponding to the entire hand in which the motion occurred.

The event signal may include an ON event or an OFF event depending on the type of the detected event. For example, black pixels include pixels in which an OFF event is detected in which the input light is darker than a threshold corresponding to the OFF event, white pixels include an ON event in which the input light is brighter than a threshold corresponding to the ON event May include a detected pixel.

The user input processing device can detect a tip of an object for a pointing operation based on a plurality of pixels.

In some cases, the user input processing device can detect the tip of the object in consideration of the type of the detected event. For example, the user input processing apparatus can detect the tip of the object by using both the ON event signal and the OFF event signal. Alternatively, the user input processing device can detect a tip of an object using an event signal of any one of an ON event signal and an OFF event signal.

More details related to the operation of detecting the tip of the object for the pointing operation will be described later with reference to FIGS. 3A to 3D and FIGS. 7A to 7D.

Referring to FIG. 2B, an event signal according to an exemplary embodiment may include a plurality of pixels corresponding to a movement of a finger.

For example, when the user bends the first node of the index finger as if the user actually clicked the mouse, the event signal may include pixels corresponding to the finger of the segment where the motion occurred.

The user input processing device can determine that the operation input by the user corresponds to the click operation based on the distribution of the pixels included in the event signal.

In some cases, the user input processing device may calculate the scatter, taking into account the type of event detected. For example, the user input processing apparatus can calculate the scatter using both the ON event signal and the OFF event signal. Alternatively, the user input processing apparatus can calculate the scatter using the event signal of any one of the ON event signal and the OFF event signal.

3A to 3D are diagrams for explaining a technique of recognizing a tip of an object according to an embodiment.

Referring to FIG. 3A, a user input processing apparatus according to an exemplary embodiment may calculate a center of gravity 310 of a plurality of pixels included in an event signal.

For example, although not shown in the drawing, the processing unit 120 of FIG. 1 may include a center of gravity calculation unit. The center-of-gravity calculator may calculate the x-coordinate of the center of gravity 310 by summing all the x-coordinate values of the plurality of pixels included in the event signal and dividing the summed result by the number of pixels. In addition, the center-of-gravity calculator may calculate the y-coordinate of the center of gravity 310 by summing all the y-coordinate values of the plurality of pixels included in the event signal and dividing the summed result by the number of pixels.

Referring to FIG. 3B, the user input processing apparatus according to an exemplary embodiment may extract a dominant line 320 from a plurality of pixels included in an event signal.

For example, although not shown in the drawing, the processing unit 120 of FIG. 1 may include a line detection unit. The dominant line 320 is the dominant line segment among a plurality of line segments that can be configured using a plurality of pixels included in the event signal. For example, when the object is a hand with only the index finger, The finger 320 may be a line in the direction indicated by the index finger.

In order to extract the dominant line, the line detecting unit may display a plurality of lines that can be formed using the plurality of pixels included in the event signal, in angle coordinates. The line detecting unit may obtain a plurality of lines by performing a hough transform on a plurality of pixels included in the event signal, and may obtain a dominant line from the obtained plurality of lines.

Referring to FIG. 3C, in order to recognize a tip of an object, a user input processing apparatus according to an exemplary embodiment may project a point corresponding to the center of gravity to a dominant line.

For example, the user input processing device can project the center of gravity 310 of FIG. 3A to the dominant line 320 of FIG. 3B, and thereby obtain the projection point 330.

Referring to FIG. 3D, a user input processing apparatus according to an exemplary embodiment may search a tip along a dominant line in a predetermined direction from a projection point, in order to recognize a tip of an object.

For example, the user input processing apparatus can search the tip from the projection point 330 of FIG. 3C in the direction in which the value of the y-axis coordinate increases along the dominant line of FIG. 3B. The user input processing apparatus can recognize the last region 340 in which a plurality of pixels included in the event signal among the plurality of regions existing on the dominant line exist as tips of the object.

Although not shown in the figure, the user input processing apparatus according to another embodiment may determine whether or not the click operation is performed based on the number of regions searched until the tip of the object is recognized.

For example, the user input processing apparatus can estimate the scattering of a plurality of pixels included in the event signal using the number of regions searched until the tip of the object is recognized. The user input processing apparatus can determine that the click operation has been input when the number of the searched regions is smaller than a predetermined threshold until the tip of the object is recognized.

4A to 4D are diagrams for explaining a technique of recognizing a predetermined pause corresponding to the pause mode according to an embodiment.

Although not shown in the drawing, the determination unit 110 in FIG. 1 can check whether a pattern composed of at least one pixel corresponds to a predetermined pose in the event signal.

The determination unit 110 may determine that the event signal corresponds to the third mode input indicating the stop of the tracking when the pattern corresponds to the predetermined pose.

The pause corresponding to the interruption of the tracking can be predetermined, and the determination unit 110 can determine whether or not the predetermined pose has been taken. Hereinafter, for convenience of explanation, it is assumed that a predetermined pose is a pose that stretches five fingers.

Referring to FIG. 4A, the determination unit 110 may calculate a center of gravity 410 of a plurality of pixels included in an event signal. For the technique of calculating the center of gravity, the matters described with reference to FIG. 3A can be applied as they are, so that a detailed description will be omitted.

Referring to FIG. 4B, the determination unit 110 may identify regions in a predetermined direction from the center of gravity.

For example, the determination unit 110 may include a first region 421 that is closest to the center of gravity 410 of FIG. 4A in the direction of increasing y-axis coordinate value, a second region 422 ), And the third region 423 at the farthest distance.

Referring to FIG. 4C, the determination unit 110 may measure a pattern of pixels included in a region distant from the center of gravity by a predetermined distance in a predetermined direction. The pattern of pixels may include an aspect in which a plurality of pixels are scattered.

For example, the determination unit 110 may acquire a third region 430 that is separated from the center of gravity by a predetermined distance or more in the direction of increasing the y-axis coordinate value. The determination unit 110 may select the region 423 farthest from the center of gravity among the plurality of regions 421, 422, and 423 classified in FIG. 4B.

The determination unit 110 may measure the pattern of the pixels included in the third area 430 to determine whether the object has taken a pause corresponding to the interruption of the space input recognition operation.

For example, the determination unit 110 may obtain the graph 442 of FIG. 4D by summing the number of pixels having the same x-coordinate value among a plurality of pixels along the x-axis of the third region 430.

Referring to FIG. 4D, the x-axis of the graph 442 of FIG. 4D may correspond to the x-axis of the third region 430 of FIG. 4C. The y-axis of the graph 442 in FIG. 4D may represent the number of pixels having the x-coordinate corresponding to the value of each x-axis among the plurality of pixels included in the third area 430.

The determination unit 110 can determine whether or not the object has taken a pause corresponding to the interruption of the space input recognition operation by comparing the predetermined stoppage threshold value 441 with the graph 442. [

For example, the graph 442 repeatedly has an aspect in which the value of the y coordinate becomes larger and smaller than the stop pause threshold value 441 as the value of the x coordinate increases. The determination unit 110 may count the number of such aspects. The determination unit 110 may determine that the object has taken a pose that stretches five fingers when the sum is equal to or greater than a predetermined number (e.g., 3).

Alternatively, the determination unit 110 may count the number of peaks larger than the stop pause threshold value 441 among a plurality of peaks included in the graph 442. The determination unit 110 may determine that the object has taken a pose that stretches five fingers when the sum is equal to or greater than a predetermined number (e.g., 3).

Although not shown in the drawing, the determination unit 110 according to another embodiment applies a line filter technique to a plurality of pixels included in an event signal, It is possible to judge whether or not the user has taken it.

The determination unit 110 may apply various line filter techniques. For example, the determination unit 110 may use an orientation filter, a Hough filter, a Gabor filter, or a template matching filter.

More specifically, the determination unit 110 applies a line filter technique to a plurality of pixels included in an event signal to exclude a horizontal component included in a plurality of pixels and to extract vertical components Can be extracted.

The determination unit 110 may measure a pattern of pixels using the extracted vertical component. For example, the determination unit 110 may calculate the vertical component of the extracted vertical component by summing the number of pixels having the same x-coordinate value among the pixels included in the extracted vertical component along the x- Lt; RTI ID = 0.0 > a < / RTI >

In this case, the determination unit 110 can obtain a more sharp graph compared to the graph 442 in FIG. 4D, whereby the determination unit 110 can more accurately recognize the pose input by the object have.

5 is a flowchart illustrating a method of processing a user input for tracking movement of an object according to an exemplary embodiment of the present invention.

Referring to FIG. 5, a user input processing method according to an embodiment can recognize an input using an object in a predetermined space.

The user input processing method may acquire an event signal in which the movement of the object is sensed at step 510. The user input processing method may calculate the center of gravity of the plurality of pixels included in the event signal in operation 530 and extract a dominant line included in the plurality of pixels in operation 540.

The user input processing method can recognize the tip of the object based on the center of gravity and the dominant line in step 550. [

Steps 510, 530, 540, and 550 may be applied to each of the items described with reference to FIGS. 1 through 4D, so that a detailed description thereof will be omitted.

The user input processing method according to another embodiment may apply the quantization technique in order to reduce errors due to the noise included in the event signal in step 520. [

For example, the user input processing method may classify a plurality of pixels included in an event signal into a plurality of blocks. The user input processing method can be classified into N * M-dimensional matrix blocks according to the positions of a plurality of pixels. Here, N and M are positive integers and can be predetermined.

The user input processing method can select blocks including pixels of a predetermined threshold or more among a plurality of blocks. The user input processing method can determine that blocks including pixels having a number smaller than a predetermined threshold value include only noise, which is not meaningful information.

The user input processing method may obtain the pixels included in the selected blocks and perform steps 530, 540, and subsequent steps using the obtained pixels.

A user input processing method according to yet another embodiment may track a tip of an object in step 560. The user input processing method stores the tip of the recognized object through steps 510 to 550 and stores the tip of the newly recognized object according to the next event signal in the changed locus . ≪ / RTI >

Although not shown in the drawing, in the user input processing method according to another embodiment, the filtering technique may be applied so that the aspect of moving the tip of the object is smooth between steps 550 and 560. For example, the user input processing method can utilize a Kalman filter or the like.

6 is a flowchart illustrating a method of processing a user input for recognizing a selection input according to an exemplary embodiment of the present invention.

Referring to FIG. 6, in step 610, a user input processing method according to an exemplary embodiment may acquire an event signal according to an object motion. The user input processing method may calculate the scatter of the plurality of pixels included in the event signal in step 630. [ The user input processing method may switch to a mode of processing the selection input based on the scattering of the plurality of pixels in step 640. [ The user input processing method can generate a signal for a click operation in a mode for processing a selection input.

In addition, the user input processing method according to another embodiment may apply the quantization technique in order to reduce errors due to noise included in the event signal in step 620.

The steps described in FIG. 1 through FIG. 5 may be applied to each step shown in FIG. 6, so that a detailed description will be omitted.

7A to 7D are diagrams for explaining a technique of recognizing a tip of an object according to another embodiment.

Referring to FIG. 7A, a user input processing apparatus according to an embodiment may divide a plurality of pixels 710 included in an event signal into a predetermined number of sections 720. For example, a predetermined number of sections 720 may have the form of a 4 x 4 matrix.

Referring to FIG. 7B, a user input processing apparatus according to an embodiment may calculate feature vectors of a predetermined number of sections 720. FIG. The feature vector of any one of the predetermined number of sections 720 can be calculated based on the orientation vectors of the pixels included in the corresponding section.

The orientation vector of one pixel can be calculated using the surrounding pixels of the corresponding pixel. The x-axis variation and the y-axis variation can be calculated to calculate the orientation vector. For example, assuming that eight neighboring pixels located in the periphery of one pixel are used, the x-axis change amount can be increased when an event is detected in the right pixel, the upper right pixel, and the lower right pixel, respectively. Also, when the event is detected in the left pixel, the upper left pixel, and the lower left pixel, the amount of x-axis change can be reduced. In addition, the y-axis change amount can be increased when an event is detected in the upper pixel, the upper left pixel, and the upper right pixel, respectively. In addition, the y-axis change amount can be reduced when an event is detected in the lower pixel, the lower left pixel, and the lower right pixel, respectively.

The size of the orientation vector can be calculated as {(x-axis variation) ² + (y-axis variation) ² } ^0.5 . The direction of the orientation vector can be calculated as arctan {(y-axis variation) / (x-axis variation)}.

In one example, the user input processing device may vector the orientation vectors of the pixels included in the section 721 of Fig. 7A to calculate the feature vector 722 of the section 721. [ In this case, the feature vector 722 of the section 721 may be the result of vector summing the orientation vectors of the pixels included in the section 721 of FIG. 7A.

As another example, the user input processing device may extract the most dominant vector of the orientation vectors of the pixels included in the section 721 of Fig. 7A to calculate the feature vector 722 of the section 721 have. The most dominant vector is the orientation vector which occupies the largest weight among the plurality of orientation vectors included in the section. For example, the most dominant vector may be the largest number of orientation vectors among the plurality of orientation vectors included in the section. In this case, the feature vector 722 of the section 721 may be the most dominant vector extracted from the orientation vectors of the pixels included in the section 721 of FIG. 7A.

The user input processing device may quantize the orientation vectors to one of a predetermined number of quantization directions. For example, the user input processing apparatus can quantize the orientation vectors in one of 0 degrees, 45 degrees, 90 degrees, and 135 degrees by dividing 0 to 180 degrees into four equal parts.

The user input processing device can calculate the feature vectors of the sections based on the direction of the quantized orientation vectors. For example, the user input processing apparatus counts the number of pixels corresponding to each of the quantization directions among the pixels included in the section 721 of Fig. 7A to calculate the feature vector 722 of the section 721 . As a result of counting, one pixel may be zero-quantized pixels, four pixels may be quantized at 45 degrees, five pixels may be 135-degree quantized pixels, and 30 pixels may be quantized at 90 degrees.

In this case, the user input processing apparatus may generate a feature vector including the magnitude of the orientation vector corresponding to each of the quantization directions. For example, the size of the orientation vector corresponding to one quantization direction can be calculated by the number of pixels counted corresponding to the quantization direction. In this case, the feature vector 722 of the section 721 can be calculated as (1, 4, 5, 30). Here, (1, 4, 5, 30) is a vector including the number of pixels quantized in 0 degree, 45 degree, 90 degree, 135 degree in order. As another example, the size of the orientation vector corresponding to any one quantization direction can be calculated by dividing the number of pixels counted corresponding to the quantization direction by the total number of pixels included in the section. In this case, the feature vector 722 of the section 721 can be calculated as (1/40, 4/40, 5/40, 30/40). Here, (1/40, 4/40, 5/40, 30/40) denotes the result obtained by dividing the number of pixels quantized at 0 degree, 45 degree, 90 degree, 135 degree by the total number of pixels included in the section It is a vector containing.

The user input processing device can select the most dominant orientation vector based on the count result. The user input processing apparatus can select the orientation vector corresponding to the quantization direction having the largest number of counted pixels among the quantization directions. In this case, the feature vector 722 of section 721 may be an orientation vector corresponding to 90 degrees.

Referring to FIG. 7C, the user input processing apparatus according to an embodiment can recognize the tip 760 of the object based on the feature vectors 730. FIG. The user input processing device can obtain the section 761 corresponding to the tip of the object among the predetermined number of sections by inputting the learned parameter 740 and the feature vectors 730 to the classifier 750. [

The classification algorithm used in the classifier 750 may be a Support Vector Machine (SVM), a k nearest neighbor algorithm, an adaptive resonance theory, a deep neural network a deep neural network, and a multilayer perceptron.

7D, the learned parameter 740 according to one embodiment includes feature vectors 773 and 773 of a plurality of training images 771 and 772 and an object in a plurality of training images 771 and 772 And can be learned in advance based on the sections 775 and 776 corresponding to the tip of FIG.

For example, the user input processing apparatus according to an exemplary embodiment may detect a fingertip by recognizing a hand-shaped image. The user input processing device may divide the output screen of the event-based image sensor into a plurality of sections in order to recognize a hand-shaped image. The user input processing device can calculate the orientation vectors of the pixels included in each section. The user input processing device can learn the calculated orientation vectors and the section corresponding to the fingertip using a classification algorithm.

In the learning step, a training image including an event corresponding to the user's hand as well as an event corresponding to the background may be used. Using the learned parameters, the user input processing device can detect the finger end robustly to the background motion acting as noise. For example, when using an event-based image sensor mounted on a mobile device, it can be recognized that the mobile device itself moves as well as the user's hand as well as the background. The user input processing device can detect a user's fingertip in an event signal measured in a mobile device by using learned parameters in preparation for a mobile device environment.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (28)

A determination unit for determining whether the input image corresponds to a first mode input requesting tracking of the motion using an input image providing at least one point at which motion of the object is sensed; And
And when the input image corresponds to the first mode input,
To the user input device. The method according to claim 1,
Wherein the input image is an output image of a dynamic vision sensor that captures the object. The method according to claim 1,
The determination unit
Comparing the scatter of the at least one point with a threshold value and determining that the input image corresponds to the first mode input if the scatter is greater than or equal to the threshold value. The method according to claim 1,
The determination unit
Compares the scatter of the at least one point with a threshold value and determines that the input image corresponds to a second mode input indicating a selection input if the scatter is smaller than the threshold value. The method according to claim 3 or 4,
The determination unit
Wherein said at least one point is a scattering calculation unit for calculating a scattering in a predetermined direction,
To the user input device. The method according to claim 1,
The processing unit
A center of gravity calculation unit for calculating a center of gravity of the at least one point;
A line detector for detecting a dominant line associated with the at least one point; And
A tip detecting unit for detecting a tip of the object based on the center of gravity and the dominant line,
To the user input device. The method according to claim 6,
The line detector
A conversion unit for obtaining a plurality of lines by performing a hough transform on the at least one point; And
A line acquiring unit for acquiring the dominant line based on the plurality of lines;
To the user input device. The method according to claim 6,
The tip detecting unit
A projection unit for projecting the center of gravity to the dominant line; And
And a search unit for searching the tip along the dominant line in a predetermined direction from the projected position,
To the user input device. The method according to claim 1,
The processing unit
Calculating an orientation vector of the at least one point, dividing the at least one point into a predetermined number of sections, calculating feature vectors of the sections based on the orientation vector, And recognizes a tip of the object. 10. The method of claim 9,
The orientation of the orientation vectors being quantized in any one of a predetermined number of quantization directions. 10. The method of claim 9,
The processing unit
Acquires a section corresponding to a tip of the object among the predetermined number of sections by inputting a previously learned parameter and the feature vectors into a classifier. The method according to claim 1,
The determination unit
Determining whether the pattern of the at least one point corresponds to a predetermined pose and determining that the input image corresponds to a third mode input instructing to stop the tracking when the pattern corresponds to the predetermined pose User input processing device. 13. The method of claim 12,
The determination unit
A center of gravity calculation unit for calculating a center of gravity of the at least one point;
A pattern detector for detecting an area pattern of a point included in an area at least a predetermined distance away from the center of gravity of the at least one point in a predetermined direction; And
A confirmation unit for confirming whether or not the area pattern corresponds to the predetermined pose;
To the user input device. 13. The method of claim 12,
The determination unit
A line filter unit for extracting a component in a specific direction by applying a line filter technique to the at least one point;
A pattern detector for detecting a component pattern of the at least one point based on a component in the specific direction; And
A confirmation unit for confirming whether or not the component pattern corresponds to the predetermined pose;
To the user input device. Obtaining an event signal in which motion of an object is sensed;
Calculating a center of gravity of a plurality of pixels included in the event signal;
Extracting a dominant line included in the plurality of pixels; And
Recognizing a tip of the object based on the center of gravity and the dominant line
Lt; / RTI > 16. The method of claim 15,
The step of acquiring the event signal
Classifying the plurality of pixels into a plurality of blocks;
Selecting blocks comprising a predetermined number or more of the plurality of blocks; And
Acquiring pixels included in the selected blocks
Lt; / RTI > 16. The method of claim 15,
Calculating a disparity of a plurality of pixels included in the event signal;
Comparing the scatter to a threshold; And
And switching to a mode for processing a selection input when the scattering is smaller than the threshold value
Further comprising the steps of: 16. The method of claim 15,
Determining whether a pattern of the plurality of pixels corresponds to a predetermined pause corresponding to an interruption of the tracking mode; And
Switching to an interruption mode according to the determination result
Further comprising the steps of: Obtaining an event signal in which motion of an object is sensed;
Dividing a plurality of pixels included in the event signal into a predetermined number of sections;
Calculating orientation vectors of the plurality of pixels;
Calculating feature vectors of the sections based on the orientation vectors; And
Recognizing a tip of the object based on the feature vectors
Lt; / RTI > 20. The method of claim 19,
The step of calculating the orientation vectors
Quantizing the orientation of the orientation vectors into one of a predetermined number of quantization directions
Gt; user input < / RTI > 20. The method of claim 19,
The step of calculating the feature vectors
Calculating a vector sum of orientation vectors of pixels contained in each of the plurality of sections; And
Generating a feature vector of each of the plurality of sections based on the vector sum
Gt; user input < / RTI > 20. The method of claim 19,
If the direction of the orientation vectors is quantized to any one of a predetermined number of quantization directions,
The step of calculating the feature vectors
Counting the number of pixels corresponding to each of the quantization directions among the pixels included in each of the plurality of sections; And
Generating a feature vector of each of the plurality of sections based on the number of pixels counted corresponding to each of the quantization directions
Gt; user input < / RTI > 20. The method of claim 19,
The feature vector of each of the plurality of sections is
Wherein each of the elements is generated based on a number of pixels counted corresponding to each of the quantization directions. 20. The method of claim 19,
The feature vector of each of the plurality of sections is
Wherein the number of pixels counted among the quantization directions is generated based on an orientation vector corresponding to a largest quantization direction. 20. The method of claim 19,
The step of calculating the feature vectors
Extracting a dominant vector among orientation vectors of pixels included in each of the plurality of sections; And
Generating a feature vector of each of the plurality of sections based on the most dominant vector
Gt; user input < / RTI > 20. The method of claim 19,
The step of recognizing the tip of the object
Acquiring a section corresponding to a tip of the object among the predetermined number of sections by inputting the learned parameter and the feature vectors into a classifier
Gt; user input < / RTI > 20. The method of claim 19,
The learned parameter
And learning is performed in advance based on the feature vectors of the plurality of training images and the sections corresponding to the tips of the objects in the plurality of training images. A computer-readable recording medium having recorded thereon a program for executing the method according to any one of claims 15 to 27.

Images