KR101079270B1 - Three dimensional input device - Google Patents

Abstract

The three-dimensional input device of the present invention, the body portion to be held in the user's hand; Sensor unit for measuring the acceleration or angular velocity of the body portion; A controller for recognizing an operation pattern input by the user by processing the acceleration signal or the angular velocity signal received in the sensor unit; A haptic feedback unit installed in the body unit and generating a feedback signal for stimulating the user's tactile sense upon recognition of the operation pattern; It includes.

Description

THREE DIMENSIONAL INPUT DEVICE}

The present invention relates to a three-dimensional input device for interfacing user input in various multimedia environments.

Today, many attempts have been made to recognize not only numbers and letters but also user gestures as function keys in various multimedia environments such as computers, wireless communication, movies, IPTV, presentations, and the like.

For example, in the presentation environment, many advanced devices have been commercialized to facilitate the presentation process, but most of them assign some operations required for the presentation to the corresponding number of buttons, and each time the buttons are pressed. It's just a way to get it to work.

In addition, a method of linking the traces drawn on the screen by using a laser or other optical device such as a camera to recognize the laser pointer on the screen at the research level was introduced.

However, the above methods expose the following problems.

First, when a button is used for a related operation, the number of buttons also increases whenever the number of operations increases. 1 is a commercially available embodiment, which is equipped with a total of seven buttons. If the device is simply for presentation, the operation is simple, and the complexity caused by the increase in the number of buttons is not a big problem, but considering the digital fusion phenomenon in which several functions are integrated into one device, the increase in the number of buttons is disadvantageous for ease of use. Do.

Second, when the light point of the laser pointer is recognized by a camera or the like, and the trajectory is estimated and used for presentation manipulation, the proposed methodology has a disadvantage in that it depends only on excessive visual feedback. In other words, since vision-based interaction is essentially a vision system, it depends on the robust detection performance of laser light spots.

Third, when the camera tracking-based laser pointer system uses visual-based interactions, the laser light spot projected on the projection screen is not only a basic function called 'instruction', but also a visual result of abuse of visual information such as providing trajectory information for additional presentation manipulation. It causes an overload of information.

Fourth, it is worth mentioning that the trajectory information for presentation manipulation is unnecessary information for the audience sharing the screen. Smooth presentation by the presenter is essential for presentations in conferences or classes, but it must be taken into account that the presentation itself is conducted to help the audience understand and to seek consent from the audience. Visual feedback for presenters and moderators is only a misuse of unnecessary information for the audience.

The present invention is to improve the above-described problems, to provide a three-dimensional input device with improved intuitiveness and ease of use.

The technical problem to be achieved by the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

In one embodiment, the three-dimensional input device of the present invention, the body portion to be held in the user's hand; Sensor unit for measuring the acceleration or angular velocity of the body portion; A controller for recognizing an operation pattern input by the user by processing the acceleration signal or the angular velocity signal received in the sensor unit; A haptic feedback unit installed in the body unit and generating a feedback signal for stimulating the user's tactile sense upon recognition of the operation pattern; It includes.

In one embodiment, the three-dimensional input device of the present invention, the body portion having a sensor unit for receiving an operation pattern by the user; A control unit for searching which of the gestures stored in the lookup table corresponds to a gesture and recognizing the gesture as a number or a letter or performing a function key command mapped to the gesture; It includes.

In one embodiment, the three-dimensional input device of the present invention, a first axis accelerometer, a second axis accelerometer, a third axis accelerometer for measuring the linear acceleration for each of the virtual first axis, second axis, and third axis orthogonal to each other At least one of the axis accelerometer is provided in the body portion held by the user's hand, and the starting point for the motion pattern recognition in the linear acceleration signal output from at least one of the first axis accelerometer, the second axis accelerometer, the third axis accelerometer and Extracting an end point, selecting a pole as a feature point among the linear acceleration signals between the start point and the end point, and determining whether the input operation pattern corresponds to a gesture stored in a lookup table based on the feature point; Perform a function key command recognized as a character or mapped to the gesture, and stimulate the user's tactile sensation upon recognition of the operation pattern And it generates a feedback signal.

In one embodiment, the three-dimensional input device of the present invention, a first axis accelerometer, a second axis accelerometer, a third axis accelerometer for measuring the linear acceleration for each of the virtual first axis, second axis, and third axis orthogonal to each other At least one of the axis accelerometer is provided in the body portion, the linear acceleration signal output from at least one of the first axis accelerometer, the second axis accelerometer, the third axis accelerometer and the linear acceleration signal integrating the linear acceleration signal once And a preprocessing step of passing a position signal obtained by integrating the linear acceleration signal twice through a low pass filter, and determining a starting point and an end point of the signals according to an on / off state of a button provided in the body part. Determining which one of the gestures stored in the look-up table corresponds to the operation pattern input through the body portion based on the recognition of the gesture as a number or a letter or A function key command mapped to the gesture is performed and a feedback signal for stimulating the user's tactile sensation is generated upon recognition of the operation pattern.

In one embodiment, the three-dimensional input device of the present invention, at least one of the first axis, the second axis, and the third axis of the first axis, the second axis accelerometer, the third axis of the rotational tachometer relative to each other orthogonal to each other; One is provided in the body portion, the start point and the end point of the signal output from at least one of the first axis accelerometer, the second axis accelerometer, the third axis angular accelerometer according to the on / off state of the button provided in the body portion And extracting a feature point by processing the signals with a fast fourier transform (FFT) or a discrete cosine transform (DCT), and determining which of the gestures stored in the lookup table corresponds to the input operation pattern based on the feature point. Recognize a gesture as a number or letter or perform a function key command mapped to the gesture, and generate a feedback signal that stimulates the user's tactile sense when the operation pattern is recognized. do.

According to the present invention, a gesture taken by a user can be recognized as a number or a letter, a function key command mapped to various gestures can be performed, and if a gesture pattern is properly recognized, a feedback signal for stimulating a user's tactile sense is generated. As a result, the gesture recognition process may be fed back to the user without disturbing the audience's attention during the presentation.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this process, the size or shape of the components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms that are specifically defined in consideration of the configuration and operation of the present invention may vary depending on the intention or custom of the user or operator. Definitions of these terms should be made based on the contents throughout the specification.

2 is a block diagram showing the configuration of the three-dimensional input device of the present invention.

The 3D input device 100 of the present invention includes a body portion 101, a sensor portion 110, a controller 190, and a haptic feedback unit 130. The body 101 is a part held by a user's hand and is a means for inputting a user's movement for recognizing an operation pattern.

The sensor unit 110 measures an acceleration or an angular velocity of the body 101, and a three-axis accelerometer measuring linear acceleration or rotational angular velocity about each of the virtual first, second and third axes that are orthogonal to each other. And at least one of a three-axis angometer, a gyroscope, and an inertial navigation device. For example, a 3-axis accelerometer may include a first axis accelerometer measuring linear acceleration on a first axis, a second axis accelerometer measuring linear acceleration on a second axis, and a third axis measuring linear acceleration on a third axis. It has an accelerometer.

Although the controller 190 is provided in the server 180 and is connected to the 3D input apparatus 100 through wireless communication, as illustrated, the controller 190 may be provided inside the body portion 101 in an embodiment not shown. The controller 190 recognizes the operation pattern input by the user by processing the acceleration signal or the angular velocity signal received by the sensor unit 110.

That is, the controller 190 searches for which of the gestures stored in the look-up table corresponds to a gesture, and thus recognizes the detected gesture as a number or a letter or maps the gesture to the gesture. Execute a function key command. Examples of the function keys include a left and right scroll key, an enter key, a space key, a page up / page down key, and the like.

In an embodiment, the controller 190 detects a start point and an end point of an operation pattern from data processing an acceleration signal or an angular velocity signal, extracts feature points from an acceleration signal or an angular velocity signal between the start point and the end point, and correlates the feature points. The relationship is recognized using the relationship. Here, the feature points are preferably extreme values of the acceleration signal or the angular velocity signal.

As another embodiment for recognizing the starting point and the end point, the body portion 101 includes a button which is pressed at the input of the operation pattern and released at the end of the operation pattern to inform the start and end of the operation pattern.

In this embodiment, the body portion 101 is a first button 121 for blinking the laser pointer light at the time of presentation on the screen 10, the start of the operation pattern at the time of presentation At least one of the second button 122 that is pressed when the operation pattern is input to indicate the end and the end. In addition, a plurality of buttons corresponding to remote control functions of various information home appliances such as a computer, a telephone IP TV, and a TV may be provided so that the 3D input device 100 of the present invention can be used for an IP TV or other multimedia equipment. .

The haptic feedback unit 130 is installed in the body unit 101 and generates a feedback signal for stimulating the user's tactile sense when the motion pattern is recognized. That is, when a gesture corresponding to the operation pattern is searched in the lookup table and the mapping of numbers, letters, and function keys succeeds, a feedback signal for stimulating the user's touch is generated to inform the user that the correct gesture is input.

Accordingly, the user is not limited to visual information or audio information, whether the user inputs the correct motion or the controller 190 correctly recognizes the motion pattern when the 3D input device is used. It can be confirmed tactilely through.

The feedback signal transmitted to the user for the purpose of confirming the gesture input during the presentation focusing the attention of the audience is not preferable because it distracts the audience's attention when transmitted as a visual signal. The same is true for the acoustic signal. Thus, the feedback signal is transmitted as a feedback signal through the tactile touch of the hand so that only the user can recognize the distraction without disturbing the audience's attention.

In one embodiment, the haptic feedback unit 130 stimulates at least one of the user's Pacinian Corpuscle, Meissner's Corpuscle, Merkel's Disc, and Ruffini's Ending. Thereby stimulating the user's sense of touch.

To this end, the haptic feedback unit 130 includes at least one of a piezo actuator, a solenoid actuator, an ultrasonic actuator, an electrically active polymer actuator, and a vibration motor.

3 and 8, the configuration and operation of the three-dimensional input device of the present invention will be described in more detail.

3 is a flowchart showing the operating principle of the three-dimensional input device of the present invention. Linear acceleration A _b as raw data 210 input to the sensor unit 110, such as an inertial navigation device, measured primarily by the inertial navigation device. = [A _bx , A _by , A _bz ] ^T Rotational angular velocity with ω _b = [ω _bx , ω _by , ω _bz ] The linear velocity V _b , which can be the first measured value, such as ^T , is calculated and processed secondarily = [V _bx , V _by , V _bz ] ^T or angle of rotation R _b = [R _bx , R _by , R _bz ] ^T , Or derivative of linear acceleration / rotational angular velocity, position information P _b = [P _bx , P _by , P _bz ] There may be secondary measurements such as ^T.

In FIG. 3, the part labeled 'Qualitative' refers to a case in which a motion recognition is attempted by using the first measurement value or the second measurement value qualitatively, and the 'Quantitative' is estimated through the first measurement value. This refers to a case in which motion recognition is attempted quantitatively using location information.

Since the first measurement is already distorted by noise, a separate methodology for removing the noise is needed to obtain the second measurement. If the linear acceleration A _b and the rotational angular velocity ω _b are used, the position information that is one of the second measured values, that is, the trajectory estimation process 270 is performed, and the motion recognition is performed using the result. In this case, the method of applying the methodology of pattern recognition, which is resistant to errors such as noise, to the first measured value itself is more preferable because limitations such as accuracy occur.

As one of such methods, a method such as low pass filtering that removes noise included in a signal input as raw data in the pre-processing process 220 or restores a distorted signal is mobilized. Unnecessary data that does not affect the recognition result in the feature extraction process 240 that separates the signal input in the end point detection process 230 into a signal meaningful and meaningless in the recognition process To eliminate only the components that have a high impact on recognition results.

As a result, the amount of time-series data input is compressed, processing speed is increased, or recognition rate is improved. Which one of the above-mentioned primary and secondary measurements can be used depends on the type of operation to be recognized, the recognition accuracy required, or the processing speed.

An example of using the above-described primary measurement value is a case where a number written in a space is recognized by an information input device having three three-axis accelerometers installed in a three-dimensional input device. FIG. 4 is a graph with respect to this, which illustrates one embodiment of extracting a starting point and an end point feature point for an acceleration signal.

Since the number gesture can be recognized on a plane, for convenience of explanation, the first axis accelerometer and the second axis accelerometer are linear acceleration A _b among three accelerometers. = [A _bx , A _by ] Only ^T is used as raw data and recognizer uses Dynamic Time Warping (DTW).

The DTW recognition method can be described as follows. Patterns that are subject to pattern recognition are divided into static and dynamic patterns. Fingerprints, irises in general. Fixed images, such as letters, correspond to static patterns, and time-series patterns that change over time, such as speech and stock prices, correspond to movement patterns. The DTW algorithm can be easily applied to time series patterns such as voice and acceleration.

The most basic way we can intuitively think about pattern recognition is pattern matching. In the case of the static pattern, the pattern matching method is relatively easy, but in the case of the dynamic pattern, the pattern matching is required in a manner that allows the expansion and contraction of the pattern in the time domain. For example, if you pronounce the same word "to go", there are people who speak fast and speak slowly, but in both cases you must recognize that you are "go". The DTW algorithm enables pattern matching for two patterns of different lengths.

The definition of the DTW algorithm is a pattern matching algorithm that allows nonlinear stretching on the time axis. In order to compare two columns based on one column in two columns of different lengths, either column must be stretched or shrunk. When a long column is compared based on a shorter column, a matching function is used. When the matching function has linearity, it is referred to as a 'linear stretch comparison' and a nonlinear 'non-linear stretch comparison'.

The DTW algorithm sets the cost of the distance measure for each component in two rows. In addition, it is an algorithm that compares two columns while searching for a matching function by using an ignition equation that uses the minimum cost in the cost table starting from the starting component of each column and ending with the last component on the grid formed by the two columns.

Finally, the cost value stored in the cost table at the end component is the similarity for the two columns. The trajectory of the mapping function is found by storing the path that holds the minimum cost in each step in a separate path table, such as finding the optimal search path of dynamic programming, and backtracking after finding the final minimum cost from the end component.

Meanwhile, referring to FIG. 4, the end point detection and the feature point extraction specialized for the linear acceleration signal are applied and described. The hollow mark shown in FIG. 4 is an extract of the pole of the acceleration signal, all of which can be used as feature point candidates to express the signal compressively, but the solid mark found in a significant section (between the start point and the end point). Only (X1, X2, X3, X4) becomes a feature point used for motion recognition.

If the solid display points (X1, X2, X3, X4) are stored in the lookup table as a typical pattern of acceleration signals corresponding to the number '2', then the controller 190 looks at the feature points of FIG. It is determined that there is a gesture input.

5 is a diagram illustrating various gestures input by the three-dimensional input device of the present invention. With reference to this, the experimental results of the gesture recognition accuracy of the three-dimensional input device of the present invention will be described.

For the experiment, the recognition rate was verified and compared with 1680 gesture data from 12 persons, 6 men and women in their 20s and 40s, and the recognition results are summarized in the following table.

gesture Recognition rate (%) 0 64.17 One 95 2 93.33 3 95.83 4 97.5 5 85.83 6 69.17 7 97.5 8 92.5 9 99.17 Ch Right 96.67 Ch left 96.67 Delete 95 Enter 100 Total 91.31

In summary, the start point and the end point for motion pattern recognition are extracted from a linear acceleration signal output from at least one of a first axis accelerometer, a second axis accelerometer, and a third axis accelerometer, and the linear point between the starting point and the end point is extracted. A pole is selected as a feature point among the acceleration signals, and based on the feature point, it is determined whether the input operation pattern corresponds to a gesture stored in a lookup table, and the gesture is recognized as a number or a letter or a function key command mapped to the gesture is used. The controller generates a feedback signal for stimulating the user's tactile sense upon recognition of the operation pattern.

As another embodiment, it is possible to mix the primary and secondary measurements and use them as raw data. 6 is a velocity signal integrating an acceleration signal when recognizing the operation of the numbers '5', '6', '7', '8', '9' using the three-dimensional input device of the present invention, and twice It is a graph showing the integrated position signal.

That is, the linear acceleration A _b measured by the first axis accelerometer, the second axis accelerometer, and the third axis accelerometer = [A _bx , A _by , A _bz ] ^T And linear velocity V _{b that is} integrated once = [V _bx , V _by , V _bz ] ^T , two integrated positions P _b = [P _bx , P _by , P _bz Using ^T as raw data, pre-processing process such as low pass filter and simple process of starting and end point detection process using buttons attached to the outside of body 101 The separate feature point extraction is not applied. The recognizer recognizes numbers written in space using Dynamic Time Warping (DTW) and Hidden Markov Model (HMM).

Hold the 3D input device and write the number 5 times in 3D space and measure the acceleration. FIG. 6 shows a velocity signal obtained by firstly integrating an acceleration signal obtained when the numbers 5, 6, 7, 8, and 9 are written, and a position signal by second integral integration.

When only the acceleration signal is a feature (A), when only the speed signal is a feature (V), when only the position signal is a feature (P), the acceleration signal and the speed signal are both characterized The recognition rate obtained by distinguishing each of the case of using the feature (AV), the case of using the acceleration signal, the speed signal, and the position signal as the feature (AVP), is given as shown in Table 2 below. Under-Sampling Rate refers to the change in the signal acquisition period used to read the signal. As a reference, 1kHz sampling rate is used and it is processed by changing it into 200Hz, 100Hz, 66Hz and so on.

In summary, the linear acceleration signal output from at least one of a first axis accelerometer, a second axis accelerometer, and a third axis accelerometer, a linear velocity signal obtained by integrating the linear acceleration signal once, and the linear acceleration signal Through the body portion based on the pre-processing step of passing the integrated position signal to the low pass filter, and the process of determining the start point and the end point of the signals according to the on / off state of the button provided in the body portion Determining which of the gestures stored in the look-up table corresponds to a gesture and recognizing the gesture as a number or letter or performing a function key command mapped to the gesture, and stimulating the user's tactile sensation upon recognition of the gesture pattern To generate a feedback signal.

The last embodiment is a case in which motion recognition is attempted by using both linear acceleration and rotational angular velocity among the first measured values. FIG. 7 is a graph illustrating a sample of signals output from a first axis accelerometer, a second axis accelerometer, and a third axis rotational oximeter with respect to the three-dimensional input device of the present invention.

It is a case used for the purpose of recognizing numbers or letters drawn in space as in the above-described embodiment, and it is a linear data of two axes measured by the first axis accelerometer and the second axis accelerometer as raw data A _b = [A _bx , A _by ] ^T and the angular velocity measured on the 3rd axis tachometer ω _b = [ω _bz ] ^T is used.

The start and end of the motion are detected using the buttons attached to the outside of the 3D input device, and Fast Fourier Transform (FFT) and Discrete Cosine Transform (DCT) are applied to each axis information for feature point extraction.

FIG. 8 is a graph obtained by FFT processing of FIG. 7.

FFT and DCT are calculated using the following equations, of which X _1fft ~ X _20fft and X _1dct ~ X _20dct are used as feature points.

Here, N is the magnitude of the coefficient generated after the FFT and DCT operation, and X _kfft and X _kdct represent the kth coefficient. Self-Organizing Map (SOM) was used as a recognizer, and the results of using FFT and DCT as feature points are given in the table below.

In summary, the start point and the end point of the signal output from at least one of the first axis accelerometer, the second axis accelerometer, and the third axis tachometer are set to the on / off state of the button provided in the body portion 101. And extracting a feature point by processing the signals with a fast fourier transform (FFT) or a discrete cosine transform (DCT), and determining which of the gestures stored in the lookup table corresponds to the input operation pattern based on the feature point. The gesture may be recognized as a number or a letter or a function key command mapped to the gesture may be generated, and a feedback signal may be generated to stimulate the user's tactile sense when the operation pattern is recognized.

Next, an embodiment in which the three-dimensional input device of the present invention is specialized for presentation progress will be described.

The three-dimensional input device of the present invention includes at least one of a three-axis accelerometer, a three-axis angometer, a gyroscope, an inertial navigation device as the sensor unit 110 to recognize a gesture by motion-based interaction. Attempts to use gesture recognition can functionally execute multiple functions with a single button and mitigate visual bias by utilizing various modalities in addition to conventional vision.

The hardware is designed to resemble a pen, a user-friendly medium to minimize the number of buttons (two), maximize intuition and minimize rejection.

It consists of two buttons and the basic function assignments are as follows. The first button 121 (hereinafter referred to as SW1) is responsible for the flashing of the laser pointer 140. When the first button 121 is pressed, the laser pointer 140 is turned on. The second button 122 (hereinafter referred to as SW2) is used to distinguish between an operation for everyday operation and a presentation operation. When a predetermined operation is executed while pressing the second button 122, the gesture is recognized and recognized. The function mapped to the pattern is performed.

The operation of the presentation and the embodiment of the function mapped for this are described in the table below.

The operation pattern used to execute the functions shown in Table 4 was selected with intuition as the top priority. The beginning and end of the slide are performed by drawing '○' and '×' in space.

In general, when a cursor disappears in a slide show such as PowerPoint, the user focuses on 'waking up' the cursor by moving the mouse from side to side to perform the same function by shaking in a constant direction. Designed.

In the slide show, we prepared a circle drawing to the right (clockwise rotation) and a circle drawing to the left (counterclockwise rotation) for 'turn to next chapter' or 'return to the previous chapter'. This operation method is friendly to the user and reduces the fatigue of the wrist, and it is easy to apply to the continuous operation of turning multiple pages.

Although embodiments according to the present invention have been described above, these are merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent embodiments of the present invention are possible therefrom. Therefore, the true technical protection scope of the present invention will be defined by the following claims.

1 is a photograph showing a conventional remote controller for presentation.

2 is a block diagram showing the configuration of the three-dimensional input device of the present invention.

3 is a flowchart showing the operating principle of the three-dimensional input device of the present invention.

4 is a graph illustrating an embodiment of extracting a starting point and an end point feature point for an acceleration signal in the 3D input device of the present invention.

5 is a diagram illustrating various gestures input by the three-dimensional input device of the present invention.

6 is a velocity signal integrating an acceleration signal when recognizing the operation of the numbers '5', '6', '7', '8', '9' using the three-dimensional input device of the present invention, and twice It is a graph showing the integrated position signal.

FIG. 7 is a graph illustrating a sample of signals output from a first axis accelerometer, a second axis accelerometer, and a third axis rotational oximeter with respect to the three-dimensional input device of the present invention.

FIG. 8 is a graph obtained by FFT processing of FIG. 7.

<Explanation of symbols for the main parts of the drawings>

10 ... screen 100 ... 3D input device

101 Body 110 Sensor

121 ... First button 122 ... Second button

130 Haptic feedback 140 Laser pointer

150 Interface unit 180 Server

190.Control panel

Claims (14)

A body portion held in a user's hand; Sensor unit for measuring the acceleration or angular velocity of the body portion; A controller for recognizing an operation pattern input by the user by processing the acceleration signal or the angular velocity signal received in the sensor unit; A haptic feedback unit installed in the body unit and generating a feedback signal for stimulating the user's tactile sense upon recognition of the operation pattern; Three-dimensional input device comprising a. The method of claim 1, The haptic feedback unit stimulates the user's tactile sense by stimulating at least one of the user's Pacinian Corpuscle, Meissner's Corpuscle, Merkel's Disc, and Ruffini's Ending. 3D input device. The method of claim 1, The haptic feedback unit includes a vibration module including at least one of a piezo actuator, a solenoid actuator, an ultrasonic actuator, an electrically active polymer actuator, and a vibration motor. The method of claim 1, The controller is provided in the server spaced apart from the body portion, The haptic feedback unit and the control unit is a three-dimensional input device connected by wireless communication. The method of claim 1, The controller detects a start point and an end point of the operation pattern from the acceleration signal or the angular velocity signal, extracts feature points from the acceleration signal or angular velocity signal between the start point and the end point, and uses the correlation between the feature points. Three-dimensional input device that recognizes the movement pattern. The method of claim 5, The feature points are extreme values of the acceleration signal or the angular velocity signal. The method of claim 1, The body portion, A first button for blinking the pointing light at the presentation; And at least one of a second button pressed when the operation pattern is input to notify the start and end of the operation pattern during the presentation. The method of claim 1, The body portion has a three-dimensional input device having a plurality of buttons corresponding to the remote control function of the home appliance. The method of claim 1, The sensor unit includes at least one of a three-axis accelerometer, a three-axis angometer, a gyroscope, and an inertial navigation device that measures linear acceleration or rotational angular velocity about each of the virtual first, second, and third axes that are orthogonal to each other. 3D input device. A body part having a sensor part configured to receive an operation pattern by a user; A control unit for searching which of the gestures stored in the lookup table corresponds to a gesture and recognizing the gesture as a number or a letter or performing a function key command mapped to the gesture; Including, The body unit includes a haptic feedback unit for generating a feedback signal for stimulating the user's haptic when searching for the gesture corresponding to the operation pattern. delete Body portion in which at least one of a first axis accelerometer, a second axis accelerometer, and a third axis accelerometer that measures linear acceleration about each of the virtual first, second, and third axes that are orthogonal to each other is held in the user's hand Being laid in, Extracting a start point and an end point for motion pattern recognition from a linear acceleration signal output from at least one of the first axis accelerometer, the second axis accelerometer, and the third axis accelerometer, Selecting a pole as a feature point among the linear acceleration signals between the start point and the end point, Determining whether the input operation pattern corresponds to a gesture stored in a lookup table based on the feature point, recognizing the gesture as a number or a letter, or performing a function key command mapped to the gesture; 3D input device for generating a feedback signal to stimulate the user's sense of touch upon the recognition of the operation pattern. Body portion in which at least one of a first axis accelerometer, a second axis accelerometer, and a third axis accelerometer that measures linear acceleration about each of the virtual first, second, and third axes that are orthogonal to each other is held in the user's hand Being laid in, A position obtained by integrating the linear acceleration signal output from at least one of the first axis accelerometer, the second axis accelerometer, and the third axis accelerometer, the linear velocity signal integrating the linear acceleration signal once, and the linear acceleration signal integrating twice The operation pattern input through the body part is looked up based on a preprocessing process of passing a signal through a low pass filter and a process of determining a start point and an end point of the signals according to on / off states of buttons provided in the body part. Determine which of the gestures is stored in a table, recognize the gesture as a number or letter, or perform a function key command mapped to the gesture, 3D input device for generating a feedback signal to stimulate the user's sense of touch upon the recognition of the operation pattern. At least one of a first axis accelerometer, a second axis accelerometer, and a third axis rotational tachometer with respect to an imaginary first axis, a second axis, and a third axis that are orthogonal to each other is provided in a body part which is held by a user's hand, A start point and an end point of a signal output from at least one of the first axis accelerometer, the second axis accelerometer, and the third axis tachometer are determined according to the on / off state of the button provided in the body part. The signal is processed by fast Fourier transform (FFT) or discrete cosine transform (DCT) to extract feature points, Determine which of the gestures stored in the lookup table corresponds to a gesture pattern based on the feature point, recognize the gesture as a number or letter, or perform a function key command mapped to the gesture; 3D input device for generating a feedback signal to stimulate the user's sense of touch upon the recognition of the operation pattern.

Images